Saurabh Srivastava
ABSTRACT
The temptation to be able to talk to a machine is not new. Recent advancements in the field of Natural Language Understanding has made it possible to build conversational components that can be plugged inside an application, similar to other components. These components, called chatbots, can be created from scratch or with the help of commercially available platforms. These platforms make it easier to build and deploy chatbots, often without writing a single line of code. However, similar to any other software component, chatbots also have quality concerns. Despite significant contributions in the field, an architectural perspective of building chatbots with desired quality requirements is missing in the literature.
In the current work, we highlight the impact of features provided by these platforms (along with their quality) on the application design process and overall quality attributes. We propose a methodological framework to evaluate support provided by a chatbot platform towards achieving quality in the application. The framework, called Hospitality Framework, is based on software architectural body of knowledge, especially architectural tactics. The framework produces a metric, called Hospitality Index, which has utilities for making various design decisions for the overall application. We present the use of our framework on a simple use case to highlight the phases of evaluation. We showcase the process by picking three popular chatbot platforms - Watson Assistant, DialogFlow and Lex, over four quality attributes - Modifiability, Security & Privacy, Interoperability and Reliability. Our results show that different platforms provide different support for these four quality attributes.
- {n.d.}. Amazon Lex – Build Conversation Bots. https://aws.amazon.com/lex/. Accessed: 2019-05-30.Google Scholar
- {n.d.}. Blog - Your chatbot and mobile messaging resource - ubisend. https: //blog.ubisend.com/. Accessed: 2019-05-30.Google Scholar
- {n.d.}. Chatbots.org. https://www.chatbots.org/. Accessed: 2019-05-30.Google Scholar
- {n.d.}. Dialogflow. https://dialogflow.com/. Accessed: 2019-05-30.Google Scholar
- {n.d.}. Docs - Dialogflow. https://dialogflow.com/docs. Accessed: 2019-05-30.Google Scholar
- {n.d.}. IBM Cloud Functions. https://cloud.ibm.com/openwhisk/. Accessed: 2019-05-30.Google Scholar
- {n.d.}. LUIS (Language Understanding) – Cognitive Services – Microsoft Azure. https://www.luis.ai/home. Accessed: 2019-05-30.Google Scholar
- {n.d.}. PHP: cGoogle Scholar
- {n.d.}. RASA. https://rasa.com. Accessed: 2019-05-30.Google Scholar
- {n.d.}. Stack Overflow - Where Developers Learn, Share, & Build Careers. https: //stackoverflow.com/. Accessed: 2019-05-30.Google Scholar
- {n.d.}. Watson Assistant - IBM Cloud. https://www.ibm.com/cloud/watsonassistant/. Accessed: 2019-05-30.Google Scholar
- {n.d.}. Watson Speech to Text. https://www.ibm.com/watson/services/speechto-text/. Accessed: 2019-05-30.Google Scholar
- Ashish Agrawal and TV Prabhakar. 2013. Hospitality of cloud platforms. In Proceedings of the 28th Annual ACM Symposium on Applied Computing. ACM, 389–396. Google ScholarDigital Library
- Felix Bachmann, Len Bass, and Robert Nord. 2007. Modifiability tactics. Technical Report. CARNEGIE-MELLON UNIV PITTSBURGH PA SOFTWARE ENGINEERING INST.Google Scholar
- Len Bass, Paul Clements, and Rick Kazman. 2013. Software Architecture in Practice.Google Scholar
- Valerie Belton and Theodor Stewart. 2002. Multiple criteria decision analysis: an integrated approach. Springer Science & Business Media.Google Scholar
- Rene Berger, Markus Ebner, and Martin Ebner. 2019. Conception of a Conversational Interface to Provide a Guided Search of Study Related Data. International Journal of Emerging Technologies in Learning 14, 7 (2019).Google ScholarCross Ref
- Pierre Bourque, Richard E Fairley, et al. 2014. Guide to the software engineering body of knowledge (SWEBOK (R)): Version 3.0. IEEE Computer Society Press. Google ScholarDigital Library
- Daniel Braun, Adrian Hernandez-Mendez, Florian Matthes, and Manfred Langen. 2017. Evaluating Natural Language Understanding Services for Conversational Question Answering Systems. In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue. Association for Computational Linguistics, Saarbrücken, Germany, 174–185.Google ScholarCross Ref
- D. Braun and F. Matthes. 2019. Towards a Framework for Classifying Chatbots. In Proceedings of the 21th International Conference on Enterprise Information Systems - Volume 1: ICEIS.Google Scholar
- Frank Buschmann, Regine Meunier, Hans Rohnert, Peter Sommerlad, and Michael Stal. 1996. Pattern oriented software architecture: A system of patterns. Hoboken. Google ScholarDigital Library
- Massimo Canonico and Luigi De Russis. 2018. A comparison and critique of natural language understanding tools. In Cloud Computing 2018. 110–115.Google Scholar
- Robert Dale. 2016. The return of the chatbots. Natural Language Engineering 22, 5 (2016), 811–817.Google ScholarCross Ref
- Laurent Deruelle, Mustapha Derras, Jordi Cabot, and Gwendal Daniel. 2019. Multi-Platform Chatbot Modeling and Deployment with the Jarvis Framework.Google Scholar
- Ching-Lai Hwang and Kwangsun Yoon. 2012. Multiple attribute decision making: methods and applications a state-of-the-art survey. Vol. 186. Springer Science & Business Media.Google Scholar
- ISO/IEC. 2010. ISO/IEC 25010 System and software quality models. Technical Report.Google Scholar
- Rick Kazman, Len Bass, Gregory Abowd, and Mike Webb. 1994. SAAM: A method for analyzing the properties of software architectures. In Proceedings of 16th International Conference on Software Engineering. IEEE, 81–90. Google ScholarDigital Library
- Rick Kazman, Mark Klein, Mario Barbacci, Tom Longstaff, Howard Lipson, and Jeromy Carriere. 1998. The architecture tradeoff analysis method. In Proceedings. Fourth IEEE International Conference on Engineering of Complex Computer Systems (Cat. No. 98EX193). IEEE, 68–78.Google ScholarCross Ref
- Xingkun Liu, Arash Eshghi, Pawel Swietojanski, and Verena Rieser. 2019. Benchmarking Natural Language Understanding Services for building Conversational Agents. arXiv e-prints, Article arXiv:1903.05566 (Mar 2019), arXiv:1903.05566 pages. arXiv: cs.CL/1903.05566Google Scholar
- Dijana Peras. {n.d.}. Chatbot evaluation metrics. In Economic and Social Development: Book of Proceedings, 89-97.Google Scholar
- Philip Resnik and Jimmy Lin. 2010. Evaluation of NLP Systems. John Wiley & Sons, Ltd, Chapter 11, 271–295.Google Scholar
Index Terms
- Hospitality of chatbot building platforms
Recommendations
Contextual Reactive Pattern on Chatbot building Platforms
EuroPLoP '20: Proceedings of the European Conference on Pattern Languages of Programs 2020Building a chatbot with an iterative development process poses certain challenges for the chatbot developer. The developer is expected to produce a deployable version of the chatbot at the end of a short development cycle. Every iteration should ...
Unveiling Quality in Chatbot Conversations: Quantitative Analysis of Chatbot Requirements
SBQS '23: Proceedings of the XXII Brazilian Symposium on Software QualityAs conversational assistants and natural language interfaces proliferate, the demand for a precise understanding of quality software requirements for chatbots becomes increasingly critical. In this work, we adopted a quantitative methodology, ...
Comments