ABSTRACT
Conversational agents stand to play an important role in supporting behavior change and well-being in many domains. With users able to interact with conversational agents through both text and voice, understanding how designing for these channels supports behavior change is important. To begin answering this question, we designed a conversational agent for the workplace that supports workers' activity journaling and self-learning through reflection. Our agent, named Robota, combines chat-based communication as a Slack Bot and voice interaction through a personal device using a custom Amazon Alexa Skill. Through a 3-week controlled deployment, we examine how voice-based and chat-based interaction affect workers' reflection and support self-learning. We demonstrate that, while many current technical limitations exist, adding dedicated mobile voice interaction separate from the already busy chat modality may further enable users to step back and reflect on their work. We conclude with discussion of the implications of our findings to design of workplace self-tracking systems specifically and to behavior-change systems in general.
Supplemental Material
- Elena Agapie, Daniel Avrahami, and Jennifer Marlow. 2016. Staying the Course: System-Driven Lapse Management for Supporting Behavior Change. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 1072--1083. Google ScholarDigital Library
- Rusul Alrubail. Scaffolding Student Reflections + Sample Questions. Edutopia. Retrieved January 8, 2018 from https://www.edutopia.org/discussion/scaffoldingstudent-reflections-sample-questionsGoogle Scholar
- Teresa M. Amabile and Steven J. Kramer. 2011. The power of small wins. Harvard Business Review 89, 5: 70--80.Google Scholar
- Frederik Anseel, Filip Lievens, and Eveline Schollaert. 2009. Reflection as a strategy to enhance task performance after feedback. Organizational Behavior and Human Decision Processes 110, 1: 23--35.Google ScholarCross Ref
- Daniel Avrahami, Mitesh Patel, Yusuke Yamaura, and Sven Kratz. 2018. Below the Surface: Unobtrusive Activity Recognition for Work Surfaces Using RFradar Sensing. In 23rd International Conference on Intelligent User Interfaces (IUI '18), 439--451. Google ScholarDigital Library
- John D. Bain, Roy Ballantyne, Jan Packer, and Colleen Mills. 1999. Using journal writing to enhance student teachers' reflectivity during field experience placements. Teachers and Teaching 5, 1: 51--73.Google ScholarCross Ref
- Yehuda Baruch. 2003. Career systems in transition: A normative model for organizational career practices. Personnel review 32, 2: 231--251.Google Scholar
- Eric P.S. Baumer, Vera Khovanskaya, Mark Matthews, Lindsay Reynolds, Victoria Schwanda Sosik, and Geri Gay. 2014. Reviewing Reflection: On the Use of Reflection in Interactive System Design. In Proceedings of the 2014 Conference on Designing Interactive Systems (DIS '14), 93--102. Google ScholarDigital Library
- Timothy Bickmore, Daniel Mauer, Francisco Crespo, and Thomas Brown. 2007. Persuasion, task interruption and health regimen adherence. In International Conference on Persuasive Technology, 1--11. Google ScholarDigital Library
- David Boud, Rosemary Keogh, and David Walker. 2013. Reflection: Turning experience into learning. Routledge.Google ScholarCross Ref
- Matthew Carrasco, Eunyee Koh, and Sana Malik. 2017. popHistory: Animated Visualization of Personal Web Browsing History. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '17), 2429-- 2436. Google ScholarDigital Library
- Sunny Consolvo, Predrag Klasnja, David W. McDonald, Daniel Avrahami, Jon Froehlich, Louis LeGrand, Ryan Libby, Keith Mosher, and James A. Landay. 2008. Flowers or a robot army?: encouraging awareness & activity with personal, mobile displays. In Proceedings of the 10th international conference on Ubiquitous computing, 54--63. Google ScholarDigital Library
- Sunny Consolvo, David W. McDonald, and James A. Landay. 2009. Theory-driven Design Strategies for Technologies That Support Behavior Change in Everyday Life. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09), 405--414. Google ScholarDigital Library
- Felicia Cordeiro, Elizabeth Bales, Erin Cherry, and James Fogarty. 2015. Rethinking the mobile food journal: Exploring opportunities for lightweight photobased capture. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 3207--3216. Google ScholarDigital Library
- Felicia Cordeiro, Daniel A. Epstein, Edison Thomaz, Elizabeth Bales, Arvind K. Jagannathan, Gregory D. Abowd, and James Fogarty. 2015. Barriers and negative nudges: Exploring challenges in food journaling. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 1159--1162. Google ScholarDigital Library
- Justin Cranshaw, Emad Elwany, Todd Newman, Rafal Kocielnik, Bowen Yu, Sandeep Soni, Jaime Teevan, and Andrés Monroy-Hernández. 2017. Calendar. help: Designing a Workflow-Based Scheduling Agent with Humans in the Loop. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 2382--2393. Google ScholarDigital Library
- Beant Dhillon, Rafal Kocielnik, Ioannis Politis, Marc Swerts, and Dalila Szostak. 2011. Culture and facial expressions: A case study with a speech interface. In IFIP Conference on Human-Computer Interaction, 392--404. Retrieved from http://link.springer.com/chapter/10.1007/978--3--64223771--3_29 Google ScholarDigital Library
- Giada Di Stefano, Francesca Gino, Gary P. Pisano, and Bradley Staats. 2015. Learning by thinking: Overcoming the bias for action through reflection. Harvard Business School Cambridge, MA, USA. Retrieved from http://k12accountability.org/resources/ForPrincipals/Learning_Through_Reflection.pdfGoogle Scholar
- Giada Di Stefano, Francesca Gino, Gary P. Pisano, Bradley Staats, and Giada Di-Stefano. 2014. Learning by thinking: How reflection aids performance. Harvard Business School Boston.Google Scholar
- Daniel A. Epstein, Felicia Cordeiro, James Fogarty, Gary Hsieh, and Sean A. Munson. 2016. Crumbs: lightweight daily food challenges to promote engagement and mindfulness. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 5632--5644. Google ScholarDigital Library
- Michael Eraut. 2004. Informal learning in the workplace. Studies in continuing education 26, 2: 247-- 273.Google Scholar
- Andrew Faulring, Brad Myers, Ken Mohnkern, Bradley Schmerl, Aaron Steinfeld, John Zimmerman, Asim Smailagic, Jeffery Hansen, and Daniel Siewiorek. 2010. Agent-assisted Task Management That Reduces Email Overload. In Proceedings of the 15th International Conference on Intelligent User Interfaces (IUI '10), 61--70. Google ScholarDigital Library
- Graham Gibbs. 1988. Learning by doing: A guide to teaching and learning methods. Oxford Centre for Staff and Learning Development, Oxford Brookes University.Google Scholar
- Yolanda Gil and Varun Ratnakar. 2008. Towards Intelligent Assistance for To-do Lists. In Proceedings of the 13th International Conference on Intelligent User Interfaces (IUI '08), 329--332. Google ScholarDigital Library
- K. Gustafson and W. Bennett. 1999. Issues and difficulties in promoting learner reflection: Results from a three-year study. WWW: http://it. coe. uga. edu/ kgustafs/document/promoting. html.Google Scholar
- Bruce Hanington and Bella Martin. 2012. Universal methods of design: 100 ways to research complex problems, develop innovative ideas, and design effective solutions. Rockport Publishers.Google Scholar
- Neville Hatton and David Smith. 1995. Reflection in teacher education: Towards definition and implementation. Teaching and teacher education 11, 1: 33--49.Google Scholar
- Tom Hewitson. 2017. The conversation designer's handbook - or how to design chatbots, Google Home actions and Alexa?. Medium. Retrieved January 8, 2018 from https://medium.com/labworks-io/theconversation-designers-handbook-or-how-to-designchatbots-google-home-actions-and-alexa17ebb87b332cGoogle Scholar
- Gary Hsieh, Ian Li, Anind Dey, Jodi Forlizzi, and Scott E. Hudson. 2008. Using Visualizations to Increase Compliance in Experience Sampling. In Proceedings of the 10th International Conference on Ubiquitous Computing (UbiComp '08), 164--167. Google ScholarDigital Library
- Barry Hutchinson and Peter Bryson. 1997. Video, reflection and transformation: action research in vocational education and training in a European context. Educational action research 5, 2: 283--303.Google Scholar
- Yuan Jia, Bin Xu, Yamini Karanam, and Stephen Voida. 2016. Personality-targeted gamification: a survey study on personality traits and motivational affordances. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 2001--2013. Google ScholarDigital Library
- Prashant Kale and Harbir Singh. 2007. Building firm capabilities through learning: the role of the alliance learning process in alliance capability and firm-level alliance success. Strategic Management Journal 28, 10: 981--1000.Google ScholarCross Ref
- Jie Kang, Kyle Condiff, Shuo Chang, Joseph A. Konstan, Loren Terveen, and F. Maxwell Harper. 2017. Understanding How People Use Natural Language to Ask for Recommendations. In Proceedings of the Eleventh ACM Conference on Recommender Systems (RecSys '17), 229--237. Google ScholarDigital Library
- Florence-Emilie Kinnafick, Cecilie ThøgersenNtoumani, and Joan L. Duda. 2014. Physical activity adoption to adherence, lapse, and dropout: A selfdetermination theory perspective. Qualitative health research 24, 5: 706--718.Google Scholar
- Lorenz Cuno Klopfenstein, Saverio Delpriori, Silvia Malatini, and Alessandro Bogliolo. 2017. The Rise of Bots: A Survey of Conversational Interfaces, Patterns, and Paradigms. In Proceedings of the 2017 Conference on Designing Interactive Systems (DIS '17), 555--565. Google ScholarDigital Library
- Rafal Kocielnik and Gary Hsieh. Send Me a Different Message: Utilizing Cognitive Space to Create Engaging Message Triggers. Proceedings of the 20th ACM Conference on Computer-Supported Cooperative Work & Social Computing.: 2017. Google ScholarDigital Library
- Rafal Kocielnik, Fabrizio Maria Maggi, and Natalia Sidorova. 2013. Enabling self-reflection with LifelogExplorer: Generating simple views from complex data. In Pervasive Computing Technologies for Healthcare (PervasiveHealth), 2013 7th International Conference on, 184--191. Google ScholarDigital Library
- Rafal Kocielnik, Mykola Pechenizkiy, and Natalia Sidorova. 2012. Stress analytics in education. In Educational Data Mining 2012.Google Scholar
- Rafal Kocielnik, Natalia Sidorova, Fabrizio Maria Maggi, Martin Ouwerkerk, and Joyce HDM Westerink. 2013. Smart technologies for long-term stress monitoring at work. In Computer-Based Medical Systems (CBMS), 2013 IEEE 26th International Symposium on, 53--58.Google ScholarCross Ref
- Birgit R. Krogstie, Michael Prilla, Daniel Wessel, Kristin Knipfer, and Viktoria Pammer. 2012. Computer support for reflective learning in the workplace: A model. In Advanced Learning Technologies (ICALT), 2012 IEEE 12th International Conference on, 151--153. Google ScholarDigital Library
- Sang-su Lee, Jaemyung Lee, and Kun-pyo Lee. 2017. Designing Intelligent Assistant Through User Participations. In Proceedings of the 2017 Conference on Designing Interactive Systems (DIS '17), 173--177. Google ScholarDigital Library
- Jingyi Li, Michelle X. Zhou, Huahai Yang, and Gloria Mark. 2017. Confiding in and Listening to Virtual Agents: The Effect of Personality. In Proceedings of the 22Nd International Conference on Intelligent User Interfaces (IUI '17), 275--286. Google ScholarDigital Library
- Q. Vera Liao, Matthew Davis, Werner Geyer, Michael Muller, and N. Sadat Shami. 2016. What Can You Do?: Studying Social-Agent Orientation and Agent Proactive Interactions with an Agent for Employees. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems, 264--275. Google ScholarDigital Library
- Robert Loo and Karran Thorpe. 2002. Using reflective learning journals to improve individual and team performance. Team Performance Management: An International Journal 8, 5/6: 134--139.Google ScholarCross Ref
- Gale M. Lucas, Jonathan Gratch, Aisha King, and Louis-Philippe Morency. 2014. It's only a computer: Virtual humans increase willingness to disclose. Computers in Human Behavior 37: 94--100. Google ScholarDigital Library
- Ewa Luger and Abigail Sellen. 2016. Like having a really bad PA: the gulf between user expectation and experience of conversational agents. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 5286--5297. Google ScholarDigital Library
- Michal Luria, Guy Hoffman, and Oren Zuckerman. 2017. Comparing Social Robot, Screen and Voice Interfaces for Smart-Home Control. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 580--628. Google ScholarDigital Library
- Moira McGregor and John C. Tang. 2017. More to Meetings: Challenges in Using Speech-Based Technology to Support Meetings. In CSCW, 2208-- 2220. Google ScholarDigital Library
- Sallyanne Miller. 2005. What it's like being the 'holder of the space': a narrative on working with reflective practice in groups. Reflective Practice 6, 3: 367--377.Google ScholarCross Ref
- Jennifer A. Moon. 2013. Reflection in learning and professional development: Theory and practice. Routledge.Google Scholar
- Christine M. Neuwirth, Ravinder Chandhok, David Charney, Patricia Wojahn, and Loel Kim. 1994. Distributed collaborative writing: A comparison of spoken and written modalities for reviewing and revising documents. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 51--57. Google ScholarDigital Library
- Fredrik Ohlin and Carl Magnus Olsson. 2015. Intelligent computing in personal informatics: Key design considerations. In Proceedings of the 20th International Conference on Intelligent User Interfaces, 263--274. Google ScholarDigital Library
- Shereen Oraby, Pritam Gundecha, Jalal Mahmud, Mansurul Bhuiyan, and Rama Akkiraju. 2017. How May I Help You?: Modeling Twitter Customer ServiceConversations Using Fine-Grained Dialogue Acts. In Proceedings of the 22nd International Conference on Intelligent User Interfaces, 343--355. Google ScholarDigital Library
- Viktoria Pammer, Marina Bratic, Sandra Feyertag, and Nils Faltin. 2015. The Value of Self-tracking and the Added Value of Coaching in the Case of Improving Time Management. In Design for Teaching and Learning in a Networked World. Springer, Cham, 467-- 472.Google Scholar
- Rifca Peters, Joost Broekens, and Mark A. Neerincx. 2017. Guidelines for Tree-based Collaborative Goal Setting. In Proceedings of the 22nd International Conference on Intelligent User Interfaces, 401--405. Google ScholarDigital Library
- Bernd Ploderer, Wolfgang Reitberger, Harri OinasKukkonen, and Julia Gemert-Pijnen. 2014. Social Interaction and Reflection for Behaviour Change. Personal Ubiquitous Comput. 18, 7: 1667--1676. Google ScholarDigital Library
- Mark Pope. 2000. A brief history of career counseling in the United States. The career development quarterly 48, 3: 194--211.Google Scholar
- John Rooksby, Parvin Asadzadeh, Mattias Rost, Alistair Morrison, and Matthew Chalmers. 2016. Personal tracking of screen time on digital devices. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 284--296. Google ScholarDigital Library
- Donald A. Schon. 1983. The reflective practitioner. New York: Basic Books.Google Scholar
- Kevin Scott. 2016. Usability Heuristics for Bots. Chatbots Magazine. Retrieved January 8, 2018 from https://chatbotsmagazine.com/usability-heuristics-forbots-7075132d2c92Google Scholar
- Eunji "Jinny" Seo. 2017. 19 Best UX Practices for Building Chatbots. Chatbots Magazine. Retrieved January 8, 2018 from https://chatbotsmagazine.com/19-best-practices-forbuilding-chatbots-3c46274501b2Google Scholar
- Miriam Sherin and Elizabeth van Es. 2002. Using video to support teachers' ability to interpret classroom interactions. In Society for Information Technology & Teacher Education International Conference, 2532-- 2536.Google Scholar
- Jessica Stillman. 2013. Hiring a Remote Worker? 7 Interview Questions to Ask. Inc.com. Retrieved October 6, 2017 from https://www.inc.com/jessicastillman/hiring-remote-workers-interview-questionsto-ask.htmlGoogle Scholar
- Margaret-Anne Storey and Alexey Zagalsky. 2016. Disrupting Developer Productivity One Bot at a Time. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE 2016), 928--931. Google ScholarDigital Library
- Arnold Tannenbaum. 2013. Social psychology of the work organization. Routledge.Google Scholar
- Shelley E. Taylor. 1991. Asymmetrical effects of positive and negative events: the mobilizationminimization hypothesis. Psychological bulletin 110, 1: 67.Google Scholar
- Steve Whittaker, Vaiva Kalnikaite, Victoria Hollis, and Andrew Guydish. 2016. "Don't Waste My Time": Use of Time Information Improves Focus. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 1729--1738. Google ScholarDigital Library
- Anbang Xu, Zhe Liu, Yufan Guo, Vibha Sinha, and Rama Akkiraju. 2017. A New Chatbot for Customer Service on Social Media. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Google ScholarDigital Library
- Yu-Chun Grace Yen, Steven P. Dow, Elizabeth Gerber, and Brian P. Bailey. 2017. Listen to Others, Listen to Yourself: Combining Feedback Review and Reflection to Improve Iterative Design. In Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition, 158--170. Google ScholarDigital Library
- Dongwook Yoon, Nicholas Chen, Bernie Randles, Amy Cheatle, Corinna E. Löckenhoff, Steven J. Jackson, Abigail Sellen, and François Guimbretière. 2016. RichReview++: Deployment of a Collaborative Multi-modal Annotation System for Instructor Feedback and Peer Discussion. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing, 195--205. Google ScholarDigital Library
- Maurizio Zollo and Sidney G. Winter. 2002. Deliberate learning and the evolution of dynamic capabilities. Organization science 13, 3: 339--351. Google ScholarDigital Library
- Alexa is for fun, Siri is because typing is hard: survey. Retrieved October 6, 2017 from https://www.usatoday.com/story/tech/talkingtech/2017 /06/05/alexa-fun-siri-because-typing-hardsurvey/102436072/Google Scholar
- Slack API. Retrieved January 8, 2018 from https://api.slack.com/Google Scholar
- Sample PBI Questions - Performance Based Interviewing (PBI). Retrieved October 6, 2017 from https://www.va.gov/PBI/Questions.aspGoogle Scholar
Index Terms
Designing for Workplace Reflection: A Chat and Voice-Based Conversational Agent
Recommendations
Reflection Companion: A Conversational System for Engaging Users in Reflection on Physical Activity
Mobile, wearable and other connected devices allow people to collect and explore large amounts of data about their own activities, behavior, and well-being. Yet, learning from-, and acting upon such data remain a challenge. The process of reflection has ...
Combining Momentary and Retrospective Self-Reflection in a Mobile Photo-Based Journaling Application
NordiCHI '22: Nordic Human-Computer Interaction ConferenceThe concept of self-reflection is investigated in a wide range of fields, from Psychology to HCI. Different fields use different methods to trigger reflection, such as visualizing accumulated personal quantitative data, personal photos, journaling, or ...
Comparing How a Chatbot References User Utterances from Previous Chatting Sessions: An Investigation of Users' Privacy Concerns and Perceptions
HAI '23: Proceedings of the 11th International Conference on Human-Agent InteractionChatbots are capable of remembering and referencing previous conversations, but does this enhance user engagement or infringe on privacy? To explore this trade-off, we investigated the format of how a chatbot references previous conversations with a ...
Comments