skip to main content
10.1145/3196709.3196784acmconferencesArticle/Chapter ViewAbstractPublication PagesdisConference Proceedingsconference-collections
research-article

Designing for Workplace Reflection: A Chat and Voice-Based Conversational Agent

Authors Info & Claims
Published:08 June 2018Publication History

ABSTRACT

Conversational agents stand to play an important role in supporting behavior change and well-being in many domains. With users able to interact with conversational agents through both text and voice, understanding how designing for these channels supports behavior change is important. To begin answering this question, we designed a conversational agent for the workplace that supports workers' activity journaling and self-learning through reflection. Our agent, named Robota, combines chat-based communication as a Slack Bot and voice interaction through a personal device using a custom Amazon Alexa Skill. Through a 3-week controlled deployment, we examine how voice-based and chat-based interaction affect workers' reflection and support self-learning. We demonstrate that, while many current technical limitations exist, adding dedicated mobile voice interaction separate from the already busy chat modality may further enable users to step back and reflect on their work. We conclude with discussion of the implications of our findings to design of workplace self-tracking systems specifically and to behavior-change systems in general.

Skip Supplemental Material Section

Supplemental Material

suppl.mov

Supplemental video

References

  1. Elena Agapie, Daniel Avrahami, and Jennifer Marlow. 2016. Staying the Course: System-Driven Lapse Management for Supporting Behavior Change. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 1072--1083. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Rusul Alrubail. Scaffolding Student Reflections + Sample Questions. Edutopia. Retrieved January 8, 2018 from https://www.edutopia.org/discussion/scaffoldingstudent-reflections-sample-questionsGoogle ScholarGoogle Scholar
  3. Teresa M. Amabile and Steven J. Kramer. 2011. The power of small wins. Harvard Business Review 89, 5: 70--80.Google ScholarGoogle Scholar
  4. Frederik Anseel, Filip Lievens, and Eveline Schollaert. 2009. Reflection as a strategy to enhance task performance after feedback. Organizational Behavior and Human Decision Processes 110, 1: 23--35.Google ScholarGoogle ScholarCross RefCross Ref
  5. Daniel Avrahami, Mitesh Patel, Yusuke Yamaura, and Sven Kratz. 2018. Below the Surface: Unobtrusive Activity Recognition for Work Surfaces Using RFradar Sensing. In 23rd International Conference on Intelligent User Interfaces (IUI '18), 439--451. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. John D. Bain, Roy Ballantyne, Jan Packer, and Colleen Mills. 1999. Using journal writing to enhance student teachers' reflectivity during field experience placements. Teachers and Teaching 5, 1: 51--73.Google ScholarGoogle ScholarCross RefCross Ref
  7. Yehuda Baruch. 2003. Career systems in transition: A normative model for organizational career practices. Personnel review 32, 2: 231--251.Google ScholarGoogle Scholar
  8. Eric P.S. Baumer, Vera Khovanskaya, Mark Matthews, Lindsay Reynolds, Victoria Schwanda Sosik, and Geri Gay. 2014. Reviewing Reflection: On the Use of Reflection in Interactive System Design. In Proceedings of the 2014 Conference on Designing Interactive Systems (DIS '14), 93--102. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Timothy Bickmore, Daniel Mauer, Francisco Crespo, and Thomas Brown. 2007. Persuasion, task interruption and health regimen adherence. In International Conference on Persuasive Technology, 1--11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. David Boud, Rosemary Keogh, and David Walker. 2013. Reflection: Turning experience into learning. Routledge.Google ScholarGoogle ScholarCross RefCross Ref
  11. Matthew Carrasco, Eunyee Koh, and Sana Malik. 2017. popHistory: Animated Visualization of Personal Web Browsing History. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '17), 2429-- 2436. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Sunny Consolvo, Predrag Klasnja, David W. McDonald, Daniel Avrahami, Jon Froehlich, Louis LeGrand, Ryan Libby, Keith Mosher, and James A. Landay. 2008. Flowers or a robot army?: encouraging awareness & activity with personal, mobile displays. In Proceedings of the 10th international conference on Ubiquitous computing, 54--63. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Sunny Consolvo, David W. McDonald, and James A. Landay. 2009. Theory-driven Design Strategies for Technologies That Support Behavior Change in Everyday Life. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09), 405--414. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Felicia Cordeiro, Elizabeth Bales, Erin Cherry, and James Fogarty. 2015. Rethinking the mobile food journal: Exploring opportunities for lightweight photobased capture. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 3207--3216. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Felicia Cordeiro, Daniel A. Epstein, Edison Thomaz, Elizabeth Bales, Arvind K. Jagannathan, Gregory D. Abowd, and James Fogarty. 2015. Barriers and negative nudges: Exploring challenges in food journaling. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 1159--1162. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Justin Cranshaw, Emad Elwany, Todd Newman, Rafal Kocielnik, Bowen Yu, Sandeep Soni, Jaime Teevan, and Andrés Monroy-Hernández. 2017. Calendar. help: Designing a Workflow-Based Scheduling Agent with Humans in the Loop. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 2382--2393. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Beant Dhillon, Rafal Kocielnik, Ioannis Politis, Marc Swerts, and Dalila Szostak. 2011. Culture and facial expressions: A case study with a speech interface. In IFIP Conference on Human-Computer Interaction, 392--404. Retrieved from http://link.springer.com/chapter/10.1007/978--3--64223771--3_29 Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Giada Di Stefano, Francesca Gino, Gary P. Pisano, and Bradley Staats. 2015. Learning by thinking: Overcoming the bias for action through reflection. Harvard Business School Cambridge, MA, USA. Retrieved from http://k12accountability.org/resources/ForPrincipals/Learning_Through_Reflection.pdfGoogle ScholarGoogle Scholar
  19. Giada Di Stefano, Francesca Gino, Gary P. Pisano, Bradley Staats, and Giada Di-Stefano. 2014. Learning by thinking: How reflection aids performance. Harvard Business School Boston.Google ScholarGoogle Scholar
  20. Daniel A. Epstein, Felicia Cordeiro, James Fogarty, Gary Hsieh, and Sean A. Munson. 2016. Crumbs: lightweight daily food challenges to promote engagement and mindfulness. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 5632--5644. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Michael Eraut. 2004. Informal learning in the workplace. Studies in continuing education 26, 2: 247-- 273.Google ScholarGoogle Scholar
  22. Andrew Faulring, Brad Myers, Ken Mohnkern, Bradley Schmerl, Aaron Steinfeld, John Zimmerman, Asim Smailagic, Jeffery Hansen, and Daniel Siewiorek. 2010. Agent-assisted Task Management That Reduces Email Overload. In Proceedings of the 15th International Conference on Intelligent User Interfaces (IUI '10), 61--70. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Graham Gibbs. 1988. Learning by doing: A guide to teaching and learning methods. Oxford Centre for Staff and Learning Development, Oxford Brookes University.Google ScholarGoogle Scholar
  24. Yolanda Gil and Varun Ratnakar. 2008. Towards Intelligent Assistance for To-do Lists. In Proceedings of the 13th International Conference on Intelligent User Interfaces (IUI '08), 329--332. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. K. Gustafson and W. Bennett. 1999. Issues and difficulties in promoting learner reflection: Results from a three-year study. WWW: http://it. coe. uga. edu/ kgustafs/document/promoting. html.Google ScholarGoogle Scholar
  26. Bruce Hanington and Bella Martin. 2012. Universal methods of design: 100 ways to research complex problems, develop innovative ideas, and design effective solutions. Rockport Publishers.Google ScholarGoogle Scholar
  27. Neville Hatton and David Smith. 1995. Reflection in teacher education: Towards definition and implementation. Teaching and teacher education 11, 1: 33--49.Google ScholarGoogle Scholar
  28. Tom Hewitson. 2017. The conversation designer's handbook - or how to design chatbots, Google Home actions and Alexa?. Medium. Retrieved January 8, 2018 from https://medium.com/labworks-io/theconversation-designers-handbook-or-how-to-designchatbots-google-home-actions-and-alexa17ebb87b332cGoogle ScholarGoogle Scholar
  29. Gary Hsieh, Ian Li, Anind Dey, Jodi Forlizzi, and Scott E. Hudson. 2008. Using Visualizations to Increase Compliance in Experience Sampling. In Proceedings of the 10th International Conference on Ubiquitous Computing (UbiComp '08), 164--167. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Barry Hutchinson and Peter Bryson. 1997. Video, reflection and transformation: action research in vocational education and training in a European context. Educational action research 5, 2: 283--303.Google ScholarGoogle Scholar
  31. Yuan Jia, Bin Xu, Yamini Karanam, and Stephen Voida. 2016. Personality-targeted gamification: a survey study on personality traits and motivational affordances. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 2001--2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Prashant Kale and Harbir Singh. 2007. Building firm capabilities through learning: the role of the alliance learning process in alliance capability and firm-level alliance success. Strategic Management Journal 28, 10: 981--1000.Google ScholarGoogle ScholarCross RefCross Ref
  33. Jie Kang, Kyle Condiff, Shuo Chang, Joseph A. Konstan, Loren Terveen, and F. Maxwell Harper. 2017. Understanding How People Use Natural Language to Ask for Recommendations. In Proceedings of the Eleventh ACM Conference on Recommender Systems (RecSys '17), 229--237. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Florence-Emilie Kinnafick, Cecilie ThøgersenNtoumani, and Joan L. Duda. 2014. Physical activity adoption to adherence, lapse, and dropout: A selfdetermination theory perspective. Qualitative health research 24, 5: 706--718.Google ScholarGoogle Scholar
  35. Lorenz Cuno Klopfenstein, Saverio Delpriori, Silvia Malatini, and Alessandro Bogliolo. 2017. The Rise of Bots: A Survey of Conversational Interfaces, Patterns, and Paradigms. In Proceedings of the 2017 Conference on Designing Interactive Systems (DIS '17), 555--565. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Rafal Kocielnik and Gary Hsieh. Send Me a Different Message: Utilizing Cognitive Space to Create Engaging Message Triggers. Proceedings of the 20th ACM Conference on Computer-Supported Cooperative Work & Social Computing.: 2017. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Rafal Kocielnik, Fabrizio Maria Maggi, and Natalia Sidorova. 2013. Enabling self-reflection with LifelogExplorer: Generating simple views from complex data. In Pervasive Computing Technologies for Healthcare (PervasiveHealth), 2013 7th International Conference on, 184--191. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Rafal Kocielnik, Mykola Pechenizkiy, and Natalia Sidorova. 2012. Stress analytics in education. In Educational Data Mining 2012.Google ScholarGoogle Scholar
  39. Rafal Kocielnik, Natalia Sidorova, Fabrizio Maria Maggi, Martin Ouwerkerk, and Joyce HDM Westerink. 2013. Smart technologies for long-term stress monitoring at work. In Computer-Based Medical Systems (CBMS), 2013 IEEE 26th International Symposium on, 53--58.Google ScholarGoogle ScholarCross RefCross Ref
  40. Birgit R. Krogstie, Michael Prilla, Daniel Wessel, Kristin Knipfer, and Viktoria Pammer. 2012. Computer support for reflective learning in the workplace: A model. In Advanced Learning Technologies (ICALT), 2012 IEEE 12th International Conference on, 151--153. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Sang-su Lee, Jaemyung Lee, and Kun-pyo Lee. 2017. Designing Intelligent Assistant Through User Participations. In Proceedings of the 2017 Conference on Designing Interactive Systems (DIS '17), 173--177. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Jingyi Li, Michelle X. Zhou, Huahai Yang, and Gloria Mark. 2017. Confiding in and Listening to Virtual Agents: The Effect of Personality. In Proceedings of the 22Nd International Conference on Intelligent User Interfaces (IUI '17), 275--286. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Q. Vera Liao, Matthew Davis, Werner Geyer, Michael Muller, and N. Sadat Shami. 2016. What Can You Do?: Studying Social-Agent Orientation and Agent Proactive Interactions with an Agent for Employees. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems, 264--275. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Robert Loo and Karran Thorpe. 2002. Using reflective learning journals to improve individual and team performance. Team Performance Management: An International Journal 8, 5/6: 134--139.Google ScholarGoogle ScholarCross RefCross Ref
  45. Gale M. Lucas, Jonathan Gratch, Aisha King, and Louis-Philippe Morency. 2014. It's only a computer: Virtual humans increase willingness to disclose. Computers in Human Behavior 37: 94--100. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Ewa Luger and Abigail Sellen. 2016. Like having a really bad PA: the gulf between user expectation and experience of conversational agents. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 5286--5297. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Michal Luria, Guy Hoffman, and Oren Zuckerman. 2017. Comparing Social Robot, Screen and Voice Interfaces for Smart-Home Control. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 580--628. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Moira McGregor and John C. Tang. 2017. More to Meetings: Challenges in Using Speech-Based Technology to Support Meetings. In CSCW, 2208-- 2220. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Sallyanne Miller. 2005. What it's like being the 'holder of the space': a narrative on working with reflective practice in groups. Reflective Practice 6, 3: 367--377.Google ScholarGoogle ScholarCross RefCross Ref
  50. Jennifer A. Moon. 2013. Reflection in learning and professional development: Theory and practice. Routledge.Google ScholarGoogle Scholar
  51. Christine M. Neuwirth, Ravinder Chandhok, David Charney, Patricia Wojahn, and Loel Kim. 1994. Distributed collaborative writing: A comparison of spoken and written modalities for reviewing and revising documents. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 51--57. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Fredrik Ohlin and Carl Magnus Olsson. 2015. Intelligent computing in personal informatics: Key design considerations. In Proceedings of the 20th International Conference on Intelligent User Interfaces, 263--274. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Shereen Oraby, Pritam Gundecha, Jalal Mahmud, Mansurul Bhuiyan, and Rama Akkiraju. 2017. How May I Help You?: Modeling Twitter Customer ServiceConversations Using Fine-Grained Dialogue Acts. In Proceedings of the 22nd International Conference on Intelligent User Interfaces, 343--355. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Viktoria Pammer, Marina Bratic, Sandra Feyertag, and Nils Faltin. 2015. The Value of Self-tracking and the Added Value of Coaching in the Case of Improving Time Management. In Design for Teaching and Learning in a Networked World. Springer, Cham, 467-- 472.Google ScholarGoogle Scholar
  55. Rifca Peters, Joost Broekens, and Mark A. Neerincx. 2017. Guidelines for Tree-based Collaborative Goal Setting. In Proceedings of the 22nd International Conference on Intelligent User Interfaces, 401--405. Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Bernd Ploderer, Wolfgang Reitberger, Harri OinasKukkonen, and Julia Gemert-Pijnen. 2014. Social Interaction and Reflection for Behaviour Change. Personal Ubiquitous Comput. 18, 7: 1667--1676. Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. Mark Pope. 2000. A brief history of career counseling in the United States. The career development quarterly 48, 3: 194--211.Google ScholarGoogle Scholar
  58. John Rooksby, Parvin Asadzadeh, Mattias Rost, Alistair Morrison, and Matthew Chalmers. 2016. Personal tracking of screen time on digital devices. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 284--296. Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Donald A. Schon. 1983. The reflective practitioner. New York: Basic Books.Google ScholarGoogle Scholar
  60. Kevin Scott. 2016. Usability Heuristics for Bots. Chatbots Magazine. Retrieved January 8, 2018 from https://chatbotsmagazine.com/usability-heuristics-forbots-7075132d2c92Google ScholarGoogle Scholar
  61. Eunji "Jinny" Seo. 2017. 19 Best UX Practices for Building Chatbots. Chatbots Magazine. Retrieved January 8, 2018 from https://chatbotsmagazine.com/19-best-practices-forbuilding-chatbots-3c46274501b2Google ScholarGoogle Scholar
  62. Miriam Sherin and Elizabeth van Es. 2002. Using video to support teachers' ability to interpret classroom interactions. In Society for Information Technology & Teacher Education International Conference, 2532-- 2536.Google ScholarGoogle Scholar
  63. Jessica Stillman. 2013. Hiring a Remote Worker? 7 Interview Questions to Ask. Inc.com. Retrieved October 6, 2017 from https://www.inc.com/jessicastillman/hiring-remote-workers-interview-questionsto-ask.htmlGoogle ScholarGoogle Scholar
  64. Margaret-Anne Storey and Alexey Zagalsky. 2016. Disrupting Developer Productivity One Bot at a Time. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE 2016), 928--931. Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. Arnold Tannenbaum. 2013. Social psychology of the work organization. Routledge.Google ScholarGoogle Scholar
  66. Shelley E. Taylor. 1991. Asymmetrical effects of positive and negative events: the mobilizationminimization hypothesis. Psychological bulletin 110, 1: 67.Google ScholarGoogle Scholar
  67. Steve Whittaker, Vaiva Kalnikaite, Victoria Hollis, and Andrew Guydish. 2016. "Don't Waste My Time": Use of Time Information Improves Focus. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 1729--1738. Google ScholarGoogle ScholarDigital LibraryDigital Library
  68. Anbang Xu, Zhe Liu, Yufan Guo, Vibha Sinha, and Rama Akkiraju. 2017. A New Chatbot for Customer Service on Social Media. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Google ScholarGoogle ScholarDigital LibraryDigital Library
  69. Yu-Chun Grace Yen, Steven P. Dow, Elizabeth Gerber, and Brian P. Bailey. 2017. Listen to Others, Listen to Yourself: Combining Feedback Review and Reflection to Improve Iterative Design. In Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition, 158--170. Google ScholarGoogle ScholarDigital LibraryDigital Library
  70. Dongwook Yoon, Nicholas Chen, Bernie Randles, Amy Cheatle, Corinna E. Löckenhoff, Steven J. Jackson, Abigail Sellen, and François Guimbretière. 2016. RichReview++: Deployment of a Collaborative Multi-modal Annotation System for Instructor Feedback and Peer Discussion. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing, 195--205. Google ScholarGoogle ScholarDigital LibraryDigital Library
  71. Maurizio Zollo and Sidney G. Winter. 2002. Deliberate learning and the evolution of dynamic capabilities. Organization science 13, 3: 339--351. Google ScholarGoogle ScholarDigital LibraryDigital Library
  72. Alexa is for fun, Siri is because typing is hard: survey. Retrieved October 6, 2017 from https://www.usatoday.com/story/tech/talkingtech/2017 /06/05/alexa-fun-siri-because-typing-hardsurvey/102436072/Google ScholarGoogle Scholar
  73. Slack API. Retrieved January 8, 2018 from https://api.slack.com/Google ScholarGoogle Scholar
  74. Sample PBI Questions - Performance Based Interviewing (PBI). Retrieved October 6, 2017 from https://www.va.gov/PBI/Questions.aspGoogle ScholarGoogle Scholar

Index Terms

  1. Designing for Workplace Reflection: A Chat and Voice-Based Conversational Agent

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      DIS '18: Proceedings of the 2018 Designing Interactive Systems Conference
      June 2018
      1418 pages
      ISBN:9781450351980
      DOI:10.1145/3196709

      Copyright © 2018 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 8 June 2018

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

      Acceptance Rates

      DIS '18 Paper Acceptance Rate107of487submissions,22%Overall Acceptance Rate1,158of4,684submissions,25%

      Upcoming Conference

      DIS '24
      Designing Interactive Systems Conference
      July 1 - 5, 2024
      Copenhagen , Denmark

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader