Understanding the Long-Term Use of Smart Speaker Assistants

Abstract

Over the past two years the Ubicomp vision of ambient voice assistants, in the form of smart speakers such as the Amazon Echo and Google Home, has been integrated into tens of millions of homes. However, the use of these systems over time in the home has not been studied in depth. We set out to understand exactly what users are doing with these devices over time through analyzing voice history logs of 65,499 interactions with existing Google Home devices from 88 diverse homes over an average of 110 days. We found that specific types of commands were made more often at particular times of day and that commands in some domains increased in length over time as participants tried out new ways to interact with their devices, yet exploration of new topics was low. Four distinct user groups also emerged based on using the device more or less during the day vs. in the evening or using particular categories. We conclude by comparing smart speaker use to a similar study of smartphone use and offer implications for the design of new smart speaker assistants and skills, highlighting specific areas where both manufacturers and skill providers can focus in this domain.

References

1. Apple Computer. 1987. Knowledge Navigator. Available at: https://www.youtube.com/watch?v=-jiBLQyUi38 Accessed 9/5/2017.
2. Frank R. Bentley, Nediyana Daskalova, and Brooke White. 2017. Comparing the Reliability of Amazon Mechanical Turk and Survey Monkey to Traditional Market Research Surveys. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '17). ACM, New York, NY, USA, 1092--1099.
3. Frank R. Bentley and Ying-Yu Chen. 2015. The Composition and Use of Modern Mobile Phonebooks. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 2749--2758.
4. Anne-Jorunn. Berg. 1994. "A Gendered Socio-technical Construction: The Smart House", Cynthia Cockburn and Ruža First-Dilić (eds.), Bringing Technology Home: Gender and Technology in a Changing Europe, Buckingham: Open University Press.
5. A.J. Bernheim Brush, Bongshin Lee, Ratul Mahajan, Sharad Agarwal, Stefan Saroiu, and Colin Dixon. 2011. Home automation in the wild: challenges and opportunities. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 2115--2124.
6. Matthias Böhmer, Brent Hecht, Johannes Schöning, Antonio Krüger, and Gernot Bauer. 2011. Falling asleep with Angry Birds, Facebook and Kindle: a large scale study on mobile application usage. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI '11). ACM, New York, NY, USA, 47--56.
7. Rodney A. Brooks. 1997. "The intelligent room project." In Cognitive Technology, 1997. Humanizing the Information Age. Proceedings., Second International Conference on, pp. 271--278. IEEE.
8. Amy S. Bruckman, Casey Fiesler, Jeff Hancock, and Cosmin Munteanu. "CSCW Research Ethics Town Hall: Working Towards Community Norms." In Companion of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, pp. 113--115. ACM, 2017.
9. Frank R. Bentley and Ying-Yu Chen. 2015. The Composition and Use of Modern Mobile Phonebooks. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 2749--2758.
10. Michael Coen, Luke Weisman, Kavita Thomas, and Marion Groh. 2000. "A context sensitive natural language modality for an intelligent room." In Managing interactions in smart environments, pp. 68--79. Springer, London.
11. Jeffrey Dean. July 7, 2016 Large-Scale Deep Learning with TensorFlow for Building Intelligent Systems, ACM Webinar
12. European Union General Data Protection Regulation. http://www.eugdpr.org/ Accessed 9/4/17
13. Michael Grace-Martin and Geri Gay. Web browsing, mobile computing and academic performance. Educational Technology 8 Society 4, no. 3 (2001): 95--107.
14. Kirstie Hawkey and Kori Inkpen. 2005. Web browsing today: the impact of changing contexts on user activity. In CHI '05 Extended Abstracts on Human Factors in Computing Systems (CHI EA '05). ACM, New York, NY, USA, 1443--1446.
15. Daqing He and Ayse Göker. "Detecting session boundaries from web user logs." In Proceedings of the BCS-IRSG 22nd annual colloquium on information retrieval research, pp. 57--66. 2000.
16. Stefan Junestrand, Konrad Tollmar, Sören Lenman, and Björn Thuresson. 2000. Private and public spaces: the use of video mediated communication in a future home environment. In CHI '00 Extended Abstracts on Human Factors in Computing Systems (CHI EA '00). ACM, New York, NY, USA, 16--17.
17. Cory Kidd, Robert Orr, Gregory Abowd, Christopher Atkeson, Irfan Essa, Blair MacIntyre, Elizabeth Mynatt, Thad Starner, and Wendy Newstetter. 1999. "The aware home: A living laboratory for ubiquitous computing research." Cooperative buildings. Integrating information, organizations, and architecture (1999): 191--198.
18. Mounia Lalmas, Heather O'Brien, and Elad Yom-Tov. "Measuring user engagement." Synthesis Lectures on Information Concepts, Retrieval, and Services 6.4 (2014): 1--132.
19. Ewa Luger and Abigail Sellen. 2016. "Like Having a Really Bad PA": The Gulf between User Expectation and Experience of Conversational Agents. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 5286--5297.
20. Sarah Mennicken and Elaine Huang. 2012. "Hacking the natural habitat: an in-the-wild study of smart homes, their development, and the people who live in them." Pervasive computing (2012): 143--160.
21. Amanda Purington, Jessie G. Taft, Shruti Sannon, Natalya N. Bazarova, and Samuel Hardman Taylor. 2017. "Alexa is my new BFF": Social Roles, User Satisfaction, and Personification of the Amazon Echo. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '17). ACM, New York, NY, USA, 2853--2859.
22. Simon Robinson, Jennifer Pearson, Shashank Ahire, Rini Ahirwar, Bhakti Bhikne, Nimish Maravi, and Matt Jones. 2018. Revisiting "Hole in the Wall" Computing: Private Smart Speakers and Public Slum Settings. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems(CHI '18). ACM, New York, NY, USA, Paper 498, 11 pages.
23. Myra Spiliopoulou, Myra, Bamshad Mobasher, Bettina Berendt, and Miki Nakagawa. "A framework for the evaluation of session reconstruction heuristics in web-usage analysis." Informs journal on computing 15, no. 2 (2003): 171--190.
24. Mark Weiser and John Seely Brown. "The coming age of calm technology." Beyond calculation. Springer, New York, NY, 1997. 75--85.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • HCI design and evaluation methods

      • Field studies

  • Ubiquitous and mobile computing

    • Empirical studies in ubiquitous and mobile computing

• Information systems

  • Information retrieval

    • Specialized information retrieval

      • Multimedia and multimodal retrieval

        • Speech / audio search