Which one is me?: Identifying Oneself on Public Displays

ABSTRACT

While user representations are extensively used on public displays, it remains unclear how well users can recognize their own representation among those of surrounding users. We study the most widely used representations: abstract objects, skeletons, silhouettes and mirrors. In a prestudy (N=12), we identify five strategies that users follow to recognize themselves on public displays. In a second study (N=19), we quantify the users' recognition time and accuracy with respect to each representation type. Our findings suggest that there is a significant effect of (1) the representation type, (2) the strategies performed by users, and (3) the combination of both on recognition time and accuracy. We discuss the suitability of each representation for different settings and provide specific recommendations as to how user representations should be applied in multi-user scenarios. These recommendations guide practitioners and researchers in selecting the representation that optimizes the most for the deployment's requirements, and for the user strategies that are feasible in that environment.

References

1. Christopher Ackad, Andrew Clayphan, Martin Tomitsch, and Judy Kay. 2015. An In-the-wild Study of Learning Mid-air Gestures to Browse Hierarchical Information at a Large Interactive Public Display. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '15). ACM, New York, NY, USA, 1227--1238.
2. Christopher Ackad, Martin Tomitsch, and Judy Kay. 2016. Skeletons and Silhouettes: Comparing User Representations at a Gesture-based Large Display. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 2343--2347.
3. Christopher Ackad, Rainer Wasinger, Richard Gluga, Judy Kay, and Martin Tomitsch. 2013. Measuring Interactivity at an Interactive Public Information Display. In Proceedings of the 25th Australian Computer-Human Interaction Conference: Augmentation, Application, Innovation, Collaboration (OzCHI '13). ACM, New York, NY, USA, 329--332.
4. Moritz Behrens, Ava Fatah gen. Schieck, Efsathia Kostopoulou, Steve North, Wallis Motta, Lei Ye, and Holger Schnadelbach. 2013. Exploring the Effect of Spatial Layout on Mediated Urban Interactions. In Proceedings of the 2Nd ACM International Symposium on Pervasive Displays (PerDis '13). ACM, New York, NY, USA, 79--84.
5. Victor Cheung and Stacey Scott. 2016. Proxemics-Based Visual Concepts to Attract and Engage Public Display Users: Adaptive Content Motion and Adaptive User Shadow. In Proceedings of the 2016 ACM on Interactive Surfaces and Spaces (ISS '16). ACM, New York, NY, USA, 473--476.
6. James E. Cutting and Lynn T. Kozlowski. 1977. Recognizing friends by their walk: Gait perception without familiarity cues. Bulletin of the Psychonomic Society 9, 5 (1977), 353--356.
7. Nigel Davies, Sarah Clinch, and Florian Alt. 2014. Pervasive Displays: Understanding the Future of Digital Signage (1st ed.). Morgan & Claypool Publishers.
8. Ivan Elhart, Federico Scacchi, Evangelos Niforatos, and Marc Langheinrich. 2015. ShadowTouch: A Multi-user Application Selection Interface for Interactive Public Displays. In Proceedings of the 4th International Symposium on Pervasive Displays (PerDis '15). ACM, New York, NY, USA, 209--216.
9. Jonna Häkkilä, Olli Koskenranta, Maaret Posti, Leena Ventä-Olkkonen, and Ashley Colley. 2013. Clearing the Virtual Window: Connecting Two Locations with Interactive Public Displays. In Proceedings of the 2Nd ACM International Symposium on Pervasive Displays (PerDis '13). ACM, New York, NY, USA, 85--90.
10. Giulio Jacucci, Ann Morrison, Gabriela T. Richard, Jari Kleimola, Peter Peltonen, Lorenza Parisi, and Toni Laitinen. 2010. Worlds of Information: Designing for Engagement at a Public Multi-touch Display. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10). ACM, New York, NY, USA, 2267--2276.
11. Mohamed Khamis, Florian Alt, and Andreas Bulling. 2015. A Field Study on Spontaneous Gaze-based Interaction with a Public Display Using Pursuits. In Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers (UbiComp/ISWC'15 Adjunct). ACM, New York, NY, USA, 863--872.
12. Mohamed Khamis, Florian Alt, and Andreas Bulling. 2016. Challenges and Design Space of Gaze-enabled Public Displays. In Ajunct Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '16). ACM, New York, NY, USA, 10.
13. Mohamed Khamis, Daniel Buschek, Tobias Thieron, Florian Alt, and Andreas Bulling. 2017a. EyePACT: Eye-Based Parallax Correction on Touch-Enabled Interactive Displays. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 4, Article 146 (Dec. 2017), 18 pages.
14. Mohamed Khamis, Regina Hasholzner, Andreas Bulling, and Florian Alt. 2017b. GTmoPass: Two-factor Authentication on Public Displays Using Gaze-touch Passwords and Personal Mobile Devices. In Proceedings of the 6th ACM International Symposium on Pervasive Displays (PerDis '17). ACM, New York, NY, USA, Article 8, 9 pages.
15. David Kieras. 2001. Using the keystroke-level model to estimate execution times. University of Michigan (2001).
16. Christian Lander, Marco Speicher, Denise Paradowski, Norine Coenen, Sebastian Biewer, and Antonio Krüger. 2015. Collaborative Newspaper: Exploring an Adaptive Scrolling Algorithm in a Multi-user Reading Scenario. In Proceedings of the 4th International Symposium on Pervasive Displays (PerDis '15). ACM, New York, NY, USA, 163--169.
17. Eva Lösch, Florian Alt, and Michael Koch. 2017. Mirror, Mirror on the Wall: Attracting Passers-by to Public Touch Displays With User Representations. In Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces (ISS '17). ACM, New York, NY, USA, 22--31.
18. Andrés Lucero, Jussi Holopainen, and Tero Jokela. 2012. MobiComics: Collaborative Use of Mobile Phones and Large Displays for Public Expression. In Proceedings of the 14th International Conference on Human-computer Interaction with Mobile Devices and Services (MobileHCI '12). ACM, New York, NY, USA, 383--392.
19. Ville Mäkelä, Tomi Heimonen, and Markku Turunen. 2014. Magnetic Cursor: Improving Target Selection in Freehand Pointing Interfaces. In Proceedings of The International Symposium on Pervasive Displays (PerDis '14). ACM, New York, NY, USA, Article 112, 6 pages.
20. Ville Mäkelä, Sumita Sharma, Jaakko Hakulinen, Tomi Heimonen, and Markku Turunen. 2017. Challenges in Public Display Deployments: A Taxonomy of External Factors. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 3426--3475.
21. Daniel Michelis and Jörg Müller. 2011. The Audience Funnel: Observations of Gesture Based Interaction With Multiple Large Displays in a City Center. International Journal of Humanâ Computer Interaction 27, 6 (2011), 562--579.
22. Robert W. Mitchell. 1993. Mental models of mirror-self-recognition: Two theories. New Ideas in Psychology 11, 3 (1993), 295 -- 325.
23. Jörg Müller, Florian Alt, Daniel Michelis, and Albrecht Schmidt. 2010. Requirements and Design Space for Interactive Public Displays. In Proceedings of the 18th ACM International Conference on Multimedia (MM '10). ACM, New York, NY, USA, 1285--1294.
24. Jörg Müller, Gilles Bailly, Thor Bossuyt, and Niklas Hillgren. 2014a. MirrorTouch: Combining Touch and Mid-air Gestures for Public Displays. In Proceedings of the 16th International Conference on Human-computer Interaction with Mobile Devices & Services (MobileHCI '14). ACM, New York, NY, USA, 319--328.
25. Jörg Müller, Dieter Eberle, and Konrad Tollmar. 2014b. Communiplay: A Field Study of a Public Display Mediaspace. In Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 1415--1424.
26. Jörg Müller, Robert Walter, Gilles Bailly, Michael Nischt, and Florian Alt. 2012. Looking Glass: A Field Study on Noticing Interactivity of a Shop Window. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, New York, NY, USA, 297--306.
27. Philipp Panhey, Tanja Döring, Stefan Schneegass, Dirk Wenig, and Florian Alt. 2015. What People Really Remember: Understanding Cognitive Effects When Interacting with Large Displays. In Proceedings of the 2015 International Conference on Interactive Tabletops & Surfaces (ITS '15). ACM, New York, NY, USA, 103--106.
28. Gonzalo Parra, Robin De Croon, Joris Klerkx, and Erik Duval. 2014. Quantifying the Interaction Stages of a Public Display Campaign in the Wild. In Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational (NordiCHI '14). ACM, New York, NY, USA, 757--760.
29. Peter Peltonen, Esko Kurvinen, Antti Salovaara, Giulio Jacucci, Tommi Ilmonen, John Evans, Antti Oulasvirta, and Petri Saarikko. 2008. It's Mine, Don'T Touch!: Interactions at a Large Multi-touch Display in a City Centre. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '08). ACM, New York, NY, USA, 1285--1294.
30. Martin Tomitsch, Christopher Ackad, Oliver Dawson, Luke Hespanhol, and Judy Kay. 2014. Who Cares About the Content? An Analysis of Playful Behaviour at a Public Display. In Proceedings of The International Symposium on Pervasive Displays (PerDis '14). ACM, New York, NY, USA, Article 160, 6 pages.
31. Nina Valkanova, Robert Walter, Andrew Vande Moere, and Jörg Müller. 2014. MyPosition: Sparking Civic Discourse by a Public Interactive Poll Visualization. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '14). ACM, New York, NY, USA, 1323--1332.
32. Jouni Vepsäläinen, Petri Savolainen, Jouni Ojala, Antonella Di Rienzo, Matti Nelimarkka, Kai Kuikkaniemi, Sasu Tarkoma, and Giulio Jacucci. 2016. Web-Based Public-Screen Gaming: Insights from Deployments. IEEE Pervasive Computing 15, 3 (July 2016), 40--46.
33. Daniel Vogel and Ravin Balakrishnan. 2004. Interactive Public Ambient Displays: Transitioning from Implicit to Explicit, Public to Personal, Interaction with Multiple Users. In Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology (UIST '04). ACM, New York, NY, USA, 137--146.
34. Robert Walter, Gilles Bailly, and Jörg Müller. 2013. StrikeAPose: Revealing Mid-air Gestures on Public Displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, New York, NY, USA, 841--850.
35. Robert Walter, Gilles Bailly, Nina Valkanova, and Jörg Müller. 2014. Cuenesics: Using Mid-air Gestures to Select Items on Interactive Public Displays. In Proceedings of the 16th International Conference on Human-computer Interaction with Mobile Devices & Services (MobileHCI '14). ACM, New York, NY, USA, 299--308.
36. Robert Walter, Andreas Bulling, David Lindlbauer, Martin Schuessler, and Jörg Müller. 2015. Analyzing Visual Attention During Whole Body Interaction with Public Displays. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '15). ACM, New York, NY, USA, 1263--1267.
37. Niels Wouters, John Downs, Mitchell Harrop, Travis Cox, Eduardo Oliveira, Sarah Webber, Frank Vetere, and Andrew Vande Moere. 2016. Uncovering the Honeypot Effect: How Audiences Engage with Public Interactive Systems. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems (DIS '16). ACM, New York, NY, USA, 5--16.
38. Yanxia Zhang, Ken Pfeuffer, Ming Ki Chong, Jason Alexander, Andreas Bulling, and Hans Gellersen. 2017. Look Together: Using Gaze for Assisting Co-located Collaborative Search. Personal Ubiquitous Comput. 21, 1 (Feb. 2017), 173--186.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)