User experience and usage scenarios of audio-tactile interaction with virtual objects in a physical environment

ABSTRACT

Augmented reality (AR) systems enable new user experiences while the user is interacting with virtual objects in the physical space. The virtual objects have mostly been presented visually, overlaid on the physical world. In this paper, we present an explorative user study of a prototype system AHNE with the aim to understand the user experience and usage potential of non-visual audio-tactile augmented reality interactions. The qualitative results reveal that non-visual interaction was experienced confusing at first, but also fun, playful, and exciting. Even though audio feedback clearly dominates the spatial interaction experience, tactile feedback makes the virtual objects feel more concrete. The ideated and evaluated set of 23 usage scenarios includes both game-like and pragmatic application ideas for individual and social usage. The results can be used to inform the design of novel AR applications that utilise human senses beyond vision.

References

1. Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S. and Macintyre, B. Recent advances in augmented reality. Computer Graphics and Applications, IEEE, 21, 6 (2001), 34--47.
2. Benko, H. and Wilson, A. D. Multi-point interactions with immersive omnidirectional visualizations in a dome. Proc. ITS'10 ACM, (2010), 19--28.
3. Carroll, J. M. Five reasons for scenario-based design. Interacting with Computers 13, 1 (2000), 43--60.
4. Dourish, P. Where the action is: The foundations of embodied interaction. MIT press (2004).
5. Hunt, A. and Hermann, T. The importance of interaction in sonification. Proc. International Conference on Auditory Display (ICAD), (2004).
6. ISO 9241-210: 2010. Ergonomics of human system interaction-Part 210: Human-centred design for interactive systems. Switzerland International Standardization Organization (2010).
7. Iwamoto, T., Tatezono, M., Hoshi, T. and Shinoda, H. Airborne ultrasound tactile display. Proc. ACM SIGGRAPH 2008 new tech demos ACM, (2008).
8. Johnson, M. The body in the mind: The bodily basis of meaning, imagination, and reason. University of Chicago Press (1987).
9. Kildal, J. 3D-Press: Haptic Illusion of compliance when pressing on a rigid surface. Proc.ICMI'10 ACM, (2010).
10. Lai, C.-H., Niinimaki, M., Tahiroglu, K., Kildal, J. and Ahmaniemi, T. Perceived physicality in audio-enhanced force input. Proc. ICMI'11 ACM, (2011), 287--294.
11. Looser, J., Billinghurst, M. and Cockburn, A. Through the looking glass: The use of lenses as an interface tool for Augmented Reality interfaces. Proc. GRAPHITE'04 ACM, (2004), 204--211.
12. Molyneaux, D., Gellersen, H., Kortuem, G. and Schiele, B. Cooperative augmentation of smart objects with projector-camera systems. Proc. Ubicomp'07, Springer-Verlag, (2007), 501--518.
13. Nigay, L., Salembier, P., Marchand, T., Renevier, P. and Pasqualetti, L. Mobile and collaborative augmented reality: A scenario based design approach. MobileHCI'02, Springer-Verlag, (2002), 24--255.
14. Oron-Gilad, T., Downs, J. L., Gilson, R. D. and Hancock, P. A. Vibrotactile guidance cues for target acquisition. Systems, Man, and Cybernetics: Applications and Reviews, IEEE Transactions 37,5 (2007), 993--1004.
15. Parente, P. and Bishop, G. BATS: the blind audio tactile mapping system. Proc. of the ACM Southeast Regional Conference (2003).
16. Petrelli, D. and Not, E. User-centred design of flexible hypermedia for a mobile guide: Reflections on the HyperAudio experience. User Modeling and User-Adapted Interaction 15, 3-4 (2005), 303--338.
17. Pick, H. L., Warren, D. H. and Hay, J. C. Sensory conflict in judgments of spatial direction. Perception & Psychophysics 6, 4 (1969), 203--205.
18. Reponen, E., Koponen, T., Keränen, J. and Väänänen-Vainio-Mattila, K. The whole world under your feet: field trial of embodied browsing of geotagged content. Proc. TEI'12 ACM, (2012), 291--298.
19. Rma, A. et al. Augmented reality audio for mobile and wearable appliances. J. Audio Eng. Soc 52, 6 (2004).
20. Spelmezan, D. An investigation into the use of tactile instructions in snowboarding. Proc. MobileHCI'12, ACM, (2012), 417--426.
21. Tahiroğlu, K., Kildal, J., Ahmaniemi, T., Overstall, S. and Wikström, V. Embodied interactions with audio-tactile virtual objects in AHNE. Proc. HAID'12 Springer, (2012), 101--110.
22. Takacs, G. et al. Outdoors augmented reality on mobile phone using loxel-based visual feature organization. Proc. MIR'08 ACM, (2008), 427--434.
23. Tomitsch, M., Schlögl, R., Grechenig, T., Wimmer, C. and Költringer, T. Accessible real-world tagging through audio-tactile location markers. Proc. NordiCHI'08, ACM, (2008), 551--554.
24. Vazquez-Alvarez, Y., Oakley, I. and Brewster, S. Auditory display design for exploration in mobile audio-augmented reality. Personal and Ubiquitous Computing, 16, 8 (2011), 1--13.
25. Wakkary, R. and Hatala, M. Situated play in a tangible interface and adaptive audio museum guide. Personal and Ubiquitous Computing 11, 3 (2007), 171--191.
26. Wang, Q., Levesque, V., Pasquero, J., and Hayward, V. A haptic memory game using the STRESS 2 tactile display. Proc. CHI'06 EA, ACM, (2006), 271--274.

• Computing methodologies

  • Computer graphics

    • Graphics systems and interfaces

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction paradigms