Understanding Visual-Haptic Integration of Avatar Hands Using a Fitts' Law Task in Virtual Reality

ABSTRACT

Virtual reality (VR) is becoming more and more ubiquitous to interact with digital content and often requires renderings of avatars as they enable improved spatial localization and high levels of presence. Previous work shows that visual-haptic integration of virtual avatars depends on body ownership and spatial localization in VR. However, there are different conclusions about how and which stimuli of the own appearance are integrated into the own body scheme. In this work, we investigate if systematic changes of model and texture of a users' avatar affect the input performance measured in a two-dimensional Fitts' law target selection task. Interestingly, we found that the throughput remained constant between our conditions and that neither model nor texture of the avatar significantly affected the average duration to complete the task even when participants felt different levels of presence and body ownership. In line with previous work, we found that the illusion of virtual limb-ownership does not necessarily correlate to the degree to which vision and haptics are integrated into the own body scheme. Our work supports findings indicating that body ownership and spatial localization are potentially independent mechanisms in visual-haptic integration.

References

1. Ferran Argelaguet, Ludovic Hoyet, Michaël Trico, and Anatole Lécuyer. 2016. The role of interaction in virtual embodiment: Effects of the virtual hand representation. In 2016 IEEE Virtual Reality (VR). 3--10.
2. Xiaojun Bi, Yang Li, and Shumin Zhai. 2013. FFitts Law: Modeling Finger Touch with Fitts' Law. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, New York, NY, USA, 1363--1372.
3. Matthew Botvinick and Jonathan Cohen. 1998. Rubber hands 'feel' touch that eyes see. 391 (1998).
4. Volkert Buchmann, Trond Nilsen, and Mark Billinghurst. 2005. Interaction with Partially Transparent Hands and Objects. In Proceedings of the Sixth Australasian Conference on User Interface - Volume 40 (AUIC '05). Australian Computer Society, Inc., Darlinghurst, Australia, Australia, 17--20. http://dl.acm.org/citation.cfm?id=1082243.1082246
5. Zhe Chen, Pei-Luen Patrick Rau, and Cuiling Chen. 2014. The Effects of Human Finger and Chinese Character on Chinese Handwriting Performance on Mobile Touch Devices. Human Factors 56, 3 (2014), 553--568.
6. James Joseph Clark and Alan L. Yuille. 1990. Data Fusion for Sensory Information Processing Systems. Kluwer Academic Publishers, Norwell, MA, USA.
7. Francesco Della Gatta, Francesca Garbarini, Guglielmo Puglisi, Antonella Leonetti, Annamaria Berti, and Paola Borroni. 2016. Decreased motor cortex excitability mirrors own hand disembodiment during the rubber hand illusion. eLife 5 (oct 2016).
8. H. Henrik Ehrsson, Nicholas P Holmes, and Richard E Passingham. 2005. Touching a rubber hand: feeling of body ownership is associated with activity in multisensory brain areas. The Journal of neuroscience: the official journal of the Society for Neuroscience 25, 45 (nov 2005), 10564--73.
9. Marc O. Ernst and Martin S. Banks. 2002. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415, 6870 (jan 2002), 429--433.
10. Marc O. Ernst and Heinrich H. Bülthoff. 2004. Merging the senses into a robust percept. Trends in Cognitive Sciences 8, 4 (apr 2004), 162--169.
11. Paul M Fitts. 1954. The information capacity of the human motor system in controlling the amplitude of movement. Journal of experimental psychology 47, 6 (1954), 381.
12. Niels Henze, Markus Funk, and Alireza Sahami Shirazi. 2016. Software-reduced Touchscreen Latency. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '16). ACM, New York, NY, USA, 434--441.
13. James M. Hillis, Simon J. Watt, Michael S. Landy, and Martin S. Banks. 2004. Slant from texture and disparity cues: Optimal cue combination. Journal of Vision 4, 12 (12 2004), 1--1.
14. ISO. 2012. 9241--411 Ergonomics of human-system interaction-Part 411: Evaluation methods for the design of physical input devices. International Organization for Standardization (2012).
15. Todd Johnsgard. 1994. Fitts' Law with a Virtual Reality Glove and a Mouse: Effects of Gain. In Proceedings of Graphics Interface '94 (GI '94). Canadian Human-Computer Communications Society, Toronto, Ontario, Canada, 8--15. http://graphicsinterface.org/wp-content/uploads/gi1994-2.pdf
16. Sungchul Jung, Gerd Bruder, Pamela J. Wisniewski, Christian Sandor, and Charles E. Hughes. 2018. Over My Hand: Using a Personalized Hand in VR to Improve Object Size Estimation, Body Ownership, and Presence. In Proceedings of the Symposium on Spatial User Interaction (SUI '18). ACM, New York, NY, USA, 60--68.
17. D. B. Kaber, Y. Li, M. Clamann, and Y. Lee. 2012. Investigating Human Performance in a Virtual Reality Haptic Simulator as Influenced by Fidelity and System Latency. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans 42, 6 (Nov 2012), 1562--1566.
18. Andreas Kalckert and H. Henrik Ehrsson. 2014. The moving rubber hand illusion revisited: Comparing movements and visuotactile stimulation to induce illusory ownership. Consciousness and Cognition 26 (2014), 117--132.
19. Konstantina Kilteni, Raphaela Groten, and Mel Slater. 2012. The Sense of Embodiment in Virtual Reality. Presence: Teleoperators and Virtual Environments 21, 4 (2012), 373--387.
20. Pascal Knierim, Valentin Schwind, Anna Maria Feit, Florian Nieuwenhuizen, and Niels Henze. 2018. Physical Keyboards in Virtual Reality: Analysis of Typing Performance and Effects of Avatar Hands, In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (2018-04-21). Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems.
21. Michael S. Landy, Laurence T. Maloney, Elizabeth B. Johnston, and Mark Young. 1995. Measurement and modeling of depth cue combination: in defense of weak fusion. Vision Research 35, 3 (1995), 389--412.
22. Huy Viet Le, Valentin Schwind, Philipp GÃűttlich, and Niels Henze. 2017. PredicTouch: A System to Reduce Touchscreen Latency using Neural Networks and Inertial Measurement Units. In Proceedings of the Interactive Surfaces and Spaces on ZZZ - ISS '17 (2017-01-01). ACM Press, New York, New York, USA, 230--239.
23. Lorraine Lin and Sophie Jörg. 2016. Need a hand? How Appearance Affects the Virtual Hand Illusion. In Proceedings of the ACM Symposium on Applied Perception - SAP '16. ACM Press, New York, New York, USA, 69--76.
24. Jean-Luc Lugrin, Johanna Latt, and Marc Erich Latoschik. 2015. Avatar anthropomorphism and illusion of body ownership in VR. In 2015 IEEE Virtual Reality (VR), Masataka Imura, Pablo Figueroa, and Betty Mohler (Eds.). IEEE, 229--230.
25. Ke Ma and Bernhard Hommel. 2015. Body-ownership for actively operated non-corporeal objects. Consciousness and Cognition 36 (nov 2015), 75--86.
26. I. Scott MacKenzie and William Buxton. 1992. Extending Fitts' Law to Two-dimensional Tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '92). ACM, New York, NY, USA, 219--226.
27. I. Scott MacKenzie and Poika Isokoski. 2008. Fitts' Throughput and the Speed-accuracy Tradeoff. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '08). ACM, New York, NY, USA, 1633--1636.
28. Andrea H. Mason, Masuma A. Walji, Elaine J. Lee, and Christine L. MacKenzie. 2001. Reaching Movements to Augmented and Graphic Objects in Virtual Environments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '01). ACM, New York, NY, USA, 426--433.
29. Kazumichi Matsumiya. 2019. Separate multisensory integration processes for ownership and localization of body parts. Scientific Reports 9 (12 2019).
30. Sven Mayer, Valentin Schwind, Robin Schweigert, and Niels Henze. 2018. The Effect of Offset Correction and Cursor on Mid-Air Pointing in Real and Virtual Environments. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, Article 653, 13 pages.
31. Mike McGee, Brian Amento, Patrick Brooks, and Hope Harley. 1997. Fitts and VR: Evaluating Display and Input Devices with Fitts' Law. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 41, 2 (1997), 1259--1262.
32. Masahiro Mori, Karl F. MacDorman, and Norri Kageki. 2012. The uncanny valley. IEEE Robotics and Automation Magazine 19, 2 (jun 2012), 98--100.
33. Kristine L. Nowak and Frank Biocca. 2003. The Effect of the Agency and Anthropomorphism on Users' Sense of Telepresence, Copresence, and Social Presence in Virtual Environments. Presence: Teleoperators and Virtual Environments 12, 5 (2003), 481--494.
34. Sarah Prange, Daniel Buschek, and Florian Alt. 2018. An Exploratory Study on Correlations of Hand Size and Mobile Touch Interactions. In Proceedings of the 17th International Conference on Mobile and Ubiquitous Multimedia (MUM 2018). ACM, New York, NY, USA, 279--283.
35. Pedro Rosas, Felix A. Wichmann, and Johan Wagemans. 2007. Texture and object motion in slant discrimination: Failure of reliability-based weighting of cues may be evidence for strong fusion. Journal of Vision 7, 6 (04 2007), 3--3.
36. Valentin Schwind, Pascal Knierim, Lewis Chuang, and Niels Henze. 2017. "Where's Pinky?": The Effects of a Reduced Number of Fingers in Virtual Reality. In Proceedings of the 2017 CHI Conference on Computer-Human Interaction in Play (CHI PLAY'17). ACM, New York, NY, USA.
37. Valentin Schwind, Pascal Knierim, Nico Haas, and Niels Henze. 2019. Using Presence Questionnaires in Virtual Reality. In CHI Conference on Human Factors in Computing Systems Proceedings (CHI 2019) (2019-05-04). ACM, Glasgow, Scotland Uk, 12.
38. Valentin Schwind, Pascal Knierim, Cagri Tasci, Patrick Franczak, Nico Haas, and Niels Henze. 2017. "These Are Not My Hands!": Effect of Gender on the Perception of Avatar Hands in Virtual Reality. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 1577--1582.
39. Valentin Schwind, Lorraine Lin, Massimiliano Di Luca, Sophie Jörg, and James Hillis. 2018. Touch with Foreign Hands: The Effect of Virtual Hand Appearance on Visual-Haptic Integration. In Proceedings of the ACM Symposium on Applied Perception (SAP '18) (2018-08-03). ACM, New York, NY.
40. Valentin Schwind, Sven Mayer, Alexandre Comeau-Vermeersch, Robin Schweigert, and Niels Henze. 2018. Up to the Finger Tip: The Effect of Avatars on Mid-Air Pointing Accuracy in Virtual Reality. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '18). ACM, New York, NY, USA, 477--488.
41. Andrea Serino, Adrian Alsmith, Marcello Costantini, Alisa Mandrigin, Ana Tajadura-Jimenez, and Christophe Lopez. 2013. Bodily ownership and self-location: Components of bodily self-consciousness. Consciousness and Cognition 22, 4 (2013), 1239--1252.
42. Mel Slater, Vasilis Linakis, Martin Usoh, and Rob Kooper. 1996. Immersion, Presence and Performance in Virtual Environments: An Experiment with Tri-dimensional Chess. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology (VRST '96). ACM, New York, NY, USA, 163--172.
43. Mel Slater, Daniel Pérez Marcos, Henrik Ehrsson, and Maria Sanchez-Vives. 2009. Inducing illusory ownership of a virtual body. Frontiers in Neuroscience 3 (2009), 29.
44. Mel Slater, Daniel Pérez Marcos, and H. Henrik Ehrsson. 2009. Inducing illusory ownership of a virtual body. Frontiers in neuroscience 3, 2 (Sep 2009), 214--220.
45. Mel Slater, Bernhard Spanlang, Maria V. Sanchez-Vives, and Olaf Blanke. 2010. First Person Experience of Body Transfer in Virtual Reality. PLOS ONE 5, 5 (2010), 1--9.
46. Bruce H. Thomas. 2012. A Survey of Visual, Mixed, and Augmented Reality Gaming. Comput. Entertain. 10, 1, Article 3 (Dec. 2012), 33 pages.
47. Manos Tsakiris, Gita Prabhu, and Patrick Haggard. 2006. Having a body versus moving your body: How agency structures body-ownership. Consciousness and Cognition 15, 2 (2006), 423--432.
48. Vinoba Vinayagamoorthy and Andrea Brogni. 2004. An Investigation of Presence Response across Variations in Visual Realism. Proceedings of Presence 2004: The 7th Annual International Workshop on Presence (2004), 119--126.
49. Eric-Jan Wagenmakers. 2007. A practical solution to the pervasive problems of p-values. Psychonomic Bulletin & Review 14, 5 (01 Oct 2007), 779--804.
50. Jacob O. Wobbrock, Leah Findlater, Darren Gergle, and James J. Higgins. 2011. The Aligned Rank Transform for Nonparametric Factorial Analyses Using Only Anova Procedures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 143--146.
51. Ye Yuan and Anthony Steed. 2010. Is the rubber hand illusion induced by immersive virtual reality?. In 2010 IEEE Virtual Reality Conference (VR). IEEE, 95--102.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • HCI design and evaluation methods

      • User studies

    • Interaction paradigms

      • Virtual reality