PuPoP: Pop-up Prop on Palm for Virtual Reality

ABSTRACT

The sensation of being able to feel the shape of an object when grasping it in Virtual Reality (VR) enhances a sense of presence and the ease of object manipulation. Though most prior works focus on force feedback on fingers, the haptic emulation of grasping a 3D shape requires the sensation of touch using the entire hand. Hence, we present Pop-up Prop on Palm (PuPoP), a light-weight pneumatic shape-proxy interface worn on the palm that pops several airbags up with predefined primitive shapes for grasping. When a user's hand encounters a virtual object, an airbag of appropriate shape, ready for grasping, is inflated by way of the use of air pumps; the airbag then deflates when the object is no longer in play. Since PuPoP is a physical prop, it can provide the full sensation of touch to enhance the sense of realism for VR object manipulation. For this paper, we first explored the design and implementation of PuPoP with multiple shape structures. We then conducted two user studies to further understand its applicability. The first study shows that, when in conflict, visual sensation tends to dominate over touch sensation, allowing a prop with a fixed size to represent multiple virtual objects with similar sizes. The second study compares PuPoP with controllers and free-hand manipulation in two VR applications. The results suggest that utilization of dynamically-changing PuPoP, when grasped by users in line with the shapes of virtual objects, enhances enjoyment and realism. We believe that PuPoP is a simple yet effective way to convey haptic shapes in VR.

References

1. 2018. Cybergrasp, CyberGlove Systems Inc. http://www.cyberglovesystems.com/cybergrasp/. (2018). Accessed: March 19, 2018.
2. Mahdi Azmandian, Mark Hancock, Hrvoje Benko, Eyal Ofek, and Andrew D. Wilson. 2016. Haptic Retargeting: Dynamic Repurposing of Passive Haptics for Enhanced Virtual Reality Experiences. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 1968--1979.
3. Y. Ban, T. Kajinami, T. Narumi, T. Tanikawa, and M. Hirose. 2012a. Modifying an identified curved surface shape using pseudo-haptic effect. In 2012 IEEE Haptics Symposium (HAPTICS). 211--216.
4. Yuki Ban, Takuji Narumi, Tomohiro Tanikawa, and Michitaka Hirose. 2012b. Modifying an Identified Position of Edged Shapes Using Pseudo-haptic Effects. In Proceedings of the 18th ACM Symposium on Virtual Reality Software and Technology (VRST '12). ACM, New York, NY, USA, 93--96.
5. Hrvoje Benko, Christian Holz, Mike Sinclair, and Eyal Ofek. 2016. NormalTouch and TextureTouch: High-fidelity 3D Haptic Shape Rendering on Handheld Virtual Reality Controllers. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). ACM, New York, NY, USA, 717--728.
6. M. Bouzit, G. Burdea, G. Popescu, and R. Boian. 2002. The Rutgers Master II-new design force-feedback glove. IEEE/ASME Transactions on Mechatronics 7, 2 (June 2002), 256--263.
7. Brent Edward Insko. 2001. Passive haptics significantly enhances virtual environments. Ph.D. Dissertation. http://www.cs.unc.edu/techreports/01-017.pdf
8. Lung-Pan Cheng, Eyal Ofek, Christian Holz, Hrvoje Benko, and Andrew D. Wilson. 2017. Sparse Haptic Proxy: Touch Feedback in Virtual Environments Using a General Passive Prop. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 3718--3728.
9. Inrak Choi, Heather Culbertson, Mark R. Miller, Alex Olwal, and Sean Follmer. 2017. Grabity: A Wearable Haptic Interface for Simulating Weight and Grasping in Virtual Reality. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (UIST '17). ACM, New York, NY, USA, 119--130.
10. Inrak Choi and Sean Follmer. 2016. Wolverine: A Wearable Haptic Interface for Grasping in VR. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16 Adjunct). ACM, New York, NY, USA, 117--119.
11. Neil A. Dodgson. 2004. Variation and extrema of human interpupillary distance. In Stereoscopic Displays and Virtual Reality Systems XI, Vol. 5291. International Society for Optics and Photonics, 36--47.
12. Sean Follmer, Daniel Leithinger, Alex Olwal, Nadia Cheng, and Hiroshi Ishii. 2012. Jamming user interfaces: programmable particle stiffness and sensing for malleable and shape-changing devices. In Proceedings of the 25th annual ACM symposium on User interface software and technology. ACM, 519--528.
13. Eisuke Fujinawa, Shigeo Yoshida, Yuki Koyama, Takuji Narumi, Tomohiro Tanikawa, and Michitaka Hirose. 2017. Computational Design of Hand-held VR Controllers Using Haptic Shape Illusion. In Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology (VRST '17). ACM, New York, NY, USA, 28:1--28:10.
14. James J. Gibson. 1986. The theory of affordances. The Ecological Approach To Visual Perception (1986), 127--143.
15. Chris Harrison and Scott E. Hudson. 2009. Providing Dynamically Changeable Physical Buttons on a Visual Display. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). ACM, New York, NY, USA, 299--308.
16. Fabian Hemmert, Susann Hamann, Matthias Löwe, Anne Wohlauf, and Gesche Joost. 2010a. Shape-changing Mobiles: Tapering in One-dimensional Deformational Displays in Mobile Phones. In Proceedings of the Fourth International Conference on Tangible, Embedded, and Embodied Interaction (TEI '10). ACM, New York, NY, USA, 249--252.
17. Fabian Hemmert, Susann Hamann, Matthias Löwe, Anne Wohlauf, Josefine Zeipelt, and Gesche Joost. 2010b. Take Me by the Hand: Haptic Compasses in Mobile Devices Through Shape Change and Weight Shift. In Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries (NordiCHI '10). ACM, New York, NY, USA, 671--674.
18. Anuruddha Hettiarachchi and Daniel Wigdor. 2016. Annexing Reality: Enabling Opportunistic Use of Everyday Objects As Tangible Proxies in Augmented Reality. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 1957--1967.
19. Ken Hinckley, Randy Pausch, John C. Goble, and Neal F. Kassell. 1994. Passive Real-world Interface Props for Neurosurgical Visualization. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '94). ACM, New York, NY, USA, 452--458.
20. H. G. Hoffman. 1998. Physically touching virtual objects using tactile augmentation enhances the realism of virtual environments. In Proceedings. IEEE 1998 Virtual Reality Annual International Symposium (Cat. No. 98CB36180). 59--63.
21. David Holman and Roel Vertegaal. 2008. Organic User Interfaces: Designing Computers in Any Way, Shape, or Form . Commun. ACM 51, 6 (June 2008), 48--55.
22. Da-Yuan Huang, Ruizhen Guo, Jun Gong, Jingxian Wang, John Graham, De-Nian Yang, and Xing-Dong Yang. 2017. RetroShape: Leveraging Rear-Surface Shape Displays for 2.5D Interaction on Smartwatches. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (UIST '17). ACM, New York, NY, USA, 539--551.
23. Hiroo Iwata, Hiroaki Yano, and Naoto Ono. 2005. Volflex. In ACM SIGGRAPH 2005 Emerging Technologies (SIGGRAPH '05). ACM, New York, NY, USA.
24. Seoktae Kim, Hyunjung Kim, Boram Lee, Tek-Jin Nam, and Woohun Lee. 2008. Inflatable Mouse: Volume-adjustable Mouse with Air-pressure-sensitive Input and Haptic Feedback. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '08). ACM, New York, NY, USA, 211--224.
25. E. Kwon, G. J. Kim, and S. Lee. 2009. Effects of sizes and shapes of props in tangible augmented reality. In 2009 8th IEEE International Symposium on Mixed and Augmented Reality. 201--202.
26. Li-Ke Ma, Yizhonc Zhang, Yang Liu, Kun Zhou, and Xin Tong. 2017. Computational Design and Fabrication of Soft Pneumatic Objects with Desired Deformations . ACM Trans. Graph. 36, 6 (Nov. 2017), 239:1--239:12.
27. C. L. MacKenzie and T. Iberall. 1994. The Grasping Hand . Elsevier.
28. Ryuma Niiyama, Xu Sun, Lining Yao, Hiroshi Ishii, Daniela Rus, and Sangbae Kim. 2015. Sticky Actuator: Free-Form Planar Actuators for Animated Objects. In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction (TEI '15). ACM, New York, NY, USA, 77--84.
29. Jifei Ou, Mélina Skouras, Nikolaos Vlavianos, Felix Heibeck, Chin-Yi Cheng, Jannik Peters, and Hiroshi Ishii. 2016. aeroMorph - Heat-sealing Inflatable Shape-change Materials for Interaction Design (UIST '16). ACM Press, 121--132.
30. Irvin Rock and Jack Victor. 1964. Vision and Touch: An Experimentally Created Conflict between the Two Senses . Science 143, 3606 (Feb. 1964), 594--596.
31. Harpreet Sareen, Udayan Umapathi, Patrick Shin, Yasuaki Kakehi, Jifei Ou, Hiroshi Ishii, and Pattie Maes. 2017. Printflatables: Printing Human-Scale, Functional and Dynamic Inflatable Objects. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 3669--3680.
32. Samuel B. Schorr and Allison M. Okamura. 2017. Fingertip Tactile Devices for Virtual Object Manipulation and Exploration. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 3115--3119.
33. Jia Sheng, Ravin Balakrishnan, and Karan Singh. 2006. An Interface for Virtual 3D Sculpting via Physical Proxy. In Proceedings of the 4th International Conference on Computer Graphics and Interactive Techniques in Australasia and Southeast Asia (GRAPHITE '06). ACM, New York, NY, USA, 213--220.
34. Adalberto L. Simeone, Eduardo Velloso, and Hans Gellersen. 2015. Substitutional Reality: Using the Physical Environment to Design Virtual Reality Experiences. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 3307--3316.
35. Timothy M. Simon, Ross T. Smith, and Bruce H. Thomas. 2014. Wearable Jamming Mitten for Virtual Environment Haptics. In Proceedings of the 2014 ACM International Symposium on Wearable Computers (ISWC '14). ACM, New York, NY, USA, 67--70.
36. Mélina Skouras, Bernhard Thomaszewski, Peter Kaufmann, Akash Garg, Bernd Bickel, Eitan Grinspun, and Markus Gross. 2014. Designing Inflatable Structures . ACM Trans. Graph. 33, 4 (July 2014), 63:1--63:10.
37. S. H. Winter and M. Bouzit. 2007. Use of Magnetorheological Fluid in a Force Feedback Glove . IEEE Transactions on Neural Systems and Rehabilitation Engineering 15, 1 (March 2007), 2--8.
38. Lining Yao, Ryuma Niiyama, Jifei Ou, Sean Follmer, Clark Della Silva, and Hiroshi Ishii. 2013. PneUI: pneumatically actuated soft composite materials for shape changing interfaces (UIST '13). ACM Press, 13--22.
39. A. Zenner and A. Krüger. 2017. Shifty: A Weight-Shifting Dynamic Passive Haptic Proxy to Enhance Object Perception in Virtual Reality . IEEE Transactions on Visualization and Computer Graphics 23, 4 (April 2017), 1285--1294.
40. Yiwei Zhao, Lawrence H. Kim, Ye Wang, Mathieu Le Goc, and Sean Follmer. 2017. Robotic Assembly of Haptic Proxy Objects for Tangible Interaction and Virtual Reality. In Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces (ISS '17). ACM, New York, NY, USA, 82--91.
41. Igor Zubrycki and Grzegorz Granosik. 2017. Novel Haptic Device Using Jamming Principle for Providing Kinaesthetic Feedback in Glove-Based Control Interface . Journal of Intelligent & Robotic Systems 85, 3--4 (March 2017), 413--429.

Index Terms

• Hardware

  • Communication hardware, interfaces and storage

    • Tactile and hand-based interfaces

      • Haptic devices

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction paradigms

      • Virtual reality