ImpAct: Immersive haptic stylus to enable direct touch and manipulation for surface computing

Abstract

This article explores direct touch and manipulation techniques for surface computing environments using a specialized haptic force feedback stylus, called ImpAct, which can dynamically change its effective length and equipped with sensors to calculate its orientation in world coordinates. When a user pushes it against a touch screen, the physical stylus shrinks and a rendered projection of the stylus is drawn inside the screen, giving the illusion that it is submerged in the display device. Once the users can see the stylus immersed in the digital world below the screen, he or she can manipulate and interact with the virtual objects with active haptic sensations. In this article, ImpAct's functionality, design, and prototype applications are described in detail with relevance to the concept of direct touch, giving special attention to novel interaction scenarios and design challenges. Furthermore, a technical evaluation was done to study ImpAct's accuracy and controlability and the results presented. This article concludes by discussing ImpAct's current limitations and future perspectives as a direct touch and manipulation tool.

References

1. Ando, H., Miki, T., Inami, M., and Maeda. T. 2002. The nail-mounted tactile display for the behavior modeling. In Proceedings of the ACM SIGGRAPH 2002 Conference. Abstracts and Applications (SIGGRAPH'02), ACM, New York, 264--264.
2. Bi, X., Moscovich, T., Ramos, G., Balakrishnan, R., and Hinckley, K. 2008. An exploration of pen rolling for pen-based interaction. In Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology (UIST'08), ACM, New York, 191--200.
3. Driver, J. and Spence, C. 1998. Attention and the crossmodal construction of space. Trends Cognitive Sci. 2, 7, 254--262.
4. Fitzmaurice, G. W., Ishii, H., and Buxton, W. A. S. 1995. Bricks: Laying the foundations for graspable user interfaces. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI '95), ACM, New York, 442--449.
5. Gray, R. and Tan, H. Z. 2002. Dynamic and predictive links between touch and vision. Experimental Brain Res. 145, 1, 50--55.
6. Han, J. Y. 2005. Low-cost multi-touch sensing through frustrated total internal reflection. In Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology (UIST'05), ACM, New York, 145.
7. Hirsch, M., Lanman, D., Holtzman, H., and Raskar, R. 2009. BiDi screen: A thin, depth-sensing LCD for 3D interaction using light fields. In Proceedings SIGGRAPH Asia'09, ACM, New York. 1--9.
8. Kamuro, S., Minamizawa, K., Kawakami, N., and Tachi. S. 2009. Pen de touch. In Proceedings of the ACM SIGGRAPH 2009 Emerging Technologies Conference, ACM, New York, 1--1.
9. Kennett, S., Taylor-Clarke, M., and Haggard, P. 2001. Noninformative vision improves the spatial resolution of touch in humans. Current Biol. 11, 15, 1188--1191.
10. Kyung, K.-U. and Lee, J.-Y. 2008. Wubi-pen: Windows graphical user interface interacting with haptic feedback stylus. In Proceedings of the ACM SIGGRAPH 2008 New Tech Demos, ACM, New York, 1--4.
11. Lapides, P., Sharlin, E., Sousa, M. C., and Streit, L. 2006. The 3d tractus: A three-dimensional drawing board. In Proceedings of the First IEEE International Workshop on Horizontal Interactive Human-Computer Systems (TABLETOP'06), IEEE. Washington, D.C., 169--176.
12. Lee, J. C., Dietz, P. H., Leigh, D. W. S., Yerazunis, and Hudson, S. E. 2004. Haptic pen: A tactile feedback stylus for touch screens. In Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology (UIST'04), ACM, New York, 291--294.
13. Lee, S., Buxton, W., and Smith, K. C. 1985. A multi-touch three-dimensional touch-sensitive tablet. SIGCHI Bull. 16, 4, 21--25.
14. Massie, T. H. 1996. Initial haptic explorations with the phantom: Virtual touch through point interaction. MS dissertation, MIT, Cambridge, MA.
15. Minsky, M., Ming, O.-y., Steele, O., Brooks, F. P., Jr., and M. Behensky, M. 1990. Feeling and seeing: Issues in force display. SIGGRAPH Comput. Graph. 24, 2, 235--241.
16. Minsky, M. D. R. 1995. Computational haptics: The Sandpaper system for synthesizing texture for a force-feedback display. PhD dissertation, MIT, Cambridge, MA.
17. Polanyim, M. 1959. Personal Knowledge: Towards a Post-Critical Philosophy. University of Chicago Press, Chicago, IL.
18. Ramos, G., Boulos, M., and Balakrishnan, R. 2004. Pressure widgets. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'04), ACM, New York, 487--494.
19. Rekimoto, J. 2002. Smartskin: An infrastructure for freehand manipulation on interactive surfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'02), ACM, New York, 113--120.
20. Sato, T., Mamiya, H., Koike, and Fukuchi, K. 2009. Photoelastictouch: Transparent rubbery tangible interface using an and photoelasticity. In Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology (UIST'09), ACM, New York, 43--50.
21. Sinclair, M. 1997. The haptic lens. In ACM SIGGRAPH 97: Visual Proceedings, ACM, New York, 179.
22. So, E., Zhang, H., and Sheng Guan, Y. 1999. Sensing contact with analog resistive technology. In Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, vol. 2, IEEE, Washington, D.C., 806--811.
23. Suzuki, Y., Misue, K., and Tanaka, J. 2007. Stylus enhancement to enrich interaction with computers. In Proceedings of the 12th International Conference on Human-Computer Interaction (HCI'07), Springer, Berlin, 133--142.
24. Tian, F., Xu, L., Wang, H., Zhang, X., Liu, Y., Setlur, V., and Dai, G. 2008. Tilt menu: Using the 3d orientation information of pen devices to extend the selection capability of pen-based user interfaces. In Proceedings of the 26th Annual SIGCHI Conference on Human Factors in Computing Systems (CHI'08), ACM, New York, 1371--1380.
25. Wang, F., Cao, X., Ren, X., and Irani, P. 2009. Detecting and leveraging finger orientation for interaction with direct-touch surfaces. In Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology (UIST'09), ACM, New York, 23--32.
26. Wang, F. and Ren, X. 2009. Empirical evaluation for finger input properties in multi-touch interaction. In Proceedings of the 27th International Conference on Human Factors in Computing Systems (CHI'09), ACM, New York, 1063--1072
27. Wilson, A. D., Izadi, S., Hilliges, O., Garcia-Mendoza, A., and Kirk, D. 2008. Bringing physics to the surface. In Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology (UIST'08), ACM, New York, 67--76.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction devices

      • Haptic devices