Papier-Mache: toolkit support for tangible input

ABSTRACT

Tangible user interfaces (TUIs) augment the physical world by integrating digital information with everyday physical objects. Currently, building these UIs requires "getting down and dirty" with input technologies such as computer vision. Consequently, only a small cadre of technology experts can currently build these UIs. Based on a literature review and structured interviews with nine TUI researchers, we created Papier-Mâché, a toolkit for building tangible interfaces using computer vision, electronic tags, and barcodes. Papier-Mache introduces a high-level event model for working with these technologies that facilitates technology portability. For example, an application can be prototyped with computer vision and deployed with RFID. We present an evaluation of our toolkit with six class projects and a user study with seven programmers, finding the input abstractions, technology portability, and monitoring window to be highly effective.

REFERENCES

• 1: Ballagas, R., M. Ringel, et al., iStuff: a physical user interface toolkit for ubiquitous computing environments. Human Factors in Computing Systems, CHI Letters, 2003. 5(1): pp. 537 -- 44.
• 2: Card, S.K., T.P. Moran, and A. Newell, Chapter 2: The Human Information Processor, in The Psychology of Human-Computer Interaction, Lawrence Erlbaum: Hillsdale. pp. 23 -- 97, 1983.
• 3: Paul Clements , Rick Kazman, Software Architecture in Practices, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 2003
• 4: Françoise Détienne , Frank Bott, Software design---cognitive aspects, Springer-Verlag New York, Inc., New York, NY, 2001
• 5: Dey, A.K., D. Salber, and G.D. Abowd, A Conceptual Framework and a Toolkit for Supporting the Rapid Prototyping of Context-Aware Applications. Human-Computer Interaction, 2001. 16(2-4): pp. 97 -- 166.
• 6: Fails, J.A. and D.R. Olsen, A Design Tool for Camera-based Interaction. Human Factors in Computing Systems, CHI Letters, 2003. 5(1): pp. 449 -- 56.
• 7: Kenneth P. Fishkin , Thomas P. Moran , Beverly L. Harrison, Embodied User Interfaces: Towards Invisible User Interfaces, Proceedings of the IFIP TC2/TC13 WG2.7/WG13.4 Seventh Working Conference on Engineering for Human-Computer Interaction, p.1-18, September 14-18, 1998
• 8: David A. Forsyth , Jean Ponce, Computer Vision: A Modern Approach, Prentice Hall Professional Technical Reference, 2002
• 9: William T. Freeman , David B. Anderson , Paul A. Beardsley , Chris N. Dodge , Michal Roth , Craig D. Weissman , William S. Yerazunis , Hiroshi Kage , Kazuo Kyuma , Yasunari Miyake , Ken-ichi Tanaka, Computer Vision for Interactive Computer Graphics, IEEE Computer Graphics and Applications, v.18 n.3, p.42-53, May 1998 [doi>10.1109/38.674971 ]
• 10: Robert B. Grady, Practical software metrics for project management and process improvement, Prentice-Hall, Inc., Upper Saddle River, NJ, 1992
• 11: Greenberg, S. and C. Fitchett, Phidgets: easy development of physical interfaces through physical widgets. User Interface Software & Technology, CHI Letters, 2001. 3(2): pp. 209 -- 18.
• 12: Heiner, J.M., S.E. Hudson, and K. Tanaka, Linking and messaging from real paper in the paper PDA. User Interface Software & Technology, CHI Letters, 1999. 1(1): pp. 179 -- 86.
• 13: Berthold K. Horn, Robot Vision, McGraw-Hill Higher Education, 1986
• 14: Hiroshi Ishii , Brygg Ullmer, Tangible bits: towards seamless interfaces between people, bits and atoms, Proceedings of the SIGCHI conference on Human factors in computing systems, p.234-241, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258715]
• 15: Kato, H., M. Billinghurst, and I. Poupyrev. ARToolKit. University of Washington HIT Lab, 2000. http://www.hitl.washington.edu/artoolkit/
• 16: Klemmer, S.R., J. Graham, G.J. Wolff, and J.A. Landay, Books with Voices: Paper Transcripts as a Tangible Interface to Oral Histories. Human Factors in Computing Systems, CHI Letters, 2003. 5(1): pp. 89 -- 96.
• 17: Klemmer, S.R., M.W. Newman, R. Farrell, M. Bilezikjian, and J.A. Landay, The Designers' Outpost: A Tangible Interface for Collaborative Web Site Design. User Interface Software and Technology, CHI Letters, 2001. 3(2): pp. 1 -- 10.
• 18: Wendy E. Mackay , Anne-Laure Fayard , Laurent Frobert , Lionel Médini, Reinventing the familiar: exploring an augmented reality design space for air traffic control, Proceedings of the SIGCHI conference on Human factors in computing systems, p.558-565, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274719]
• 19: Mankoff, J., S.E. Hudson, and G.D. Abowd, Providing Integrated Toolkit-Level Support for Ambiguity in Recognition-Based Interfaces. Human Factors in Computing Systems, CHI Letters, 2000. 2(1): pp. 368 -- 375.
• 20: McGee, D.R., P.R. Cohen, R.M. Wesson, and S. Horman, Comparing paper and tangible, multimodal tools. Human Factors in Computing Systems, CHI Letters, 2002. 4(1): pp. 407 -- 414.
• 21: Moran, T.P., E. Saund, W. van Melle, A. Gujar, et al., Design and Technology for Collaborage: Collaborative Collages of Information on Physical Walls. UIST: User Interface Software and Technology, CHI Letters, 1999. 1(1): pp. 197 -- 206.
• 22: Brad Myers , Scott E. Hudson , Randy Pausch, Past, present, and future of user interface software tools, ACM Transactions on Computer-Human Interaction (TOCHI), v.7 n.1, p.3-28, March 2000 [doi>10.1145/344949.344959]
• 23: Brad A. Myers , Mary Beth Rosson, Survey on user interface programming, Proceedings of the SIGCHI conference on Human factors in computing systems, p.195-202, May 03-07, 1992, Monterey, California, United States [doi>10.1145/142750.142789]
• 24: Brad A. Myers, A new model for handling input, ACM Transactions on Information Systems (TOIS), v.8 n.3, p.289-320, July 1990 [doi>10.1145/98188.98204]
• 25: Les Nelson , Satoshi Ichimura , Elin Rønby Pedersen , Lia Adams, Palette: a paper interface for giving presentations, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.354-361, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303109]
• 26: Dan R. Olsen, Jr., Developing user interfaces, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1998
• 27: John Francis Pane , Brad A. Myers , David Garlan, A programming system for children that is designed for usability, 2002
• 28: Jun Rekimoto , Yuji Ayatsuka, CyberCode: designing augmented reality environments with visual tags, Proceedings of DARE 2000 on Designing augmented reality environments, p.1-10, April 2000, Elsinore, Denmark [doi>10.1145/354666.354667]
• 29: Ben Shneiderman, Empirical studies of programmers: the territory, paths, and destination, Papers presented at the first workshop on empirical studies of programmers on Empirical studies of programmers, p.1-12, June 1986, Washington, D.C., United States
• 30: Ullmer, B. and H. Ishii, Emerging Frameworks for Tangible User Interfaces, in Human-Computer Interaction in the New Millennium, Addison-Wesley. pp. 579 -- 601, 2001.
• 31: Roy Want , Kenneth P. Fishkin , Anuj Gujar , Beverly L. Harrison, Bridging physical and virtual worlds with electronic tags, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.370-377, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303111]

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.2 Design Tools and Techniques
          Subjects: Software libraries

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.2 Design Tools and Techniques
          Subjects: User interfaces

  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Artificial, augmented, and virtual realities
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation); User-centered design; Prototyping; Input devices and strategies (e.g., mouse, touchscreen)

  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.9 Applications

Keywords:
API design, RFID, augmented reality, barcode, computer vision, tangible interfaces, toolkits