Design and implementation of a built-in camera based user interface for mobile games

ABSTRACT

In this paper, we present a novel user interface design for mobile games using its built-in camera. It is motivated by the problem of limited number of keys on the small keypad of mobile devices. We implement two different ways, with and without the use of markers, to study the effectiveness of the design in a 2-dimensional and 3-dimensional game respectively. The algorithms implemented in the design are lightweight enough so that they are executable in real-time and at the same time, able to produce realism for the games.

REFERENCES

• 1: J. L. Barron , D. J. Fleet , S. S. Beauchemin, Performance of optical flow techniques, International Journal of Computer Vision, v.12 n.1, p.43-77, Feb. 1994 [doi>10.1007/BF01420984]
• 2: Beverly L. Harrison , Kenneth P. Fishkin , Anuj Gujar , Carlos Mochon , Roy Want, Squeeze me, hold me, tilt me! An exploration of manipulative user interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems, p.17-24, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274647]
• 3: M. Möhring, C. Lessig, and O. Bimber, Optical tracking and video see-through AR on consumer cell phones, in: Proc. of Workshop on Virtual and Augmented Reality of the GI-Fachgruppe AR/VR, 2004, pp. 193--204.
• 4: Jun Rekimoto, Tilting operations for small screen interfaces, Proceedings of the 9th annual ACM symposium on User interface software and technology, p.167-168, November 06-08, 1996, Seattle, Washington, United States [doi>10.1145/237091.237115]
• 5: M. Rohs, Real-world interaction with camera phones. In 2nd International Symposium on Ubiquitous Computing system (UCS 2004), Tokyo, Japan 2004., pp. 74--89.
• 6: Ben Shneiderman , Catherine Plaisant, Designing the User Interface: Strategies for Effective Human-Computer Interaction (4th Edition), Pearson Addison Wesley, 2004
• 7: Dave Shreiner , Mason Woo , Jackie Neider , Tom Davis, OpenGL(R) Programming Guide: The Official Guide to Learning OpenGL(R), Version 2 (5th Edition) (OpenGL), Addison-Wesley Professional, 2005
• 8: Luigi di Stefano , Andrea Bulgarelli, A Simple and Efficient Connected Components Labeling Algorithm, Proceedings of the 10th International Conference on Image Analysis and Processing, p.322, September 27-29, 1999
• 9: J. Vallino and K. N. Kutulakos, Augmented reality using affine object representations. Fundamental of Wearable Computers and Augmented Reality, Lawrence Erlbaum, 2001, pp.157--182.
• 10: Jingtao Wang , Shumin Zhai , John Canny, Camera phone based motion sensing: interaction techniques, applications and performance study, Proceedings of the 19th annual ACM symposium on User interface software and technology, October 15-18, 2006, Montreux, Switzerland [doi>10.1145/1166253.1166270]
• 11: P. D. Wellner, Adaptive thresholding for the DigitalDesk, Xerox Technical Report, vol. EPC-1993-110, 1993.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.4 Graphics Utilities

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.1 Digitization and Image Capture
      I.4.8 Scene Analysis

Keywords:
augmented reality, built-in camera, mobile devices, mobile games, motion detection, user interfaces