A design tool for camera-based interaction

ABSTRACT

Cameras provide an appealing new input medium for interaction. The creation of camera-based interfaces is outside the skill-set of most programmers and completely beyond the skills of most interface designers. Image Processing with Crayons is a tool for creating new camera-based interfaces using a simple painting metaphor. A transparent layers model is used to present the designer with all of the necessary information. Traditional machine learning algorithms have been modified to accommodate the rapid response time required of an interactive design tool.

References

1. Belongie, S., Carson, C., Greenspan, H. and Malik, J. "Color and texture-based image segmentation using EM and its application to content-based image retrieval." In Proc. Int. Conf. Comp. Vis., 1998.]]
2. Buck, I., Finkelstein, A., Jacobs, C., Klein, A., Salesin, D. H., Seims, J., Szeliski, R., and Toyama, K. "Performance-driven hand-drawn animation." Proceedings of NPAR 2000 (June 2000).]]
3. Carson, C., Thomas, M., Belongie, S., Hellerstein, J.M. and Malik, J. "Blobworld: A system for region-based image indexing and retrieval." In Proc. Int. Conf. Visual Inf. Sys., 1999.]]
4. Christos Faloutsos, C., Barber, R., Flickner, M., Niblack, W., Petkovic, D. and Equitz, W. "Efficient and effective querying by image content." Journal of Intelligent Information Systems, 3(3/4): 231--262, July 1994.]]
5. Cover, T., and Hart, P. "Nearest Neighbor Pattern Classification." IEEE Transactions on Information Theory, 13, (1967) 21--27.]]
6. Crowley, J.L, Bérard, F., and Coutaz, J. "Finger Tracking as an Input Device for Augmented Reality." IWAGFR '95: Zurich, Germany (June 1995).]]
7. Davis, J. "Recognizing movement using motion histograms." MIT Media lab Technical Report #487, March 1999.]]
8. Duda, R. O., Hart, P. E., and Stork, D. G., Pattern Classification. (2001).]]
9. Fails, J.A., Olsen, D.R. "LightWidgets: Interacting in Everyday Spaces." Proceedings of IUI '02 (San Francisco CA, January 2002).]]
10. Fayyad, U.M. and Irani, K. B. "On the Handling of Continuous-valued Attributes in Decision Tree Generation." Machine Learning, 8, 87--102,(1992).]]
11. Freeman, W., Anderson, D., Beardsley, P., et al. "Computer vision for interactive computer graphics." IEEE Computer Graphics and Applications, Vol. 18, Num 3, pages 42--53, May--June 1998.]]
12. Ghidary, S.S. Nakata, Y., Takamori, T. and Hattori, M. "Head and Face Detection at Indoor Environment by Home Robot." Proceedings of ICEE200 (Iran, May 2000).]]
13. Hirata, K. and Kato, T. "Query by Visual Example-Content-Based Image Retrieval." Advances in Database Technology EDBT '92, 3rd International Conference on Extending Database Technology, Vienna, Austria, A. Pirotte, C. Delobel, and G. Gottlob, eds., Lecture Notes in Computer Science, vol. 580, Springer-Verlag, Berlin, 1992, pp. 56--71.]]
14. Ishii, H., and Ullmer, B. "Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms." Human Factors in Computing Systems (CHI '97). (March 1997).]]
15. Jaimes, A. and Chang, S.-F. "A Conceptual Framework for Indexing Visual Information at Multiple Levels." IS&T/SPIE Internet Imaging 2000, (San Jose CA, January 2000).]]
16. Jaimes, A. and Chang, S.-F. "Automatic Selection of Visual Features and Classifier." Storage and Retrieval for Image and Video Databases VIII, IS&T/SPIE (San Jose CA, January 2000).]]
17. Jaimes, A. and Chang, S.-F. "Integrating Multiple Classifiers in Visual Object Detectors Learned from User Input." Invited paper, session on Image and Video Databases, 4th Asian Conference on Computer Vision (ACCV 2000), Taipei, Taiwan, January 8-11, 2000.]]
18. Kelly, P. M. and Cannon, M. "Query by Image Example: the CANDID Approach." Los Alamos National Laboratory White Paper, (1995).]]
19. Koller, D., Klinker, G., Rose, E., Breen, D., Whitaker, R. and Tuceryan, M. "Real-time Vision-based Camera Tracking for Augmented Reality Applications." Proceedings of the Symposium on Virtual Reality Software and Technology (VRST-97), Lusanne, Switzerland, Sept 1997, 87--94.]]
20. Krueger, M. W., Gionfriddo. T., and Hinrichsen, K., "VIDEOPLACE -- an artificial reality". Human Factors in Computing Systems, CHI '85 Conference Proceedings, ACM Press, 1985, 35--40.]]
21. Hardenberg, C. and Bérard, F. "Bare-Hand Human-Computer Interaction." Proceedings of the ACM Workshop on Perceptive User Interfaces (Orlando, FL, Nov. 15--16, 2001).]]
22. Olsen, D.R., Nielsen, T. "Laser Pointer Interaction." Proceedings of CHI '01 (Seattle WA, March 2001).]]
23. Quinlan, J. R. "Induction of Decision Trees." Machine Learning, 1(1); 81--106, (1986).]]
24. Rumelhart, D., Widrow, B., and Lehr, M. "The Basic Ideas in Neural Networks." Communications of the ACM, 37(3), (1994), pp 87--92.]]
25. Scott, S., Lesh, N., Klau, G. "Investigating Human-Computer Optimization." Proceedings of CHI '02 (Minneapolis MN, April 2002), pp. 155--162.]]
26. Starner, T., Auxier, J. and Ashbrook, D. "The Gesture Pendant: A Self-illuminating, Wearable, Infrared Computer Vision System for Home Automation Control and Medical Monitoring." International Symposium on Wearable Computing (Atlanta GA, October 2000).]]
27. Takagi, H. "Interactive evolutionary computation: fusion of the capabilities of EC optimization and human evaluation." Proceedings of the IEEE, Vol. 89, Issue 9, Sep 2001, pp. 1275--1296.]]
28. Underkoffler, J., Ullmer, B. and Ishii, H. "Emancipated Pixels: Real-World Graphics in the Luminous Room." Proceedings of SIGGRAPH '99 (Los Angeles CA, 1999), ACM Press, 385--392.]]
29. Underkoffler, J. and Ishii H. "Illuminating Light: An Optical Design Tool with a Luminous-Tangible Interface." Proceedings of CHI '98 (Los Angeles CA, April 1998).]]
30. Vailaya, A., Zhong, Y., and Jain, A. K. "A hierarchical system for efficient image retrieval." In Proc. Int. Conf. on Patt. Recog. (August 1996).]]
31. Viola, P. and Jones, M. "Robust real-time object detection." Technical Report 2001/01, Compaq CRL, February 2001.]]
32. Weiser, M. "The Computer for the 21st Century." Scientific American, 1991, 365(3), pp. 94--104.]]
33. Yang, M.H. and Ahuja, N. "Gaussian Mixture Model for Human Skin Color and Its Application in Image and Video Databases." Proceedings of SPIE '99 (San Jose CA, Jan 1999), 458--466.]]

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

        • Object recognition

        • Tracking

      • Computer vision tasks

        • Scene understanding

• Human-centered computing

  • Human computer interaction (HCI)

    • HCI theory, concepts and models

    • Interaction paradigms

  • Interaction design

    • Interaction design process and methods

      • Interface design prototyping

    • Interaction design theory, concepts and paradigms