Article

A study on the manipulation of 2D objects in a projector/camera-based augmented reality environment

Authors:

Mark Podlaseck,

Claudio PinhanezAuthors Info & Claims

CHI '05: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 611 - 620

https://doi.org/10.1145/1054972.1055056

Published: 02 April 2005 Publication History

Abstract

Are the object manipulation techniques traditionally used in head-mounted displays (HMDs) applicable to augmented reality based projection systems? This paper examines the differences between HMD- and projector/camera-based AR interfaces in the light of a manipulation task involving documents and applications projected on common office surfaces such as tables, walls, cabinets, and floor. We report a Wizard of Oz study where subjects were first asked to create gesture/voice commands to move 2D objects on those surfaces and then exposed to gestures created by the authors. Among the options, subjects could select the object to be manipulated using voice command; touching, pointing, and grabbing gesture; or a virtual mouse. The results show a strong preference for a manipulation interface based on pointing gestures using small hand movements and involving minimal body movement. Direct touching of the object was also common when the object being manipulated was within the subjects' arm reach. Based on these results, we expect that the preferred interface resembles, in many ways, the egocentric model traditionally used in AR.

References

[1]

Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S. and MacIntyre, B. Recent Advances in Augmented Reality. IEEE Computer Graphics and Applications, 25, 6 (2001), 24--35.

Digital Library

[2]

Bobick, A.F. and Bolles, R.C. The Representation Space Paradigm of Concurrent Evolving Object Descriptions. IEEE Trans. on Pattern Analysis and Machine Intelligence, 14, 2 (1992), 146--156.

Digital Library

[3]

Bolt, R. Put That There: Voice and Gesture at the Graphics Interface. ACM Computer Graphics, 14, 3 (1980), 262--270.

Digital Library

[4]

Bowman, D.A. and Hodges, L.F. An Evaluation of Techniques for Grabbing and Manipulating Remote Objects in Immersive Virtual Environments. In Proc. 1997 Symposium on Interactive 3D Graphics, (1997), 35--38.

Digital Library

[5]

Cao, X. and Balakrishnan, R. VisionWand: Interaction Techniques for Large Displays Using a Passive Wand Tracked in 3D. In Proc. UIST 2003, ACM Press (2003), 193--202.

Digital Library

[6]

Corradini, A. and Cohen, P.R. O the Relationships Among Speech, Gestures, and Object Manipulation in Virtual Environments: Initial Evidence. In Proc. of the International CLASS Workshop on Natural, Intelligent and Effective Interaction in Multimodal Dialogue Systems, Copenhagen, Denmark, (2002).

[7]

Dahlbäck, N., Jönsson, A. and Ahrenberg, L. Wizard of Oz Studies: Why and How. In Proc. IUI 1993, ACM Press (1993), 193--200.

Digital Library

[8]

Drucker, P.F. Management: Tasks, Responsibilities, Practices. 1st ed. Harper & Row, New York (1974).

[9]

Hinckley, K., Pausch, R., Goble, J.C. and Kassell, N.F. A Survey of Design Issues in Spatial Input. In Proc. UIST 1994, ACM Press (1994), 213--222.

Digital Library

[10]

Kidd, A. The Marks are on the Knowledge Worker. In Proc. CHI 1994, ACM Press (1994), 186--191.

Digital Library

[11]

Kjeldsen, R., Levas, A. and Pinhanez, C. Dynamically Reconfigurable Vision-Based User Interfaces. In Proc. Third International Conference on Vision Systems (ICVS 2003), Springer-Verlag (2003), 323--332.

Digital Library

[12]

Koons, D.B., Sparrell, C.J. and Thorrison, K.R. Integrating Simultaneous Input from Speech, Gaze, and Hand Gestures. In Intelligent Multimedia Interfaces, M. Maybury, ed., AAAI/MIT Press, Menlo Park, California, (1993), 257--276.

Digital Library

[13]

Lai, J., Levas, A., Chou, P., Pinhanez, C. and Viveros, M. BlueSpace: Personalizing Workspace through Awareness and Adaptability. International Journal of Human-Computer Studies, 57, (2002), 415--428.

Digital Library

[14]

MacIntyre, B., Mynatt, E.D., Voida, S., Hansen, K.M., Tullio, J. and Corso, G.M. Support for Multitasking and Task Awareness Using Interactive Peripheral Displays. In Proc. UIST 2001, ACM Press, (2001), 41--50.

Digital Library

[15]

Mackinlay, J.D., Card, S.K. and Robertson, G.G. Rapid Controlled Movement through a Virtual 3D Workspace. ACM Computer Graphics, 24, 4 (1990), 171--176.

Digital Library

[16]

Malone, T.W. How Do People Organize Their Desks? Implications for the Design of Office Information Systems. ACM Trans. on Office Information Systems, 1, 1, (1983) 99--112.

Digital Library

[17]

Oviatt, S. Mutual Disambiguation of Recognition Error in a Multimodal Architecture. In Proc. CHI 1999, ACM Press (1999), 576--583.

Digital Library

[18]

Pingali, G., Pinhanez, C., Levas, A., Kjeldsen, R., Podlaseck, M., Chen, H. and Sukaviriya, N. Steerable Interfaces for Pervasive Computing Spaces. In Proc. First IEEE International Conference on Pervasive Computing and Communications (PerCom'03), IEEE, (2003), 315--322.

Digital Library

[19]

Pinhanez, C. The Everywhere Displays Projector: A Device to Create Ubiquitous Graphics Interfaces. In Proc. 3rd International Conference on Ubiquitous Computing 2001 (UbiComp'01), Springer-Verlag (2001), 315--331.

Digital Library

[20]

Podlaseck, M., Pinhanez, C., Alvorado, N., Chan, M. and Dejesus, E. On Interfaces Projected onto Real-World Objects. In Ext. Abstracts CHI 2003, ACM Press (2003), 802--803.

Digital Library

[21]

Poupyrev, I., Weghorst, S., Billinghurst, M. and Ichikawa, T. Egocentric Object Manipulation in Virtual Environments: Empirical Evaluation of Interaction Techniques. Computer Graphics Forum, EUROGRAPHICS'98 Issue, 17, 3 (1998), 41--52.

[22]

Raskar, R., van Baar, J., Beardsley, P., Willwacher, T., Rao, S. and Forlines, C. iLamps: Geometrically Aware and Self-Configuring Projectors. ACM Trans. on Graphics, 22, 3, (2003), 809--818.

Digital Library

[23]

Raskar, R., Welch, G., Cutts, M., Lake, A. and Stesin, L. The Office of the Future: A Unified Approach to Image-Based Modeling and Spatially Immersive Displays. In Proc. SIGGRAPH 1998, ACM Press (1998), 179--188.

Digital Library

[24]

Raskar, R., Welch, G., Low, K. and Bandyopadhyay, D. Shader Lamps: Animating Real Objects with Image-Based Illumination. In Proc. 12th EUROGRAPHICS Workshop on Rendering Techniques, Springer-Verlag, (2001), 89--102.

Digital Library

[25]

Rekimoto, J. and Saitoh, M. Augmented Surfaces: A Spatially Continuous Workspace for Hybrid Computing Environments. In Proc. of CHI 1999, ACM Press (1999), 378--385.

Digital Library

[26]

Streitz, N.A., Geißler, J., Holmer, T., Konomi, S. Mnller-Tomfelde, C., Reischl, W., Rexroth, P., Seitz, P. and Steinmetz, R. i-LAND: An Interactive Landscape for Creativity and Innovation. In Proc. CHI 1999, ACM Press, (1999), 120--127.

Digital Library

[27]

Ware, C. and Balakrishnan, R. Reaching for Objects in VR Displays: Lag and Frame Rate. ACM Trans. on Computer-Human Interaction, 1, 4, (1994), 331--356.

Digital Library

[28]

Ware, C. and Rose, J. Rotating Virtual Objects with Real Handles. ACM Trans. on Computer-Human Interaction, 6, 2, (1999), 162--180.

Digital Library

[29]

Wellner, P. Interacting with Paper on the DigitalDesk. Communications of the ACM, 36, 7, (1993), 86--97.

Digital Library

[30]

Wilson, A. and Shafer, S. XWand: UI for Intelligent Spaces. In Proc. CHI 2003, ACM Press (2003), 545--552.

Digital Library

[31]

Wingrave, C.A., Bowman, D.A. and Ramakrishnan, N. Towards Preferences in Virtual Environment Interfaces. In Proceedings of the EUROGRAPHICS Workshop on Virtual Environments, (2002), 63--72.

Digital Library

Cited By

Chen JZhao LLi YXie ZWilensky UHorn M(2024)“Oh My God! It’s Recreating Our Room!” Understanding Children’s Experiences with A Room-Scale Augmented Reality Authoring ToolkitProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642043(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642043
Gong WSantosa SGrossman TGlueck MClarke DLai F(2023)Affordance-Based and User-Defined Gestures for Spatial Tangible InteractionProceedings of the 2023 ACM Designing Interactive Systems Conference10.1145/3563657.3596032(1500-1514)Online publication date: 10-Jul-2023
https://dl.acm.org/doi/10.1145/3563657.3596032
Ye HLeng JXiao CWang LFu H(2023)ProObjAR: Prototyping Spatially-aware Interactions of Smart Objects with AR-HMDProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580750(1-15)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580750
Show More Cited By

Index Terms

A study on the manipulation of 2D objects in a projector/camera-based augmented reality environment
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Computer graphics
    1. Graphics systems and interfaces
    2. Image manipulation
      1. Image processing
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Graphics input devices
      2. Touch screens
    2. Interaction paradigms

Recommendations

Haptics in Augmented Reality
ICMCS '99: Proceedings of the IEEE International Conference on Multimedia Computing and Systems - Volume 2

An augmented reality system merges synthetic sensory information into a user's perception of a three-dimensional environment. An important performance goal for an augmented reality system is that the user perceives a single seamless environment. In most ...
Multimodal augmented reality: the norm rather than the exception
MVAR '16: Proceedings of the 2016 workshop on Multimodal Virtual and Augmented Reality

Augmented reality (AR) is commonly seen as a technology that overlays virtual imagery onto a participant's view of the world. In line with this, most AR research is focused on what we see. In this paper, we challenge this focus on vision and make a case ...
Augmented reality as perceptual reality
VSMM'06: Proceedings of the 12th international conference on Interactive Technologies and Sociotechnical Systems

As shown in Paul Milgram et al’s Reality-Virtuality Continuum (1994), Augmented Reality occupies a very unique status in the spectrum of Mixed Reality. Unlike Virtual Reality, which is completely made up of the virtual and has been the most important ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '05: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

April 2005

928 pages

ISBN:1581139985

DOI:10.1145/1054972

Conference Chairs:
Wendy Kellogg
IBM T. J. Watson Research Center
,
Shumin Zhai
IBM Almaden Research Center
,
General Chair:
Gerrit van der Veer
Vrije Universiteit, The Netherlands
,
Program Chair:
Carolyn Gale
Stanford University, USA

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 April 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

CHI05

Sponsor:

CHI05: CHI 2005 Conference on Human Factors in Computing Systems

April 2 - 7, 2005

Oregon, Portland, USA

Acceptance Rates

CHI '05 Paper Acceptance Rate 93 of 372 submissions, 25%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

39
Total Citations
View Citations
1,721
Total Downloads

Downloads (Last 12 months)25
Downloads (Last 6 weeks)4

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chen JZhao LLi YXie ZWilensky UHorn M(2024)“Oh My God! It’s Recreating Our Room!” Understanding Children’s Experiences with A Room-Scale Augmented Reality Authoring ToolkitProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642043(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642043
Gong WSantosa SGrossman TGlueck MClarke DLai F(2023)Affordance-Based and User-Defined Gestures for Spatial Tangible InteractionProceedings of the 2023 ACM Designing Interactive Systems Conference10.1145/3563657.3596032(1500-1514)Online publication date: 10-Jul-2023
https://dl.acm.org/doi/10.1145/3563657.3596032
Ye HLeng JXiao CWang LFu H(2023)ProObjAR: Prototyping Spatially-aware Interactions of Smart Objects with AR-HMDProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580750(1-15)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580750
Dong ZZhang JBai XClark ALindeman RHe WPiumsomboon T(2022)Touch-Move-Release: Studies of Surface and Motion Gestures for Mobile Augmented RealityFrontiers in Virtual Reality10.3389/frvir.2022.9272583Online publication date: 17-Aug-2022
https://doi.org/10.3389/frvir.2022.927258
Kim DVogel D(2022)Everywhere Cursor: Extending Desktop Mouse Interaction into Spatial Augmented RealityExtended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491101.3519796(1-7)Online publication date: 27-Apr-2022
https://dl.acm.org/doi/10.1145/3491101.3519796
Li YTahmid ILu FBowman D(2022)Evaluation of Pointing Ray Techniques for Distant Object Referencing in Model-Free Outdoor Collaborative Augmented RealityIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.320309428:11(3896-3906)Online publication date: Nov-2022
https://doi.org/10.1109/TVCG.2022.3203094
Trappey ATrappey CChao MWu C(2022)VR-enabled engineering consultation chatbot for integrated and intelligent manufacturing servicesJournal of Industrial Information Integration10.1016/j.jii.2022.10033126(100331)Online publication date: Mar-2022
https://doi.org/10.1016/j.jii.2022.100331
Li GRempel DLiu YSong WAdamson C(2021)Design of 3D Microgestures for Commands in Virtual Reality or Augmented RealityApplied Sciences10.3390/app1114637511:14(6375)Online publication date: 9-Jul-2021
https://doi.org/10.3390/app11146375
Dalsgaard TKnibbe JBergström J(2021)Modeling Pointing for 3D Target Selection in VRProceedings of the 27th ACM Symposium on Virtual Reality Software and Technology10.1145/3489849.3489853(1-10)Online publication date: 8-Dec-2021
https://dl.acm.org/doi/10.1145/3489849.3489853
Wang INarayana PPatil DBangar RDraper BBeveridge RRuiz J(2021)It’s a Joint Effort: Understanding Speech and Gesture in Collaborative TasksHuman-Computer Interaction. Interaction Techniques and Novel Applications10.1007/978-3-030-78465-2_13(159-178)Online publication date: 3-Jul-2021
https://doi.org/10.1007/978-3-030-78465-2_13
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten