Perspective cursor: perspective-based interaction for multi-display environments

ABSTRACT

Multi-display environments and smart meeting rooms are now becoming more common. These environments build a shared display space from variety of devices: tablets, projected surfaces, tabletops, and traditional monitors. Since the different display surfaces are usually not organized in a single plane, traditional schemes for stitching the displays together can cause problems for interaction. However, there is a more natural way to compose display space -- using perspective. In this paper, we develop interaction techniques for multi-display environments that are based on the user's perspective on the room. We designed the Perspective Cursor, a mapping of cursor to display space that appears natural and logical from wherever the user is located. We conducted an experiment to compare two perspective-based techniques, the Perspective Cursor and a beam-based technique, with traditional stitched displays. We found that both perspective techniques were significantly faster for targeting tasks than the traditional technique, and that Perspective Cursor was the most preferred method. Our results show that integrating perspective into the design of multi-display environments can substantially improve performance.

REFERENCES

• 1: Baudisch, P., Cutrell, E., Robbins, D., Czerwinski, M., Tandler, P. Bederson, B., and Zierlinger, A. Drag-and-Pop and Drag-and-Pick: Techniques for Accessing Remote Screen Content on Touch- and Pen-operated Systems. Proc. Interact 2003, 57--64.
• 2: Patrick Baudisch , Edward Cutrell , Ken Hinckley , Robert Gruen, Mouse ether: accelerating the acquisition of targets across multi-monitor displays, CHI '04 extended abstracts on Human factors in computing systems, April 24-29, 2004, Vienna, Austria [doi>10.1145/985921.986069]
• 3: Patrick Baudisch , Ruth Rosenholtz, Halo: a technique for visualizing off-screen objects, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642695]
• 4: Ravin Balakrishnan, "Beating" Fitts' law: virtual enhancements for pointing facilitation, International Journal of Human-Computer Studies, v.61 n.6, p.857-874, December 2004 [doi>10.1016/j.ijhcs.2004.09.002]
• 5: Hrvoje Benko , Steven Feiner, Multi-monitor mouse, CHI '05 extended abstracts on Human factors in computing systems, April 02-07, 2005, Portland, OR, USA [doi>10.1145/1056808.1056878]
• 6: Jacob T. Biehl , Brian P. Bailey, ARIS: an interface for application relocation in an interactive space, Proceedings of the 2004 conference on Graphics interface, p.107-116, May 17-19, 2004, London, Ontario, Canada
• 7: Richard A. Bolt, “Put-that-there”: Voice and gesture at the graphics interface, Proceedings of the 7th annual conference on Computer graphics and interactive techniques, p.262-270, July 14-18, 1980, Seattle, Washington, United States
• 8: Kellogg S. Booth , Brian D. Fisher , Chi Jui Raymond Lin , Ritchie Argue, The "mighty mouse" multi-screen collaboration tool, Proceedings of the 15th annual ACM symposium on User interface software and technology, October 27-30, 2002, Paris, France [doi>10.1145/571985.572016]
• 9: Davis, J., and Chen, X. LumiPoint: Multi-user laser-based interaction on large tiled displays. Displays 2002, 23(5):205--211.
• 10: Paul Dietz , Darren Leigh, DiamondTouch: a multi-user touch technology, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502389]
• 11: Armando Fox , Brad Johanson , Pat Hanrahan , Terry Winograd, Integrating Information Appliances into an Interactive Workspace, IEEE Computer Graphics and Applications, v.20 n.3, p.54-65, May 2000 [doi>10.1109/38.844373 ]
• 12: G. W. Furnas, Generalized fisheye views, Proceedings of the SIGCHI conference on Human factors in computing systems, p.16-23, April 13-17, 1986, Boston, Massachusetts, United States
• 13: Jörg Geißler, Shuffle, throw or take it! working efficiently with an interactive wall, CHI 98 conference summary on Human factors in computing systems, p.265-266, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/286498.286745]
• 14: Tovi Grossman , Ravin Balakrishnan, The bubble cursor: enhancing target acquisition by dynamic resizing of the cursor's activation area, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1055012]
• 15: Hascoët, M. Throwing models for large displays. Proc. HCI 2003, 73--77.
• 16: Ken Hinckley, Synchronous gestures for multiple persons and computers, Proceedings of the 16th annual ACM symposium on User interface software and technology, p.149-158, November 02-05, 2003, Vancouver, Canada [doi>10.1145/964696.964713]
• 17: Ken Hinckley , Gonzalo Ramos , Francois Guimbretiere , Patrick Baudisch , Marc Smith, Stitching: pen gestures that span multiple displays, Proceedings of the working conference on Advanced visual interfaces, May 25-28, 2004, Gallipoli, Italy [doi>10.1145/989863.989866]
• 18: Brad Johanson , Greg Hutchins , Terry Winograd , Maureen Stone, PointRight: experience with flexible input redirection in interactive workspaces, Proceedings of the 15th annual ACM symposium on User interface software and technology, October 27-30, 2002, Paris, France [doi>10.1145/571985.572019]
• 19: Kohtake, N., Ohsawa, R., Yonezawa, T., Matsukura,Y. Iway, M. Takashio, K., Tokuda, H. u-Texture: Self-Organizable Universal Panels for Creating Smart Surroundings. Proc. UbiComp 2005 , 19--36.
• 20: Brad A. Myers , Choon Hong Peck , Jeffrey Nichols , Dave Kong , Robert Miller, Interacting at a Distance Using Semantic Snarfing, Proceedings of the 3rd international conference on Ubiquitous Computing, p.305-314, September 30-October 02, 2001, Atlanta, Georgia, USA
• 21: Brad A. Myers , Rishi Bhatnagar , Jeffrey Nichols , Choon Hong Peck , Dave Kong , Robert Miller , A. Chris Long, Interacting at a distance: measuring the performance of laser pointers and other devices, Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/503376.503383]
• 22: Miguel A. Nacenta , Dzmitry Aliakseyeu , Sriram Subramanian , Carl Gutwin, A comparison of techniques for multi-display reaching, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1055024]
• 23: Nacenta, M. Computer Vision approaches to solve the screen pose acquisition problem for Perspective Cursor. Tech. Rep. HCI-TR-06-01, Comp. Sci. Dept., U. of Saskatchewan, (2006).
• 24: Dan R. Olsen, Jr. , Travis Nielsen, Laser pointer interaction, Proceedings of the SIGCHI conference on Human factors in computing systems, p.17-22, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365030]
• 25: J. Karen Parker , Regan L. Mandryk , Kori M. Inkpen, TractorBeam: seamless integration of local and remote pointing for tabletop displays, Proceedings of the 2005 conference on Graphics interface, May 09-11, 2005, Victoria, British Columbia
• 26: Jun Rekimoto, Pick-and-drop: a direct manipulation technique for multiple computer environments, Proceedings of the 10th annual ACM symposium on User interface software and technology, p.31-39, October 14-17, 1997, Banff, Alberta, Canada [doi>10.1145/263407.263505]
• 27: Jun Rekimoto , Masanori Saitoh, Augmented surfaces: a spatially continuous work space for hybrid computing environments, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.378-385, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303113]
• 28: Rekimoto, J., Ayatsuka, Y., and Kohno, M. SyncTap: An interaction technique for mobile networking. Proc. Mobile HCI (2003), 104--115.
• 29: George Robertson , Mary Czerwinski , Patrick Baudisch , Brian Meyers , Daniel Robbins , Greg Smith , Desney Tan, The Large-Display User Experience, IEEE Computer Graphics and Applications, v.25 n.4, p.44-51, July 2005 [doi>10.1109/MCG.2005.88]
• 30: Manuel Román , Christopher Hess , Renato Cerqueira , Anand Ranganathan , Roy H. Campbell , Klara Nahrstedt, A Middleware Infrastructure for Active Spaces, IEEE Pervasive Computing, v.1 n.4, p.74-83, October 2002 [doi>10.1109/MPRV.2002.1158281 ]
• 31: Norbert A. Streitz , Jörg Geißler , Torsten Holmer , Shin'ichi Konomi , Christian Müller-Tomfelde , Wolfgang Reischl , Petra Rexroth , Peter Seitz , Ralf Steinmetz, i-LAND: an interactive landscape for creativity and innovation, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.120-127, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303010]
• 32: Kishore Swaminathan , Steve Sato, Interaction design for large displays, interactions, v.4 n.1, p.15-24, Jan./Feb. 1997 [doi>10.1145/242388.242395]
• 33: Mike Wu , Ravin Balakrishnan, Multi-finger and whole hand gestural interaction techniques for multi-user tabletop displays, Proceedings of the 16th annual ACM symposium on User interface software and technology, p.193-202, November 02-05, 2003, Vancouver, Canada [doi>10.1145/964696.964718]
• 34: Ka-Ping Yee, Peephole displays: pen interaction on spatially aware handheld computers, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642613]
• 35: Shumin Zhai , Carlos Morimoto , Steven Ihde, Manual and gaze input cascaded (MAGIC) pointing, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.246-253, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303053]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

Keywords:
direct-manipulation interfaces, laser pointing, multi-display interaction techniques, multi-monitor environments