extended-abstract

A gesture-based and eyes-free control method for mobile devices

Authors:

Raine A. Kajastila,

Tapio LokkiAuthors Info & Claims

CHI EA '09: CHI '09 Extended Abstracts on Human Factors in Computing Systems

Pages 3559 - 3564

https://doi.org/10.1145/1520340.1520534

Published: 04 April 2009 Publication History

Get Access

Abstract

A novel interaction method for eyes-free control of a mobile phone or a media player is introduced. The method utilizes acceleration sensors along three axes to sense input gestures, such as pointing and tilting. A spherical auditory menu and feedback are provided using speech and 3D sound. A gestural pointing interface, multiple menu configurations, and their implementation details is presented. Evaluation results suggest that fast and accurate selection of menu items is possible without visual feedback. Combining the gestural interface, positions of menu items in 3D and a browsing method with a dynamically adjustable target size of the menu items allow large menus with intuitive, easy access.

Supplementary Material

FLV File (7.flv)

Download
17.21 MB

MOV File (p3559.mov)

Download
21.04 MB

References

[1]

Brewster, S., Lumsden, J., Bell, M., Hall, M., and Tasker, S. Multimodal 'eyes-free' interaction techniques for wearable devices. In Proc. CHI 2003, 463--480.

Digital Library

Google Scholar

[2]

Crossan, A., Williamson, J., Brewster,S., and Murray-Smith, R. Wrist rotation for interaction in mobile contexts. In Proc. MobileHCI 2008, 435--438.

Digital Library

Google Scholar

[3]

Furnas, G. Generalized fisheye views. In Proc. CHI 1986, 16--23.

Digital Library

Google Scholar

[4]

Lorho, G., Marila, J., and Hiipakka, J. Feasibility of multiple non-speech sounds presentation using headphones. In Proc. ICAD 2001, 32--37.

Google Scholar

[5]

Marentakis, G. and Brewster, S. Effects of feedback, mobility and index of difficulty on deictic spatial audio target acquisition in the horizontal plane. In Proc. CHI 2006, 359--368.

Digital Library

Google Scholar

[6]

Partridge, K., Chatterjee, S., Sazawal, V., Borriello, G., and Want, R. TiltType: accelerometer-supported text entry for very small devices. In Proc. UIST 2002, 201--204.

Digital Library

Google Scholar

[7]

Pure Data. http://www.puredata.org

Google Scholar

[8]

Savidis, A., Stephanidis, C., Korte, A., Crispien, K., and Fellbaum, K. A generic direct-manipulation 3D-auditory environment for hierarchical navigation in non-visual interaction. In Proc. ASSETS 1996, 117--123.

Digital Library

Google Scholar

[9]

Sawhney, N. and Schmandt. Nomadic radio: speech and audio interaction for contextual messaging in nomadic environments. ACM TOCHI 3, 7 (2000), 353--383.

Digital Library

Google Scholar

[10]

Tian, F., Xu, L., Wang, H., Zhang, X., Liu, Y., Setlur, V., and Dai, G. Tilt menu: Using the 3D orientation information of pen devices to extend the selection capability of pen-based user interfaces. In Proc. CHI 2008, 1371--1380.

Digital Library

Google Scholar

[11]

Walker, A.,Brewster, s., McGookin, d., and Ng. A. Diary in the sky: A spatial audio display for a mobile calendar. In Proc. BCS IHM-CHI 2001, 531--540.

Crossref

Google Scholar

[12]

Zhao, S., Dragicevic, P., Chignell, M., Balakrishnan, R., and Baudisch, P. earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback. In Proc. CHI 2007, 1395--1404.

Digital Library

Google Scholar

Cited By

View all

Youn EKim TLee G(2022)WristMenu with Tactons: An Eyes- and Ears-Free Menu with Tactons Describing Menu Items in the Wrist Rotation SpaceInternational Journal of Human–Computer Interaction10.1080/10447318.2022.215978040:9(2314-2325)Online publication date: 28-Dec-2022
https://doi.org/10.1080/10447318.2022.2159780
Youn ELee SKim SShim YChan LLee G(2021)WristDial: An Eyes-Free Integer-Value Input Method by Quantizing the Wrist RotationInternational Journal of Human–Computer Interaction10.1080/10447318.2021.189884837:17(1607-1624)Online publication date: 13-Apr-2021
https://doi.org/10.1080/10447318.2021.1898848
Kruijff EBiswas STrepkowski CMaiero JGhinea GStuerzlinger WBrewster SFitzpatrick GCox AKostakos V(2019)Multilayer Haptic Feedback for Pen-Based Tablet InteractionProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300373(1-14)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300373
Show More Cited By

Index Terms

A gesture-based and eyes-free control method for mobile devices
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Touch screens
    2. Interaction paradigms

Recommendations

Virtual Buttons for Eyes-Free Interaction: A Study
Human-Computer Interaction – INTERACT 2015
Abstract
The touch screen of mobile devices, such as smart phones and tablets, is their primary input mechanism. While designed to be used in conjunction with its output capabilities, eyes-free interaction is also possible and useful on touch screens. ...
Gestural Interaction with Mobile Devices: An analysis based on the user's posture
Interacción '16: Proceedings of the XVII International Conference on Human Computer Interaction

Human-machine interaction requires the proposal of new systems, which can be adapted to the needs of the user at any time. This paper proposes the use of natural user interface as an alternative to the traditional contact-based Graphical User Interface. ...
Understanding back-to-front pinching for eyes-free mobile touch input
MobileHCI '16: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services

Using a smartphone touchscreen to control apps mirrored to a distant display is hard, since the user cannot see where she is touching while looking at the distant screen. Tactile landmarks at the back of the phone can mitigate this problem, especially ...

Comments

Information & Contributors

Information

Published In

CHI EA '09: CHI '09 Extended Abstracts on Human Factors in Computing Systems

April 2009

2470 pages

ISBN:9781605582474

DOI:10.1145/1520340

General Chair:
Dan R. Olsen
Brigham Young University
,
Program Chairs:
Ken Hinckley
Microsoft Research
,
Meredith Ringel Morris
Microsoft Research
,
Scott Hudson
Carnegie Mellon University
,
Saul Greenberg
University of Calgary

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 April 2009

Check for updates

Author Tags

Qualifiers

Extended-abstract

Conference

CHI '09

Sponsor:

CHI '09: CHI Conference on Human Factors in Computing Systems

April 4 - 9, 2009

MA, Boston, USA

Acceptance Rates

CHI EA '09 Paper Acceptance Rate 385 of 1,130 submissions, 34%;

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
1,086
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Youn EKim TLee G(2022)WristMenu with Tactons: An Eyes- and Ears-Free Menu with Tactons Describing Menu Items in the Wrist Rotation SpaceInternational Journal of Human–Computer Interaction10.1080/10447318.2022.215978040:9(2314-2325)Online publication date: 28-Dec-2022
https://doi.org/10.1080/10447318.2022.2159780
Youn ELee SKim SShim YChan LLee G(2021)WristDial: An Eyes-Free Integer-Value Input Method by Quantizing the Wrist RotationInternational Journal of Human–Computer Interaction10.1080/10447318.2021.189884837:17(1607-1624)Online publication date: 13-Apr-2021
https://doi.org/10.1080/10447318.2021.1898848
Kruijff EBiswas STrepkowski CMaiero JGhinea GStuerzlinger WBrewster SFitzpatrick GCox AKostakos V(2019)Multilayer Haptic Feedback for Pen-Based Tablet InteractionProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300373(1-14)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300373
Bruun AJensen KKristensen DKjeldskov J(2016)Escaping the Trough: Towards Real-World Impact of Tabletop ResearchInternational Journal of Human–Computer Interaction10.1080/10447318.2016.120676433:2(77-93)Online publication date: 3-Aug-2016
https://doi.org/10.1080/10447318.2016.1206764
Matulic FHusmann MWalter SNorrie MNunes NCostanza EOlivier PSchöning J(2015)Eyes-Free Touch Command Support for Pen-Based Digital Whiteboards via Handheld DevicesProceedings of the 2015 International Conference on Interactive Tabletops & Surfaces10.1145/2817721.2817728(141-150)Online publication date: 15-Nov-2015
https://dl.acm.org/doi/10.1145/2817721.2817728
Kajastila RLokki T(2013)Eyes-free interaction with free-hand gestures and auditory menusInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2012.11.00371:5(627-640)Online publication date: 1-May-2013
https://dl.acm.org/doi/10.1016/j.ijhcs.2012.11.003
Williamson JCrossan ABrewster SBourlard HHuang TVidal EGatica-Perez DMorency LSebe N(2011)Multimodal mobile interactionsProceedings of the 13th international conference on multimodal interfaces10.1145/2070481.2070551(361-368)Online publication date: 14-Nov-2011
https://dl.acm.org/doi/10.1145/2070481.2070551
Pasquero JStobbe SStonehouse NTan DFitzpatrick GGutwin CBegole BKellogg W(2011)A haptic wristwatch for eyes-free interactionsProceedings of the SIGCHI Conference on Human Factors in Computing Systems10.1145/1978942.1979425(3257-3266)Online publication date: 7-May-2011
https://dl.acm.org/doi/10.1145/1978942.1979425
Ni TBowman DNorth CMcMahan R(2011)Design and evaluation of freehand menu selection interfaces using tilt and pinch gesturesInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2011.05.00169:9(551-562)Online publication date: 1-Aug-2011
https://dl.acm.org/doi/10.1016/j.ijhcs.2011.05.001
Scott JDearman DYatani KTruong KPerlin KCzerwinski MMiller R(2010)Sensing foot gestures from the pocketProceedings of the 23nd annual ACM symposium on User interface software and technology10.1145/1866029.1866063(199-208)Online publication date: 3-Oct-2010
https://dl.acm.org/doi/10.1145/1866029.1866063
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

Supplementary Material

References

Cited By

Index Terms

Recommendations

Virtual Buttons for Eyes-Free Interaction: A Study

Gestural Interaction with Mobile Devices: An analysis based on the user's posture

Understanding back-to-front pinching for eyes-free mobile touch input

Comments

Information

Published In

Sponsors

Publisher

Publication History

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations