EarPut: augmenting ear-worn devices for ear-based interaction

ABSTRACT

One of the pervasive challenges in mobile interaction is decreasing the visual demand of interfaces towards eyes-free interaction. In this paper, we focus on the unique affordances of the human ear to support one-handed and eyes-free mobile interaction. We present EarPut, a novel interface concept and hardware prototype, which unobtrusively augments a variety of accessories that are worn behind the ear (e.g. headsets or glasses) to instrument the human ear as an interactive surface. The contribution of this paper is three-fold. We contribute (i) results from a controlled experiment with 27 participants, providing empirical evidence that people are able to target salient regions on their ear effectively and precisely, (ii) a first, systematically derived design space for ear-based interaction and (iii) a set of proof of concept EarPut applications that leverage on the design space and embrace mobile media navigation, mobile gaming and smart home interaction.

References

1. Amento, B., Hill, W., and Terveen, L. The sound of one hand: A wrist-mounted bio-acoustic fingertip gesture interface. In Proc. CHI EA '02, 724--725.
2. Ashbrook, D., Baudisch, P., and White, S. Nenya: Subtle and eyes-free mobile input with a magnetically-tracked finger ring. In Proc. CHI '11, 2043--2046.
3. Atmel. 8-bit avr microcontroller with 128k bytes in-system programmable flash: Atmega1284p, 2009.
4. Bailly, G., Müller, J., Rohs, M., Wigdor, D., and Kratz, S. Shoesense: A new perspective on gestural interaction and wearable applications. In Proc. CHI '12, 1239--1248.
5. Bluegiga. Ble113: Preliminary data sheet, 2013.
6. Buil, V., Hollemans, G., and van de Wijdeven, S. Headphones with touch control. In Proc. MobileHCI '05, 377--378.
7. Chen, K.-Y., Lyons, K., White, S., and Patel, S. utrack: 3d input using two magnetic sensors. In Proc. UIST '13, 237--244.
8. Dezfuli, N., Khalilbeigi, M., Huber, J., Müller, F., and Mühlhäuser, M. Palmrc: Imaginary palm-based remote control for eyes-free television interaction. In Proc. EuroiTV '12, 27--34.
9. Dezfuli, N., Khalilbeigi, M., Huber, J., Özkokmaz, M., and Mühlhäuser, M. Palmrc: Leveraging the palm surface as an imaginary eyes-free tv remote control. Behaviour & Information Technology (Dec. 2013).
10. Fukumoto, M., and Tonomura, Y. Whisper: A wristwatch style wearable handset. In Proc. CHI '99, 112--119.
11. Gustafson, S., Bierwirth, D., and Baudisch, P. Imaginary interfaces: Spatial interaction with empty hands and without visual feedback. In Proc. UIST '10, 3--12.
12. Gustafson, S., Holz, C., and Baudisch, P. Imaginary phone: Learning imaginary interfaces by transferring spatial memory from a familiar device. In Proc. UIST '11, 283--292.
13. Gustafson, S. G., Rabe, B., and Baudisch, P. M. Understanding palm-based imaginary interfaces: The role of visual and tactile cues when browsing. In Proc. CHI '13, 889--898.
14. Harrison, C., Benko, H., and Wilson, A. D. Omnitouch: Wearable multitouch interaction everywhere. In Proc. UIST '11, 441--450.
15. Harrison, C., Ramamurthy, S., and Hudson, S. E. On-body interaction: Armed and dangerous. In Proc. TEI '12, 69--76.
16. Harrison, C., Tan, D., and Morris, D. Skinput: Appropriating the body as an input surface. In Proc. CHI '10, 453--462.
17. Holz, C., Grossman, T., Fitzmaurice, G., and Agur, A. Implanted user interfaces. In Proc. CHI '12, 503--512.
18. Hue, P. Philips hue. http://meethue.com/, 2012.
19. Karrer, T., Wittenhagen, M., Lichtschlag, L., Heller, F., and Borchers, J. Pinstripe: Eyes-free continuous input on interactive clothing. In Proc. CHI '11, 1313--1322.
20. Kratz, S., and Rohs, M. Hoverflow: Expanding the design space of around-device interaction. In Proc. MobileHCI '09, 4:1--4:8.
21. Li, K. A., Baudisch, P., and Hinckley, K. Blindsight: Eyes-free access to mobile phones. In Proc. CHI '08, 1389--1398.
22. Lissermann, R., Huber, J., Hadjakos, A., and Mühlhäuser, M. Earput: Augmenting behind-the-ear devices for ear-based interaction. In Proc. CHI EA '13, 1323--1328.
23. Metzger, C., Anderson, M., and Starner, T. Freedigiter: A contact-free device for gesture control. In Proc. ISWC '04, 18--21.
24. Mistry, P., and Maes, P. Sixthsense: A wearable gestural interface. In Proc. ACM SIGGRAPH ASIA '09, 11:1--11:1.
25. Montagu, A. Touching: The human significance of the skin. Columbia U. Press (1971).
26. Ogata, M., Sugiura, Y., Makino, Y., Inami, M., and Imai, M. Senskin: Adapting skin as a soft interface. In Proc. UIST '13, 539--544.
27. Schwarz, J., Harrison, C., Hudson, S., and Mankoff, J. Cord input: An intuitive, high-accuracy, multi-degree-of-freedom input method for mobile devices. In Proc. CHI '10, 1657--1660.
28. Semiconductor, F. Proximity capacitive touch sensor controller: Mpr121, 2010.
29. Sherrington CS. On the proprioceptive system, especially in its reflex aspect. In Brain, Brain (1906).
30. Strommen, E. A. Verbal Self-Regulation in a Children's Game: Impulsive Errors on "Simon Says", vol. 44. Wiley, 1973.
31. Wagner, J., Nancel, M., Gustafson, S. G., Huot, S., and Mackay, W. E. Body-centric design space for multi-surface interaction. In Proc. CHI '13, 1299--1308.
32. Winkler, C., Steimle, J., and Maes, P. MetaSolid: on flexibility and rigidity in future user interfaces. In CHI'13 EA, ACM (2013), 2885--2886.
33. Yi, B., Cao, X., Fjeld, M., and Zhao, S. Exploring user motivations for eyes-free interaction on mobile devices. In Proc. CHI '12, 2789--2792.
34. Zik, P. Parrot zik. http://www.parrot.com/zik/usa/, 2012.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction devices

      • Touch screens

    • Interaction paradigms

      • Graphical user interfaces