ABSTRACT
The rise of affordable head-mounted displays (HMDs) has raised questions about how to best design user interfaces for this technology. This paper focuses on the use of HMDs for home and office applications that require substantial text input. A physical keyboard is a familiar and effective text input device in normal desktop computing. But without additional camera technology, an HMD occludes all visual feedback about a user's hand position over the keyboard. We describe a system that assists HMD users in typing on a physical keyboard. Our system has a virtual keyboard assistant that provides visual feedback inside the HMD about a user's actions on the physical keyboard. It also provides powerful automatic correction of typing errors by extending a state-of-the-art touchscreen decoder. In a study with 24 participants, we found our virtual keyboard assistant enabled users to type more accurately on a visually-occluded keyboard. We found users wearing an HMD could type at over 40 words-per-minute while obtaining an error rate of less than 5%.
- Bowman, D. A., Rhoton, C. J., and Pinho, M. S. Text input techniques for immersive virtual environments: An empirical comparison. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 46, 26 (2002), 2154--2158.Google ScholarCross Ref
- Budhiraja, P., Sodhi, R., Jones, B., Karsch, K., Bailey, B., and Forsyth, D. Where's my drink? Enabling peripheral real world interactions while using HMDs. arXiv preprint arXiv:1502.04744 (2015).Google Scholar
- Lee, M., and Woo, W. ARKB: 3D vision-based augmented reality keyboard. In Proceeding of the 13th International Conference on Artificial Reality and Telexistence (2003).Google Scholar
- McGill, M., Boland, D., Murray-Smith, R., and Brewster, S. A dose of reality: Overcoming usability challenges in VR head-mounted displays. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI '15, ACM (New York, NY, USA, 2015), 2143--2152. Google ScholarDigital Library
- Vertanen, K., and Kristensson, P. O. A versatile dataset for text entry evaluations based on genuine mobile emails. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services, MobileHCI '11, ACM (New York, NY, USA, 2011), 295--298. Google ScholarDigital Library
- Vertanen, K., Memmi, H., Emge, J., Reyal, S., and Kristensson, P. O. VelociTap: Investigating fast mobile text entry using sentence-based decoding of touchscreen keyboard input. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI '15, ACM (New York, NY, USA, 2015), 659--668. Google ScholarDigital Library
- Walker, J., Kuhl, S., and Vertanen, K. Decoder-assisted typing using an HMD and a physical keyboard. In Extended Abstracts of the the ACM Conference on Human Factors in Computing Systems, CHI '16 workshop (New York, NY, USA, 2016).Google Scholar
- Yi, X., Yu, C., Zhang, M., Gao, S., Sun, K., and Shi, Y. ATK: Enabling ten-finger freehand typing in air based on 3d hand tracking data. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, UIST '15, ACM (New York, NY, USA, 2015), 539--548. Google ScholarDigital Library
Index Terms
- Efficient Typing on a Visually Occluded Physical Keyboard
Recommendations
Physical Keyboards in Virtual Reality: Analysis of Typing Performance and Effects of Avatar Hands
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing SystemsEntering text is one of the most common tasks when interacting with computing systems. Virtual Reality (VR) presents a challenge as neither the user's hands nor the physical input devices are directly visible. Hence, conventional desktop peripherals are ...
The 1line keyboard: a QWERTY layout in a single line
UIST '11: Proceedings of the 24th annual ACM symposium on User interface software and technologyCurrent soft QWERTY keyboards often consume a large portion of the screen space on portable touchscreens. This space consumption can diminish the overall user experi-ence on these devices. In this paper, we present the 1Line keyboard, a soft QWERTY ...
Text Entry for Ultra-Small Touchscreens Using a Fixed Cursor and Movable Keyboard
CHI EA '16: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing SystemsEmerging ultra-small wearables like smartwatches pose a design challenge for touch-based text entry, due to the "fat finger" problem, wherein users struggle to select elements much smaller than their fingers. In order to address the challenge, we ...
Comments