Anna Maria Feit
Antti Oulasvirta
ABSTRACT
Inspired by the high keying rates of skilled pianists, we study the design of piano keyboards for rapid text entry. We review the qualities of the piano as an input device, observing four design opportunities: 1) chords, 2) redundancy (more keys than letters in English), 3) the transfer of musical skill and 4) optional sound feedback. Although some have been utilized in previous text entry methods, our goal is to exploit all four in a single design. We present PianoText, a computationally designed mapping that assigns letter sequences of English to frequent note transitions of music. It allows fast text entry on any MIDI-enabled keyboard and was evaluated in two transcription typing studies. Both show an achievable rate of over 80 words per minute. This parallels the rates of expert Qwerty typists and doubles that of a previous piano-based design from the 19th century. We also design PianoText-Mini, which allows for comparable performance in a portable form factor. Informed by the studies, we estimate the upper bound of typing performance, draw implications to other text entry methods, and critically discuss outstanding design challenges.
Supplemental Material
Available for Download
This is the supplementary material for the paper: PianoText: Redesigning the Piano Keyboard for Text Entry It contains the following files: AppendixA.pdf A pdf file describing the algorithm for creating a letter-to-key mapping AppendixB.pdf A pdf file describing the training program of the long-term study of PianoText Example_Study1.pdf An example stimulus used in study 1. The music sheet shows sentences translated into notes. Example_Study1.mp4 A video of an example performance in study 1 - a professional pianist enters sentences on the piano by playing from a sheet of music. Performance was 82 wpm. Example_Study2.mp4 A video of the last test conducted in study 2. It shows the trained piano-typist transcribing sentences. Overall performance was 81 wpm. Required software: Pdf viewer Video player
- Adler, M. Antique Typewriters, from Creed to QWERTY. Schiffer Pub., 1997.Google Scholar
- Andersen, T. H., and Zhai, S. writing with music: Exploring the use of auditory feedback in gesture interfaces. TAP 7, 3 (2010). Google ScholarDigital Library
- Beddoes, M. P., and Hu, Z. A chord stenograph keyboard: a possible solution to the learning problem in stenography. IEEE Transactions on Systems, Man and Cybernetics 24, 7 (1994), 953--960.Google ScholarCross Ref
- Bowers, B. Electrical engineering 100 years ago. British Journal of Audiology 13, S2 (1979), 1--4.Google Scholar
- Brewster, S. Overcoming the lack of screen space on mobile computers. Personal and Ubiquitous Computing, 3 (2002). Google ScholarDigital Library
- Buxton, B. Chord keyboards. http://www.billbuxton. com/input06.ChordKeyboards.pdf, 2012.Google Scholar
- Deutsch, D. Pitch circularity. deutsch.ucsd.edu.Google Scholar
- Dunlop, M., and Levine, J. Multidimensional pareto optimization of touchscreen keyboards for speed, familiarity and improved spell checking.Google Scholar
- Engelbart, D. C. Design considerations for knowledge workshop terminals. In Proc.1973, national computer conference and exposition, ACM (1973), 221--227. Google ScholarDigital Library
- Ericsson, A. K. The Cambridge handbook of expertise and expert performance, Chapter 26: Music. Cambridge University Press, 2006.Google ScholarCross Ref
- Ericsson, K. A., and Williams, A. M. Capturing naturally occurring superior performance in the laboratory: translational research on expert performance. Journal of Experimental Psychology: Applied 13, 3 (2007), 115.Google ScholarCross Ref
- Findlater, L., Wobbrock, J. O., and Wigdor, D. Typing on flat glass: examining ten-finger expert typing patterns on touch surfaces. In Proc. CHI'11, ACM (2011), 2453--2462. Google ScholarDigital Library
- Furuya, S., and Soechting, J. F. Role of auditory feedback in the control of successive keystrokes during piano playing. Experimental Brain Research, 2 (2010), 223--237.Google ScholarCross Ref
- Gentner, D. Expertise in typewriting. Tech. rep., DTIC Document, 1984.Google Scholar
- Gopher, D., and Raij, D. Typing with a two-hand chord keyboard: will the qwerty become obsolete? IEEE Transactions on Systems, Man and Cybernetics, 4 (1988), 601--609.Google ScholarCross Ref
- Lepinski, G. J., Grossman, T., and Fitzmaurice, G. The design and evaluation of multitouch marking menus. In Proc. CHI'10, ACM (2011), 2233--2242. Google ScholarDigital Library
- Lifton, J., and Knight, M. Plover, the open source steno program. http://plover.stenoknight.com/, 2012.Google Scholar
- Lyons, K. e. a. Experimental evaluations of the twiddler one-handed chording mobile keyboard. Human-Computer Interaction 21, 4 (2006). Google ScholarDigital Library
- MacKenzie, I., and Zhang, S. The design and evaluation of a high-performance soft keyboard. In Proc. CHI'99, ACM (2011), 25--31. Google ScholarDigital Library
- MacKenzie, S. I., and Soukoreff, W. R. Text entry for mobile computing: Models and methods, theory and practice. Human-Computer Interaction 17, 2--3 (2002), 147--198.Google ScholarCross Ref
- MacKenzie, S. I., T.-I. K. Text entry systems: Mobility, accessibility, universality. Morgan Kaufmann, 2010. Google ScholarDigital Library
- Norman, D. A., and Fisher, D. Why alphabetic keyboards are not easy to use: Keyboard layout doesn't much matter. Human Factors, 5 (1982).Google Scholar
- Oulasvirta, e. a. Improving two-thumb text entry on touchscreen devices. In Proc. CHI'13, 2765--2774. Google ScholarDigital Library
- Rumelhart, D. E., and Norman, D. A. Simulating a skilled typist: A study of skilled cognitive-motor performance. Cognitive Science, 1 (1982), 1--36.Google ScholarCross Ref
- Schmidt, R. A., and Lee, T. Motor Control and Learning, 5E. Human kinetics, 1988.Google Scholar
- Schmuckler, M. A., and Bosman, E. L. Interkey timing in piano performance and typing. Canadian Journal of Experimental Psychology, 2 (1997), 99.Google ScholarCross Ref
- Schneider, W. Training high-performance skills: Fallacies and guidelines. Human Factors, 3 (1985), 285--300.Google ScholarCross Ref
- Seibel, R. Data entry through chord, parallel entry devices. Human Factors: The Journal of the Human Factors and Ergonomics Society, 2 (1964), 189--192.Google ScholarCross Ref
- Vertanen, K., and Kristensson, P. A versatile dataset for text entry evaluations based on genuine mobile emails. In Proc. MobileHCI'11, ACM (2011), 295--298. Google ScholarDigital Library
- Zhai, S., Hunter, M., and Smith, B. Performance optimization of virtual keyboards. Human-Computer Interaction, 2--3 (2002), 229--269.Google Scholar
Index Terms
- PianoText: redesigning the piano keyboard for text entry
Recommendations
PianoText: redesigning the piano keyboard for text entry
DIS Companion '14: Proceedings of the 2014 companion publication on Designing interactive systemsInspired by the high keying rates of skilled pianists we study the design of piano keyboards for rapid text entry. We present PianoText, a computationally designed mapping that assigns letter sequences of English to frequent note transitions in music. ...
PianoText: transferring musical expertise to text entry
CHI EA '13: CHI '13 Extended Abstracts on Human Factors in Computing SystemsWe present PianoText, a text entry method based on a piano keyboard with an optimized mapping between notes and chords of music to letters of the English language. PianoText exemplifies the idea of transferring musical expertise to a text entry task by ...
Invisiboard: maximizing display and input space with a full screen text entry method for smartwatches
MobileHCI '16: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and ServicesThe small displays of smartwatches make text entry difficult and time consuming. While text entry rates can be increased, this continues to occur at the expense of available screen display space. Soft keyboards can easily use half the display space of ...
Comments