CueTIP: a mixed-initiative interface for correcting handwriting errors

ABSTRACT

With advances in pen-based computing devices, handwriting has become an increasingly popular input modality. Researchers have put considerable effort into building intelligent recognition systems that can translate handwriting to text with increasing accuracy. However, handwritten input is inherently ambiguous, and these systems will always make errors. Unfortunately, work on error recovery mechanisms has mainly focused on interface innovations that allow users to manually transform the erroneous recognition result into the intended one. In our work, we propose a mixed-initiative approach to error correction. We describe CueTIP, a novel correction interface that takes advantage of the recognizer to continually evolve its results using the additional information from user corrections. This significantly reduces the number of actions required to reach the intended result. We present a user study showing that CueTIP is more efficient and better preferred for correcting handwriting recognition errors. Grounded in the discussion of CueTIP, we also present design principles that may be applied to mixed-initiative correction interfaces in other domains.

References

1. Goldberg, D., & Goodisman, A. (1991). Stylus User Interfaces for Manipulating Text. Proceedings of Fourth Annual ACM Symposium on User Interface Software and Technology, 127--135.
2. Halverson, C., Horn, D., Karat, C., Karat, J. (1999). The Beauty of Errors: Patterns of Error Correction in Desktop Speech Systems. Proceedings of Interact 1999, 133--140.
3. Horvitz, E. (1999). Principles of Mixed-Initiative User Interfaces. Proceedings of CHI 1999 Conference on Human Factors in Computing Systems, 159--166.
4. Huerst, W., Yang, J., & Waibel, A. (1998). Interactive Error Repair for an Online Handwriting Interface. Proceedings of CHI 1998 Conference on Human Factors in Computing Systems, 353--354.
5. Jelinek, F (1997). Statistical Methods for Speech Recognition. The MIT Press.
6. Kristjansson, T., Culotta, A., Viola, P., & McCallum, A. (2004). Interactive Information Extraction with Constrained Conditional Random Fields. Proceedings of the 19th AAAI International Conference on Artificial Intelligence, 412--418.
7. Mankoff, J., Hudson, S. E., Abowd, G. D. (2000). Interaction Techniques for Ambiguity Resolution in Recognition-based Interfaces. Proceedings of Thirteenth Annual ACM Symposium on User Interface Software and Technology, 11--20.
8. Microsoft Tablet Input Panel. Retrieved 1 April 2006: www.microsoft.com/windowsxp/using/tabletpc/pen/correcttext.mspx.
9. Oviatt, S. L. (2000). Taming Recognition Errors with a Multimodal Interface. Communications of the ACM, 43(9), 45--51.
10. Oviatt, S. L., & R. VanGent (1996). Error Resolution During Multimodal Human-Computer Interaction. Proceedings of the 1996 International Conference on Spoken Language Processing, 1, 204--207.
11. Plamondon, R., & Srihari, S. (2000). On-line and Off-line Handwriting Recognition: A Comprehensive Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(1), 63--84.
12. Schomaker, L. R. B. (1994). User-interface Aspects in Recognizing Connected-Cursive Handwriting. Proceedings of the IEE Colloquium on Handwriting and Pen-based Input, 1994/065, 8/1-8/3.
13. Smithies, S., Novins, K., & Arvo, J. (1999). A Handwriting-Based Equation Editor. Proceedings of Graphics Interface, 84--91.
14. Tappert, C. C., Suen, C. Y., & Wakahara, T. (1990). The State of the Art in On-Line Handwriting Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(8), 787--808.

Index Terms

• Applied computing

  • Document management and text processing

    • Document capture

      • Graphics recognition and interpretation

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction devices

      • Touch screens

    • Interaction paradigms

      • Graphical user interfaces

  • Interaction design

    • Interaction design process and methods

      • User centered design