Automation of VUIto GUI mapping

ABSTRACT

This document presents a brief overview of my doctoral research, which is focused on an approach to map a VUI to a GUI in order to increase the usability of graphical applications for users with motoric challenges. This is accomplished by creating a prototype that uses voice as an input modality for Scratch. Evaluation and assessment of the research will be performed through observation of children with motor disabilities using the tool. The research will be further generalized by creating a template for mapping a VUI to any GUI. My current research focus is summarized followed by the motivation for this research. Related work is described along with a problem statement and research goals. My dissertation status and expected contributions conclude this document.

References

1. Alsuraihi, M. M. and Rigas, D. I. How effective is it to design by voice? Proc. British HCI Group Annual Conference on People and Computers (2007), 159--162.
2. Association for Computing Machinery (1992). ACM code of ethics and professional conduct. http://www.acm.org/constitution/code.html.
3. Begel, A. Programming by voice: A domain-specific application of speech recognition. AVIOS Speech Technology Symposium -- SpeechTek West (2005).
4. Cohen, P. R., Johnston, M., McGee, D., Oviatt, S. L., Clow, J., and Smith, I. The efficiency of multimodal interaction: A case study. Proc. Spoken Language (1998).
5. Dai, L., Goldman, R., Sears, A., and Lozier, J. Speech-based cursor control: A study of grid-based solutions. Proc. ACM SIGACCESS on Computers and Accessibility (2004), 94--101.
6. Désilets, A., Fox, D., and Norton, S. VoiceCode: An innovative speech interface for programming-by-voice. Proc. Human Factors in Computing Systems (2006), 239--242.
7. Harada, S., Wobbrock, J., Malkin, J., Bilmes, J., and Landay, J. Longitudinal study of people learning to use continuous voice-based cursor control. Proc. Human Factors in Computing Systems (2009), 347--356.
8. National Institute of Neurological Disorders and Stroke. National Institutes of Health. http://www.ninds.nih.gov/disorders/disorder_index.htm.
9. Rudraraju, R., Datla, S., Banerjee, A., Sudame, M., Gray, J., Shuttleworth, D., and Edwards, G. Programming by Voice with Scratch: Teaching a Cat to Obey a Bird. Scratch@MIT Conference (2010).
10. Shaik, S., Corvin, R., Sudarsan, R., Javed, F., Ijaz, Q., Roychoudhury, S., Gray, J., and Bryant, B. SpeechClipse -- An Eclipse speech plug-in. In Eclipse Technology eXchange Workshop (OOPSLA), (2003).
11. Wagner, A., Rudraraju, R., Datla, S., Banerjee, A., Sudame, M., and Gray, J. Programming by voice: A hands-free approach for motorically challenged children. Proc. ACM SIGCHI Human Factors in Computing Systems (2012), 2087--2092.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)