Multimodal interaction on mobile phones: development and evaluation using ACICARE

ABSTRACT

The development and the evaluation of multimodal interactive systems on mobile phones remains a difficult task. In this paper we address this problem by describing a component-based approach, called ACICARE, for developing and evaluating multimodal interfaces on mobile phones. ACICARE is dedicated to the overall iterative design process of mobile multimodal interfaces, which consists of cycles of designing, prototyping and evaluation. ACICARE is based on two complementary tools that are combined: ICARE and ACIDU. ICARE is a component-based platform for rapidly developing multimodal interfaces. We adapted the ICARE components to run on mobile phones and we connected them to ACIDU, a probe that gathers customer's usage on mobile phones. By reusing and assembling components, ACICARE enables the rapid development of multimodal interfaces as well as the automatic capture of multimodal usage for in-field evaluations. We illustrate ACICARE using our contact manager system, a multimodal system running on the SPV c500 mobile phone.

REFERENCES

• 1: Lynne Baillie , Raimund Schatz, Exploring multimodality in the laboratory and the field, Proceedings of the 7th international conference on Multimodal interfaces, October 04-06, 2005, Torento, Italy [doi>10.1145/1088463.1088482]
• 2: Jullien Bouchet , Laurence Nigay, ICARE: a component-based approach for the design and development of multimodal interfaces, CHI '04 extended abstracts on Human factors in computing systems, April 24-29, 2004, Vienna, Austria [doi>10.1145/985921.986055]
• 3: Jullien Bouchet , Laurence Nigay , Thierry Ganille, ICARE software components for rapidly developing multimodal interfaces, Proceedings of the 6th international conference on Multimodal interfaces, October 13-15, 2004, State College, PA, USA [doi>10.1145/1027933.1027975]
• 4: Rachel Demumieux , Patrick Losquin, Gather customer's real usage on mobile phones, Proceedings of the 7th international conference on Human computer interaction with mobile devices & services, September 19-22, 2005, Salzburg, Austria [doi>10.1145/1085777.1085828]
• 5: Dey, A., Abowd, G. and Salber, D. Conceptual Framework and a Toolkit for Supporting the Rapid Prototyping of Context-Aware Applications. In HCI'01 Journal. Vol. 16, Lawrence Erlbaum, 2001, pp. 96--166.
• 6: Jacob Eisenstein , Jean Vanderdonckt , Angel Puerta, Applying model-based techniques to the development of UIs for mobile computers, Proceedings of the 6th international conference on Intelligent user interfaces, p.69-76, January 14-17, 2001, Santa Fe, New Mexico, United States [doi>10.1145/359784.360122]
• 7: Murielle Florins , Jean Vanderdonckt, Graceful degradation of user interfaces as a design method for multiplatform systems, Proceedings of the 9th international conference on Intelligent user interfaces, January 13-16, 2004, Funchal, Madeira, Portugal [doi>10.1145/964442.964469]
• 8: Michael Johnston , Srinivas Bangalore , Gunaranjan Vasireddy , Amanda Stent , Patrick Ehlen , Marilyn Walker , Steve Whittaker , Preetam Maloor, MATCH: an architecture for multimodal dialogue systems, Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, July 07-12, 2002, Philadelphia, Pennsylvania [doi>10.3115/1073083.1073146]
• 9: Kjeldskov, J., Skov, M. B., Als, B. S. and Hoegh, R. T. Is it Worth the Hassle? Exploring the Added Value of Evaluating the Usability of Context-Aware Mobile Systems in the Field. In Proceedings of MobileHCI'04. Springer-Verlag, Berlin, Heidelberg, 2004, 61--73.
• 10: Brad Myers , Scott E. Hudson , Randy Pausch, Past, present, and future of user interface software tools, ACM Transactions on Computer-Human Interaction (TOCHI), v.7 n.1, p.3-28, March 2000 [doi>10.1145/344949.344959]
• 11: Nigay, L. and Coutaz, J. The CARE Properties and Their Impact on Software Design. Intelligence and Multimodality in Multimedia Interfaces, 1997.
• 12: Sharon Oviatt, Multimodal system processing in mobile environments, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.21-30, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354408]
• 13: Schatz, R., Simon, R., Anegg, H., et al. Developing Mobile Multimodal Applications. In Proceedings of HCI'05.
• 14: Rainer Simon , Florian Wegscheider , Konrad Tolar, Tool-supported single authoring for device independence and multimodality, Proceedings of the 7th international conference on Human computer interaction with mobile devices & services, September 19-22, 2005, Salzburg, Austria [doi>10.1145/1085777.1085793]
• 15: Thevenin, D. and Coutaz, J. Plasticity of User Interfaces: Framework and Research Agenda. In Proceedings of INTERACT'99. IOS Press, 1999.
• 16: Voice Extensible Markup Language (VoiceXML) Version 2.0. W3C Recommendation 16 March 2004. http://www.w3.org/TR/voicexml20/
• 17: XHTML + Voice Profile 1.0, W3C Note 21, December 2001. http://www.w3.org/TR/xhtml+voice/
• 18: Zouinar, M., Salembier, P., Pasqualetti, L., et al. Part 1 Interaction: Chapter 4. Multimodal Interaction on Mobile Artefacts. Communicating with Smart Objects-Developing Technology for Usable Pervasive Computing Systems. Hermes Penton/Kogan Page Science, 2003.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Input devices and strategies (e.g., mouse, touchscreen)

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.2 Design Tools and Techniques
          Subjects: User interfaces

  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: User interface management systems (UIMS); Evaluation/methodology; Prototyping; Interaction styles (e.g., commands, menus, forms, direct manipulation)

General Terms:
Algorithms, Human Factors

Keywords:
field trial, mobile device, mobile multimodal logging, multimodal interface, software component