Tool-supported single authoring for device independence and multimodality

ABSTRACT

With the growing proliferation of mobile computing devices, the vision of the web anytime, anywhere and on any device is rapidly becoming a reality. Technologies enabling device-independent presentation and new interaction modalities like voice or gesture are moving from research to commercially available products. As a result, developers are faced with the increasing challenge of providing user interfaces that match the capabilities of the different devices available. Within this paper we present our application-oriented research that has investigated single authoring of multimodal interfaces on mobile devices. After an overview of related work that explains the motivation behind our approach, we present a prototype authoring tool for the development of graphical as well as multimodal web-based user interfaces for multiple devices. We conclude by discussing the relation of our work to established web markup standards and point out noteworthy issues related to their application within our work.

REFERENCES

• 1: Marc Abrams , Constantinos Phanouriou , Alan L. Batongbacal , Stephen M. Williams , Jonathan E. Shuster, UTML: an appliance-independent XML user interface language, Proceeding of the eighth international conference on World Wide Web, p.1695-1708, May 1999, Toronto, Canada
• 2: Ali, M. F., Pérez-Quiñones, M. A., Abrams, M., Shell, E. "Building Multi-Platform User Interfaces with UIML." 4th International Conference on Computer-Aided Design of User Interfaces (CADUI'2002). France, 2002.
• 3: Authoring Techniques for Device Independence. W3C Working Group Note 18 February 2004. http://www.w3.org/TR/2004/NOTE-di-atdi-20040218/
• 4: Azevedo, P., Merrick, R., Roberts, D. "OVID to AUIML - User Oriented Interface Modeling." http://math.uma.pt/tupis00/submissions/azevedoroberts/azevedoroberts.html
• 5: Silvia Berti , Francesco Correani , Giulio Mori , Fabio Paternò , Carmen Santoro, TERESA: a transformation-based environment for designing and developing multi-device interfaces, CHI '04 extended abstracts on Human factors in computing systems, April 24-29, 2004, Vienna, Austria [doi>10.1145/985921.985939]
• 6: Bouillon, L., Vanderdonckt, J., Souchon, N. "Recovering Alternative Presentation Models of a Web Page with VAQUITA." Proceedings of CADUI'02. Valenciennes, France. May 15-17, 2002.
• 7: Cascading Style Sheets, level 2 revision 1. CSS 2.1 Specification. http://www.w3.org/TR/CSS21/
• 8: Consensus Project. http://www.consensus-online.org/
• 9: Eclipse project Main Page. http://www.eclipse.org
• 10: ECMA International website. http://www.ecma-international.org/publications/standards/Ecma-262.htm
• 11: Murielle Florins , Jean Vanderdonckt, Graceful degradation of user interfaces as a design method for multiplatform systems, Proceedings of the 9th international conference on Intelligent user interface, January 13-16, 2004, Funchal, Madeira, Portugal [doi>10.1145/964442.964469]
• 12: Kirusa company website. http://www.kirusa.com/
• 13: MobileAware company website. http://www.mobileaware.com/
• 14: Brad A. Myers, User interface software tools, ACM Transactions on Computer-Human Interaction (TOCHI), v.2 n.1, p.64-103, March 1995 [doi>10.1145/200968.200971]
• 15: Brad Myers , Scott E. Hudson , Randy Pausch, Past, present, and future of user interface software tools, ACM Transactions on Computer-Human Interaction (TOCHI), v.7 n.1, p.3-28, March 2000 [doi>10.1145/344949.344959]
• 16: Plomp, C. J., Mayora-Ibarra, O. "A Generic Widget Vocabulary for the Generation of Graphical and Speech-Driven User Interfaces." International Journal of Speech Technology, V(5), Issue 1, Kluwer Academic Publishers. January 2002.
• 17: Project MONA hompage. http://mona.ftw.at/
• 18: Angel Puerta , Jacob Eisenstein, XIML: a common representation for interaction data, Proceedings of the 7th international conference on Intelligent user interfaces, January 13-16, 2002, San Francisco, California, USA [doi>10.1145/502716.502763]
• 19: Souchon, N., Vanderdonckt, J. "A Review of XML-Compliant User Interface Description Languages." Proceedings of 10th International Conference on Design, Specification, and Verification of Interactive Systems. Madeira, 4-6 June 2003.
• 20: Speech Application Language Tags (SALT) Forum website. http://www.saltforum.org/
• 21: Szekely, P., "Retrospective and Challenges for Model-Based Interface Development". Proceedings of the 2nd International Workshop on Computer-Aided Design of User Interfaces. Namur University Press. Namur. 1996.
• 22: Hallvard Trætteberg , Pedro J. Molina , Nuno J. Nunes, Making model-based UI design practical: usable and open methods and tools, Proceedings of the 9th international conference on Intelligent user interface, January 13-16, 2004, Funchal, Madeira, Portugal [doi>10.1145/964442.964542]
• 23: Vanderdonckt, J., Limbourg, Q., Michotte, B., Bouillon, L., Trevisan, D., Florins, M. "USIXML: a User Interface Description Language for Specifying Multimodal User Interfaces." W3C Workshop on Multimodal Interaction. Sophia Antipolis, 19-20 July 2004.
• 24: Voice Extensible Markup Language (VoiceXML) Version 2.0. W3C Recommendation 16 March 2004. http://www.w3.org/TR/voicexml20/
• 25: Volantis company website. http://www.volantis.com/
• 26: WebRevEnge Homepage. http://giove.cnuce.cnr.it/webrevenge/index.html
• 27: Wong, C., Chu, H., Katagiri, M., "A Single-Authoring Technique for Building Device-Independent Presentations." W3C Device Independent Authoring Techniques Workshop, St. Leon-Rot, Germany, September 2002.
• 28: XAML.NET, a guide to XAML. http://www.xaml.net/
• 29: XForms 1.0, W3C Recommendation, 14 Oct. 2003. http://www.w3.org/TR/xforms/
• 30: XHTML+Voice Profile 1.0, W3C Note 21, December 2001. http://www.w3.org/TR/xhtml+voice/
• 31: XML Events. An Events Syntax for XML. W3C Recommendation 14 October 2003. http://www.w3.org/TR/2003/REC-xml-events-20031014/
• 32: XML Markup Languages for User Interface Definition. The Coverpages. http://xml.coverpages.org/userInterfaceXML.html
• 33: XML User Interface Language (XUL) Project. http://www.mozilla.org/projects/xul/

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Graphical user interfaces (GUI)

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Theory and methods; Input devices and strategies (e.g., mouse, touchscreen); Standardization; Screen design (e.g., text, graphics, color); Prototyping; Interaction styles (e.g., commands, menus, forms, direct manipulation)

General Terms:
Design, Experimentation, Human Factors, Standardization

Keywords:
device independence, mobile device, modality independence, multimodal interaction, single authoring, universal access, user interfaces, web authoring