MyUI: generating accessible user interfaces from multimodal design patterns

ABSTRACT

Adaptive user interfaces can make technology more accessible. Quite a number of conceptual and technical approaches have been proposed for adaptations to diverse user needs, multiple devices or multiple environments. Little work, however, has been directed at integrating all the essential aspects of adaptive user interfaces for accessibility in one system. In this paper, we present our generic MyUI infrastructure for increased accessibility through automatically generated adaptive user interfaces. The multimodal design patterns repository serves as the basis for a modular approach to individualized user interfaces. This open and extensible pattern repository makes the adaptation rules transparent for designers and developers who can contribute to the repository by sharing their knowledge about accessible design. The adaptation architecture and procedures enable user interface generation and dynamic adaptations during run-time. For the specification of an abstract user interface model, a novel statecharts-based notation has been developed. A development tool supports the interactive creation of the graphical user interface model.

References

1. Abrams, M., Phanouriou, C., Batongbacal, A. L., Williams, S. & Shuster, J. (1999), UIML: An Appliance-Independent XML User Interface Language. In Proc. WWW'8, Amsterdam: Elsevier, 1695--1708
2. Ali, M. F., Pérez-Quiñones, M. A., Abrams, M., & Shell, E. (2002). Building Multi-Platform User Interfaces with UIML. In Proceedings CADUI 2002, 255--266.
3. Borchers, J. O. (2001). A pattern approach to interactive design. Chichester, UK: John Wiley & Sons Ltd.
4. Bunt, A., Conati, C., & McGrenere, J. (2009). A Mixed-Initiative Approach to Interface Personalization. AI Magazine 30(4).
5. Coelho, J., Duarte, C., Biswas, P., & Langdon, P. (2011). Developing accessible TV applications. In Proceedings ASSETS '11. New York: ACM, 131--138.
6. Dieterich, H., Malinowski, U., Kühme, T. & Schneider-Hufschmidt, M. (1993). State of the Art in Adaptive User Interfaces. In: M. Schneider-Hufschmidt, T. Kühme & U. Malinowski (Eds.): Adaptive User Interfaces: Principles and practice. Amsterdam: North-Holland, 13--48.
7. Dessart, C.-E., Motti, V. G. & Vanderdonckt, J. (2011). Showing user interface adaptivity by animated transitions. In Proc. EICS '11. New York: ACM, 95--104.
8. Duarte, C. & Carriço, L (2006). A conceptual framework for developing adaptive multimodal applications. In Proc. IUI '06. New York: ACM, 132--139.
9. Edlin-White, R., Cobb, S., Floyde, A., Lewthwaite, S., Wang, J. & Riedel, J. (2012). From guinea pigs to design partners - involving older people in technology design. In: P. Langdon, J. Clarkson, P. Robinson, J. Lazar & A. Heylighen (Eds.): Designing Inclusive Systems. London: Springer-Verlag, 155--164.
10. Findlater, L. & McGrenere, J. (2004) A comparison of static, adaptive, and adaptable menus. In Proceedings CHI'04, New York: ACM, 89--96.
11. Gajos, K. Z., Weld, D. S., & Wobbrock, J. O. (2010). Automatically generating personalized user interfaces with Supple. Artificial Intelligence 174, 12--13. 910--950.
12. Kane, S. K., Wobbrock, J. O. & Smith, I. E. (2008). Getting off the treadmill: evaluating walking user interfaces for mobile devices in public spaces. In Proceedings MobileHCI '08. New York: ACM, 109--118.
13. Lehmann, G., Blumendorf, M. & Albayrak, S. (2010). Development of context-adaptive applications on the basis of runtime user interface models. In Proceedings EICS '10. New York: ACM, 309--314.
14. Lin, J. & Landay, J. A. (2008). Employing patterns and layers for early-stage design and prototyping of cross-device user interfaces. In Proceedings CHI '08. New York: ACM, 1313--1322.
15. Mackay, W. E. (1991). Triggers and barriers to customizing software. In Proceedings CHI '91, New York: ACM, 153--160.
16. Meskens, J., Vermeulen, J., Luyten, K. & Coninx, K. (2008). Gummy for multi-platform user interface designs: shape me, multiply me, fix me, use me. In Proceedings AVI '08. New York: ACM, 233--240.
17. Motti, V. G. (2011). A computational framework for multi-dimensional context-aware adaptation. In Proceedings EICS '11. New York: ACM, 315--318.
18. Myers, B., Hudson, S. E. & Pausch, R. (2000). Past, present, and future of user interface software tools. ACM Trans. Comp.-Hum. Interact. 7, 1, 3--28.
19. MyUI Design Patterns Repository. Available at http://myuipatterns.clevercherry.com
20. Nichols, J., Chau, D. H., & Myers, B. A. (2007). Demonstrating the viability of automatically generated user interfaces. In Proceedings CHI '07. New York: ACM, 1283--1292.
21. Nichols, J & Myers, B. A. (2009). Creating a lightweight user interface description language: An overview and analysis of the personal universal controller project. ACM Trans. Comp.-Hum. Interact. 16, 4, 37 pages.
22. Nichols, J., Myers, B. A. & Rothrock, B. (2006). UNIFORM: automatically generating consistent remote control user interfaces. In Proceedings CHI '06. New York: ACM, 611--620.
23. Nichols, J., Rothrock, B., Chau, D. H. & Myers, B. A. (2006). Huddle: automatically generating interfaces for systems of multiple connected appliances. In Proceedings UIST '06. New York: ACM, 279--288.
24. Object Management Group (2011). Unified Modeling Language Superstructure, V 2.4.1, available at http://www.omg.org/spec/UML/2.4.1/Superstructure
25. Paterno, F. (1999). Model-Based Design and Evaluation of Interactive Applications. London: Springer-Verlag.
26. Paterno, F., Santoro, C., Mantyjarvi, J., Mori, G. & Sansone, S. (2008). Authoring pervasive multimodal user interfaces. Int. J. Web Eng. Technol. 4, 2, 235--261.
27. Paterno, F., Santoro, C. & Spano, L. D. (2009). MARIA: A universal, declarative, multiple abstraction-level language for service-oriented applications in ubiquitous environments. ACM Trans. Comp.-Hum. Interact. 16, 4, 30 pages.
28. Peissner, M., Schuller, A. & Spath, D. (2011). A design patterns approach to adaptive user interfaces for users with special needs. In Proceedings HCII'11, Berlin: Springer-Verlag, 268--277.
29. Ringbauer, B., Peissner, M., & Gemou, M. (2007). From "design for all" towards "design for one" - A modular user interface approach. In: C. Stephanidis (Ed.): Universal Access in HCI, Part I, HCII 2007, LNCS 4554, Berlin: Springer-Verlag, 517--526.
30. Savidis, A. & Stephanidis, C. (2004). Unified user interface design: Designing universally accessible interactions. Int. J. Interacting w. Comp. 16, 2, 243--270.
31. Stephanidis, C., Paramythis, A., Sfyrakis, M., Stergiou, A., Maou, N., Leventis, A., Paparoulis, G. & Karagiannidis, C. (1998). Adaptable and adaptive user interfaces for disabled users in the AVANTI project, in: IS&N 98 Proceedings, Springer-Verlag, 153--166.
32. Trewin, S., Zimmermann, G. & Vanderheiden, G. (2002). Abstract user interface representations: how well do they support universal access? SIGCAPH Comput. Phys. Handicap. 73--74 (June 2002), 77--84.
33. Van den Bergh, J., Luyten, K. & Coninx, K. (2011). CAP3: context-sensitive abstract user interface specification. In Proceedings EICS '11. New York: ACM, 31--40.
34. Weld, D., Anderson, C., Domingos, P., Etzioni, O., Lau, T., Gajos, K. & Wolfman, S. (2003). Automatically personalizing user interfaces. In Proceedings IJCAI'03. San Francisco: Morgan Kaufmann, 1613--1619.
35. Wobbrock, J. O., Kane, S. K., Gajos, K. Z., Harada, S., & Froehlich, J. (2011). Ability-based design: Concept, principles and examples. ACM Transactions on Accessible Computing, Vol. 3, No. 3, Article 9.

Index Terms

• Human-centered computing

  • Interaction design

    • Interaction design process and methods

      • User centered design

• Software and its engineering

  • Software organization and properties

    • Software system structures

      • Software architectures