Service-oriented autonomic multimodal interaction in a pervasive environment

ABSTRACT

Heterogeneity and dynamicity of pervasive environments require the construction of flexible multimodal interfaces at run time. In this paper, we present how we use an autonomic approach to build and maintain adaptable input multimodal interfaces in smart building environments. We have developed an autonomic solution relying on partial interaction models specified by interaction designers and developers. The role of the autonomic manager is to build complete interaction techniques based on runtime conditions and in conformity with the predicted models. The sole purpose here is to combine and complete partial models in order to obtain an appropriate multimodal interface. We illustrate our autonomic solution by considering a running example based on an existing application and several input devices.

References

1. 1992. A metamodel for the runtime architecture of an interactive system: the UIMS tool developers workshop. SIGCHI Bull. 24, 1 (Jan. 1992), 32--37.
2. Bardin, J., Lalanda, P., Escoffier, C. 2010. Towards an Automatic Integration of Heterogeneous Services and Devices. Proc. of APSCC'10. IEEE, 171--178.
3. Bouchet, J., Nigay, L., Ganille, T. 2004. ICARE software components for rapidly developing multimodal interfaces. Proc. of ICMI'04. ACM, 251--258.
4. Dumas, B., Lalanne, D., Ingold R. 2009. HephaisTK: A Toolkit for Rapid Prototyping of Multimodal Interfaces. Proc. of ICMI-MLMI 2009. ACM 231--232.
5. Escoffier, C., Hall, R. S., Lalanda, P. 2007. iPOJO: an Extensible Service-Oriented Component Framework. Proc. of SCC'07. IEEE, 474--481.
6. France, R., Rumpe, B. 2007. Model-driven Development of Complex Software: A Research Roadmap. Proc. of FOSE'07. IEEE, 37--54.
7. Garcia, I., Pedraza, G., Debbabi, B., Lalanda, P., Hamon, C. 2010. Towards a service mediation framework for dynamic applications. Proc. of APSCC'10. IEEE, 3--10.
8. Garlan, D., Cheng, S. W., Huang, A. C., Schmerl, B., Steenkiste, P. 2004. Rainbow: Architecture-Based Self Adaptation with Reusable Infrastructure. IEEE Computer 37, 10 (Oct. 2004), 46--54.
9. Huebscher, M. C., McCann, J. A. 2008. A survey of autonomic computing. ACM Comput. Surv. 40, 3, Article 7 (Aug. 2008).
10. IBM Corporation. An Architectural Blueprint for Autonomic Computing. http://www-03.ibm.com/autonomic/pdfs/ AC20Blueprint20White20Paper20V7.pdf
11. König, W. A., Rädle, R., Reiterer, H. 2009. Squidy: a zoomable design environment for natural user interfaces. Proc. of CHI EA'09. ACM, 4561--4566.
12. Lalanda, P., Bellissard, L., Balter, R. 2006. Asynchronous Mediation for Integrating Business and Operational Processes. IEEE Internet Computing 10, 1 (Jan. 2006), 56--64.
13. Oviatt, S. 2003. Advances in Robust Multimodal Interface Design. IEEE Comput. Graph. Appl. 23, 5 (Sept. 2003), 62--68.
14. Oviatt, S. 2007. Multimodal interfaces. Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies, and Emerging Applications, 2nd edition. L. Erlbaum Assoc. Inc., Chap. 14, 286--304.
15. Papazoglou, M. P., Georgakopoulos, D. 2003. Service-Oriented Computing: Introduction. Commun. ACM 46, 10 (Oct. 2003), 24--28.
16. Satyanarayanan, M. 2001. Pervasive computing: vision and challenges. IEEE Personal Communications, 8, 4 (Aug. 2001), 10--17.
17. Serrano, M., et al.. 2008. The openinterface framework: a tool for multimodal interaction. Proc. of CHI EA'08. ACM, 3501--3506. www.oi-project.org
18. Serrano, M., Juras, D., Nigay, L. 2008. A three-dimensional characterization space of software components for rapidly developing multimodal interfaces. Proc.of ICMI'08. ACM, 149--156.
19. Serrano, M., Nigay, L. 2009. Temporal aspects of CARE-based multimodal fusion. Proc. of ICMI-MLMI'09. ACM, 177--184.
20. Weiser, M. 1991. The computer for the 21st century. Scientific American, 265, 3 (Sept. 1991), 66--75.

Index Terms

• Software and its engineering

  • Software notations and tools

    • Development frameworks and environments

      • Integrated and visual development environments