Usability Information Management: Prototype for Result Exploration Based on an Empirical Analysis of Use Cases

ABSTRACT

Empirical usability results from user research and usability evaluations are valuable as an information resource beyond the scope of the project for which they were originally collected [22]. This study presents the results of interviews with usability practitioners (n = 8) to describe existing use cases for this type of information. Based on the use cases, an information structure for usability information has been created. Its application was discussed in focus groups in two organizations. As a result, an information management system for usability information was implemented and evaluated with users in a qualitative study (n = 11). The results indicate the diversity of existing use cases. These include direct application to current design questions, exploration of results with the aim of learning and devising generalized internal guidelines, and analyzing results to support tracking of implementation, controlling, and reporting. Aspects of the product and the respective context of use appear to be essential to support exploration of usability results. In addition, information that helps to make judgements about their reliability is necessary to enable systematic reuse.

References

1. Andre, T.S., Hartson, R. and Williges, R.C. Determining the effectiveness of the usability problem inspector: a theory-based model and tool for finding usability problems. Human Factors 45, 3 (2003), 455--482.
2. Blandford, A., Attfield, S. Interacting with Information. San Rafael, Morgan & Claypool, 2010.
3. Carroll, J.M., Singley, M.K., Rosson. M.B. Integrating theory development with design evaluation. Behaviour & Information Technology 11, 5 (1992), 247--255.
4. Chilana, P.K., Wobbrock, J.O., Ko, A.J. Understanding usability practices in complex domains. SIGCHI 2010, ACM (2010), 2337--2346. http://doi.org/10.1145/1753326.1753678
5. Følstad, A., Law, E.L.-C., Hornbæk, K. Analysis in practical usability evaluation: a survey study. SIGCHI 2012, ACM (2012), 2127--2136. http://doi.org/10.1145/2208276.2208365
6. Furniss, F., Blandford, A., Curzon, P. Usability work in professional website design: insights from practitioners' perspectives. In Maturing Usability, Law, E.L.-C., Hvannberg, E.T., Cockton, G. (eds.), Springer, London, 2008, pp. 144--167.
7. Grigoreanu, V., Mohanna, M. Informal cognitive walkthroughs (ICW): paring down and pairing up for an agile world. SIGCHI 2013, ACM (2013), 3093--3096. http://doi.org/10.1145/2470654.2466421
8. Ham, D.H. A model-based framework for classifying and diagnosing usability problems. Cognition, Technology & Work (2013), 1--16.
9. Haynes, S.R., Carroll, J.M., Rosson, M.B. Integrating user-centered design knowledge with scenarios. In Human-Centered Software Engineering -- Integrating Usability in the Software Development Lifecycle, Gulliksen, J., Seffah, A. (eds.). Springer, Dordrecht, 2005, pp. 269--286.
10. Heo, J., Ham, D.H., Park, S., Song, C., Yoon, W.C. A framework for evaluating the usability of mobile phones based on multi-level, hierarchical model of usability factors. Interacting with Computers 21, 4 (2009), 263--275.
11. Hornbæk, K., Frøkjær, E. Comparison of techniques for matching of usability problem descriptions. Interacting with Computers 20, 6 (2008), 505--514.
12. Howarth, J.R., Smith-Jackson, T., Hartson, R. Supporting novice usability practitioners with usability engineering tools. International Journal of Human and Computer Studies 67, 6 (2009), 533--549.
13. Hughes, M. A pattern language approach to usability knowledge management. Journal of Usability Studies 1, 2 (2006), 76--90.
14. Hughes, M., Reeves, T. Team learning in usability testing. In Qualitative research in technical communication, Conklin, J. Hayhoe, G.F. (eds.). Routledge, New York, NY {u.a.}, 2011, pp. 331--345.
15. International Organization for Standardization. ISO/IEC TR 25060 -- Common Industry Format (CIF) for usability -- General framework for usability-related information, 2010.
16. Laugwitz, B., Held, T., Schrepp, M. Construction and Evaluation of a User Experience Questionnaire. HCI and Usability for Education and Work, Springer, 2008, pp. 63--76.
17. Pyla, P.S., Howarth, J.R., Catanzaro, C., North, C. Vizability: a tool for usability engineering process improvement through the visualization of usability problem data. Proc. 44th Annu. ACM Southeast Regional Conf., ACM (2006), 620--625. http://doi.org/10.1145/1185448.1185584
18. Regli, W.C., Hu, X., Atwood, M.E., Sun, W. A survey of design rationale systems: Approaches, representation, capture and retrieval. Engineering with Computers 16, 3 (2000), 209--235.
19. Rice, S., Thaker, J., and Wichansky, A.M. ISO 25062 usability test planning for a large enterprise applications suite. DUXU 2011, Springer (2011), 185--192. http://doi.org/10.1007/978-3-642-21675-6_22
20. Riege, A. Three-dozen knowledge-sharing barriers managers must consider. Journal of Knowledge Management 9, 3 (2005), 18--35. http://doi.org/http://dx.doi.org/10.1108/13673270510602746
21. Roschuni, C., Goodman, E., Agogino, A.M. Communicating actionable user research for human-centered design. Artificial Intelligence for Engineering Design, Analysis and Manufacturing 27, Special Issue 02 (2013), 143--154. http://doi.org/10.1017/S0890060413000048
22. Rosenbaum, R. The future of usability evaluation: increasing impact on value. In Maturing Usability, Law, E.L.-C., Hvannberg, E.T., Cockton, G. (eds.). Springer, London, 2008, pp. 344--378.
23. Rosson, M.B., Carroll, J.M. Usability Engineering: Scenario-Based Development of Human--Computer Interaction. San Francisco, Morgan Kaufmann, 2002.
24. Rosson, M.B., Maass, S., Kellogg, W.A. Designing for designers: an analysis of design practice in the real world. SIGCHI 1987, ACM (1987), 137--142. http://doi.org/10.1145/29933.30873
25. Sutcliffe, A. The domain theory: patterns for knowledge and software reuse. L. Erlbaum Associates, Mahwah, NJ, 2002.
26. Venkatesh, V., Bala, H. Technology acceptance model 3 and a research agenda on interventions. Decision Sciences 39, 2 (2008), 273--315.
27. Vilbergsdottir, S.G., Hvannberg, E.T., Law, E.L.-C. Assessing the reliability, validity and acceptance of a classification scheme of usability problems (CUP). Journal of Systems and Software 87 (2014), 18--37.
28. Woolrych, A., Hornbæk, K., Frøkjær, E., Cockton, G. Ingredients and meals rather than recipes: a proposal for research that does not treat usability evaluation methods as indivisible wholes. International Journal of Human--Computer Interaction 27, 10 (2011), 940--970. http://doi.org/10.1080/10447318.2011.555314

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)