Testing & quantifying ERP usability

ABSTRACT

An Enterprise Resource Planning (ERP) system benefits the business, although the typically complex interface can challenge end users whose errors can undermine the benefits of real-time automation. Many prior studies of ERP users measured attitudes rather than use of the ERP. In contrast, this research involved testing users as they worked with PeopleSoft™ to complete an inventory procedure. An experiment measured success and time while users completed a task using the multi-screen default interface and a simplified single-screen version. Complexity of the interfaces was quantified with two models. GOMS-KLM described the interface in cognitive terms and a visual model described the design. The default complex screens required almost twice as many steps to navigate seven times the number of on-screen elements. Trends and participant comments validated the importance of interface usability, although complexity was a significant variable only for time spent working on the task, not success. Task success was dependent on each user scanning and verifying data before submitting it, an observed behavior. In this paper two studies are presented from a larger effort blending human factors and empirical methods to assess ERP usability and to demonstrate the importance of measuring usability and actual use, not just attitudes about an ERP.

References

1. Arnowitz, J., Gray, D., Dorsch, N., Heidelberg, M., and Arent, M. (2005). The stakeholder forest: Designing an expenses application for the enterprise. CHI2005 Design Expo. 941--956.
2. Babaian, T. Lucas, W., and Topi, H. (2007) A data-driven design for deriving usability metrics. ICSOFT 2007 - International Conference on Software and Data Technologies.
3. Bagchi, K. and Dasgupta, S. (2003). Modeling use of enterprise resources planning systems: A path analytic study. European Journal of Information Systems. 12(2), 142--158.
4. Bradley, J. and Lee, C.C. (2007). ERP Training and user satisfaction: A case study. International Journal of Enterprise Information systems. 3(4), 33--50.
5. Calisir, F. and Calisir, F. (2004). The relation of interface usability characteristics, perceived usefulness, and perceived ease of use to end-user satisfaction with enterprise resource planning (ERP). Computers in Human Behavior. 20(4), 505--515.
6. Card, S. K., Moran, T. P., and Newell, A. (1983). The Psychology of Human-Computer Interaction. CRC Press Reprint.
7. Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly. 13(3), 319--340.
8. Esteves, J.M. and Pastor, J.A. (1999). An ERP life-cycle-based research agenda. Enterprise Management and Resource Planning: Methods, Tools and Architectures EMRPS. Venice, Italy.
9. Galitz, W.O. (1994). It's Time to Clean your Windows: Designing GUI's that Work. John Wiley & Sons, Inc.
10. Holleis, P., Otto, F., and Schmidt, A. (2007). Keystroke-level model for advanced mobile phone interaction. Proceedings, Models of Mobile Interaction. April 28-May 3. San Jose, CA.
11. Iansiti, M. (2007). ERP end-user business productivity: A field study of SAP & Microsoft. Keystone Strategy. Retrieved January 30, 2011 from http://download.microsoft.com/download/4/2/7/427edce8-351e-4e60-83d6-28bbf2f80d0b/KeystoneERPAssessmentWhitepaper.pdf.
12. John, B. and Kieras, D. E. (1996). The GOMS family of user interface analysis techniques: Comparison and contrast, ACM Transactions on Computer-Human Interaction. 3(4), 320--351.
13. Kelley, H. (2001). Attributional analysis of computer self-efficacy. Dissertation. School of Business Administration, University of Western Ontario.
14. Kieras, D.E. (1994). GOMS modeling of user interfaces using NGOMSL. Tutorial Notes. CHI'94 Conference on Human Factors in Computer Systems. Boston, MA. April 24--28, 1994.
15. Kieras, D.E. (2001). Using the keystroke-level model to estimate execution times. University of Michigan. Unpublished paper.
16. Kristiansen, R. (2006). Tailoring of ERP user interfaces using an model-based approach. Unpublished. Department of Computer and Information Science, Norwegian University of Science and Technology.
17. Lazar, J., Feng, J.H., and Hochheiser, H. (2010). Research Methods in Human-Computer Interaction. John Wiley & Sons, Ltd.
18. Liu, Y., Feyen, R., and Tsimhoni, O. (2006). Queueing network-model human processor (QN-MHP): A computational architecture for multitask performance in human-machine systems. ACM Transactions on Computer-Human Interaction. 13(1), 37--70.
19. Mayhew, P. (2005). Keystroke level modeling as a cost-justification tool. In Bias, R.G. and Mayhew, D.J., Cost-Justifying Usability. Chapter 16. Elsevier, Inc. Kindle edition.
20. McGee, M. (2004). Master usability scaling: magnitude estimation and master scaling applied to usability measurement. CHI 2004 April 24-29. 6(1), 335--342.
21. Nah, F.F., Tan, X., and Teh S.H. (2004). An empirical investigation on end-users' acceptance of enterprise systems. Information Resources Management Journal. 17(3), 32--53.
22. Parks, N.E. (2011). Assessing ERP usability: A user-centered and business-focused approach. Thesis. Department of Information Design & Corporate Communication, Bentley University.
23. Saitwal, H. Feng, X., Walji, M., Patel, V. and Zhang, J. (2010). Assessing performance of an electronic health record (EHR) using cognitive task analysis. International Journal of Medical Informatics. 79(7), 501--506.
24. Scherer, E. (2005). Results of the ERP user satisfaction survey: Conclusion for the real-world scheduling. i2s GmbH, Zurich. www.hops-research.org/.../M10_62_108200521387xzhhtjyqncedrxz.pdf.
25. Suchman, L.A. (2007). Human-machine Reconfigurations. Second edition. Cambridge University Press. Kindle edition.
26. Topi, H., Lucas, W., and Babaian, T. (2005). Usability issues with an ERP implementation. In Proceedings of the International Conference on Enterprise Information Systems. (ICEIS- 2005), 128--133.
27. Tullis, T.S. and Albert, B. (2008). Measuring the User Experience: Collecting, analyzing, and presenting usability metrics. Morgan Kaufman Publishers. Kindle version.
28. Venkatesh, V & Davis, F.D. (1996). A model of the perceived ease of use: development and test. Decision Sciences. 27(3), 451--481.
29. Wood, S.D. and Kieras, D.E. (2002). Modeling human error for experimentation, training, and error tolerant design. The Interservice/Industry Training, Simulation & Education Conference (I/ITSEC).
30. Yeh, J.Y. (2006). Evaluating ERP performance from the user perspective. IEEE Asia-Pacific Conference on Services Computing (APSCC'06). IEEE Computer Society.
31. Wilcocks, L.P. and Sykes, R. (2000). Enterprise resource planning: the role of the CIO and it function in ERP. Communications of the ACM. 43(4), 32--38.
32. Wu, J.H. and Wang, Y.M. (2007). Measuring ERP success: The key-users' viewpoint of the ERP to produce a viable IS in the organization. Computers in Human Behavior. 23(3), 1582--1596.

Index Terms

• Human-centered computing

  • Interaction design

    • Interaction design process and methods

      • User centered design