Automatic support for web user studies with SCONE and TEA

ABSTRACT

This paper describes the concepts of TEA, a flexible tool that supports user tests by automating repetitive tasks and collecting data of user inputs and actions. TEA was specifically designed for user studies in the World Wide Web and is able to interact with a web browser. Building on a web intermediary (WBI) and a framework for web enhancement tools (SCONE), TEA can be applied in a range of test settings - providing either a controlled laboratory environment or a quick tool for collecting informal data.

REFERENCES

• 1: Nigel Bevan , Carol Barnum , Gilbert Cockton , Jakob Nielsen , Jared Spool , Dennis Wixon, The "magic number 5": is it enough for web testing?, CHI '03 extended abstracts on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/765891.765936]
• 2: John P. Chin , Virginia A. Diehl , Kent L. Norman, Development of an instrument measuring user satisfaction of the human-computer interface, Proceedings of the SIGCHI conference on Human factors in computing systems, p.213-218, May 15-19, 1988, Washington, D.C., United States [doi>10.1145/57167.57203]
• 3: Etgen, M. & Cantor, J.: What does getting WET (Web Event-logging Tool) Mean for Web Usability? Proc. of NSCL Human Factors & the Web Conference, 1999.
• 4: WebQuilt: A proxy-based approach to remote web usability testing, ACM Transactions on Information Systems (TOIS), v.19 n.3, p.263-285, July 2001 [doi>10.1145/502115.502118]
• 5: Melody Y. Ivory , Marti A Hearst, The state of the art in automating usability evaluation of user interfaces, ACM Computing Surveys (CSUR), v.33 n.4, p.470-516, December 2001 [doi>10.1145/503112.503114]
• 6: Melody Y. Ivory , Rashmi R. Sinha , Marti A. Hearst, Empirically validated web page design metrics, Proceedings of the SIGCHI conference on Human factors in computing systems, p.53-60, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365035]
• 7: Claire-Marie Karat , Robert Campbell , Tarra Fiegel, Comparison of empirical testing and walkthrough methods in user interface evaluation, Proceedings of the SIGCHI conference on Human factors in computing systems, p.397-404, May 03-07, 1992, Monterey, California, United States [doi>10.1145/142750.142873]
• 8: Paul Maglio , Rob Barrett, Intermediaries personalize information streams, Communications of the ACM, v.43 n.8, p.96-101, Aug. 2000 [doi>10.1145/345124.345158]
• 9: Mayer, M. & Bederson, B. B.: Browsing Icons: A Task-Based Approach for a Visual Web History, HCIL-2001-19, CS-TR-4308, UMIACS-TR-2001-85, HCI Lab, University of Maryland, Maryland, USA, 2001.
• 10: Jakob Nielsen, Usability Engineering, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1995
• 11: Hartmut Obendorf , Harald Weinreich, Comparing link marker visualization techniques: changes in reading behavior, Proceedings of the 12th international conference on World Wide Web, May 20-24, 2003, Budapest, Hungary [doi>10.1145/775152.775255]
• 12: Scholtz, J., Laskowski, S. & Downey, L.: Developing Usability Tools and Techniques for Designing and Testing Web Sites, 4th Annual Human Factors and the Web Conference, June 5, ATT, NJ., 1998.
• 13: Dennis Wixon, Evaluating usability methods: why the current literature fails the practitioner, interactions, v.10 n.4, July + August 2003 [doi>10.1145/838830.838870]
• 14: Weinreich, H., Buchmann, V., Lamersdorf, W.: Scone: Ein Framework zur evaluativen Realisierung von Erweiterungen des Webs, KiVS '03, Springer, 2003, 31--42.
• 15: Harald Weinreich , Winfried Lamersdorf, Concepts for improved visualization of Web link attributes, Computer Networks: The International Journal of Computer and Telecommunications Networking, v.33 n.1-6, p.403-416, June 2000

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Evaluation/methodology

General Terms:
Experimentation, Human Factors

Keywords:
WWW, test automation, usability testing methods