SAMBA: a semi-automatic method for measuring barriers of accessibility

ABSTRACT

Although they play an important role in any assessment procedure, web accessibility metrics are not yet well developed and studied. In addition, most metrics are geared towards conformance, and therefore are not well suited to answer questions whether the web site has critical barriers with respect to some user group.

The paper addresses some open issues: how can accessibility be measured other than by conformance to certain guidelines? How can a metric merge results produced by accessibility evaluation tools and by expert reviewers? Does it consider error rates of the tool? How can a metric consider also severity of accessibility barriers? Can a metric tell us if a web site is more accessible for certain user groups rather than others?.

The paper presents a new methodology and associated metric for measuring accessibility that efficiently combine expert reviews with automatic evaluation of web pages. Examples and data drawn from tests performed on 1500 web pages are also presented.

REFERENCES

• 1: M. Arrue, M. Vigo, and J. Abascal. Quantitative allallmetrics for web accessibility evaluation. In Proceedings of the ICWE 2005 Workshop on Web Metrics and Measurement, 2005.
• 2: Giorgio Brajnik, Comparing accessibility evaluation tools: a method for tool effectiveness, Universal Access in the Information Society, v.3 n.3, p.252-263, October 2004 [doi>10.1007/s10209-004-0105-y]
• 3: G. Brajnik. Web Accessibility Testing: When the Method is the Culprit. In K. Miesenberger, J. Klaus, W. Zagler, and A. Karshmer, editors, ICCHP 2006, 10th International Conference on Computers Helping People with Special Needs, Lecture Notes in Computer Science 4061, Linz, Austria, July 2006. Springer Verlag.
• 4: G. Brajnik. Web accessibility testing with barriers walkthrough. www.dimi.uniud.it/giorgio/projects/bw, March 2006. Visited: May 2007.
• 5: G. Brajnik. Ranking websites through prioritized web accessibility barriers. In Technology and Persons with Disabilities Conference, Los Angeles, March 2007. CSUN, California State University Northridge. www.dimi.uniud.it/giorgio/papers/csun07.pdf.
• 6: Giorgio Brajnik , Andrea Mulas , Claudia Pitton, Effects of sampling methods on web accessibility evaluations, Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibility, October 15-17, 2007, Tempe, Arizona, USA [doi>10.1145/1296843.1296855]
• 7: T. DeMarco, Controlling Software Projects: Management, Measurement, and Estimates, Prentice Hall PTR, Upper Saddle River, NJ, 1986
• 8: A. Sears. Heuristic walkthroughs: finding the problems without the noise. Int. Journal of Human Computer Interaction, 9(3):213--234, 1997.
• 9: John M. Slatin , Sharron Rush, Maximum Accessibility: Making Your Web Site More Usable for Everyone, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 2002
• 10: Terry Sullivan , Rebecca Matson, Barriers to use: usability and content accessibility on the Web's most popular sites, Proceedings on the 2000 conference on Universal Usability, p.139-144, November 16-17, 2000, Arlington, Virginia, United States [doi>10.1145/355460.355549]
• 11: Andrew Kirkpatrick , Richard Rutter , Christian Heilmann , Jim Thatcher , Cynthia Waddell, Web Accessibility: Web Standards and Regulatory Compliance, friends of ED, 2006
• 12: U. S. Government. SEC. 508. ELECTRONIC AND INFORMATION TECHNOLOGY, 1998 Amendment to Section 508 of the Rehabilitation Act. www.section508.gov/index.cfm?FuseAction=Content&ID=14, 1998. Visited: January 2007.
• 13: E. Velleman, C. A. Velasco, M. Snaprud, and D. Burger. D-WAB4 Unified Web Evaluation Methodology (UWEM 1.0). Technical report, WAB Cluster, 2006.
• 14: Markel Vigo , Myriam Arrue , Giorgio Brajnik , Raffaella Lomuscio , Julio Abascal, Quantitative metrics for measuring web accessibility, Proceedings of the 2007 international cross-disciplinary conference on Web accessibility (W4A), May 07-08, 2007, Banff, Canada [doi>10.1145/1243441.1243465]
• 15: X. Zeng. Evaluation of Enhancement of Web Content Accessibility for Persons with Disabilities. PhD thesis, University of Pittsburgh, 2004.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Evaluation/methodology

  K. Computing Milieux
  K.6 MANAGEMENT OF COMPUTING AND INFORMATION SYSTEMS
      K.6.4 System Management
          Subjects: Management audit; Quality assurance; Centralization/decentralization

General Terms:
Human Factors, Measurement

Keywords:
accessibility, accessibility metric, evaluation method, quality assessment, web accessibility