SAMBA: a semi-automatic method for measuring barriers of accessibility

ABSTRACT

Although they play an important role in any assessment procedure, web accessibility metrics are not yet well developed and studied. In addition, most metrics are geared towards conformance, and therefore are not well suited to answer questions whether the web site has critical barriers with respect to some user group.

The paper addresses some open issues: how can accessibility be measured other than by conformance to certain guidelines? How can a metric merge results produced by accessibility evaluation tools and by expert reviewers? Does it consider error rates of the tool? How can a metric consider also severity of accessibility barriers? Can a metric tell us if a web site is more accessible for certain user groups rather than others?.

The paper presents a new methodology and associated metric for measuring accessibility that efficiently combine expert reviews with automatic evaluation of web pages. Examples and data drawn from tests performed on 1500 web pages are also presented.

References

1. M. Arrue, M. Vigo, and J. Abascal. Quantitative allallmetrics for web accessibility evaluation. In Proceedings of the ICWE 2005 Workshop on Web Metrics and Measurement, 2005.
2. G. Brajnik. Comparing accessibility evaluation tools: method for tool effectiveness. Universal Access in the Information Society, 3(3-4):252--263, Oct 2004. www.springerlink.com/openurl.asp?genre=article&id=doi:10.1007/s10209-004-0105-y.
3. G. Brajnik. Web Accessibility Testing: When the Method is the Culprit. In K. Miesenberger, J. Klaus, W. Zagler, and A. Karshmer, editors, ICCHP 2006, 10th International Conference on Computers Helping People with Special Needs, Lecture Notes in Computer Science 4061, Linz, Austria, July 2006. Springer Verlag.
4. G. Brajnik. Web accessibility testing with barriers walkthrough. www.dimi.uniud.it/giorgio/projects/bw, March 2006. Visited: May 2007.
5. G. Brajnik. Ranking websites through prioritized web accessibility barriers. In Technology and Persons with Disabilities Conference, Los Angeles, March 2007. CSUN, California State University Northridge. www.dimi.uniud.it/giorgio/papers/csun07.pdf.
6. G. Brajnik, A. Mulas, and C. Pitton. Effects of sampling methods on web accessibility evaluations. In S. Trewin, editor, 9th Int. ACM SIGACCESS Conference on Computers and Accessibility, ASSETS, Tempe, AZ, Oct 2007. ACM Press.
7. T. DeMarco. Controlling Software Projects: Management, Measurement, and Estimates. Prentice Hall PTR, 1986.
8. A. Sears. Heuristic walkthroughs: finding the problems without the noise. Int. Journal of Human Computer Interaction, 9(3):213--234, 1997.
9. J. Slatin and S. Rush. Maximum Accessibility: Making Your Web Site More Usable for Everyone. Addison-Wesley, 2003.
10. T. Sullivan and R. Matson. Barriers to use: usability and content accessibility on the web's most popular sites. In Proc. of ACM Conference on Universal Usability, pages 139--144, 2000.
11. J. Thatcher, M. Burks, C. Heilmann, S. Henry, A. Kirkpatrick, P. Lauke, B. Lawson, B. Regan, R. Rutter, M. Urban, and C. Waddell. Web Accessibility: Web Standards and Regulatory Compliance. Friends of ED, 2006.
12. U. S. Government. SEC. 508. ELECTRONIC AND INFORMATION TECHNOLOGY, 1998 Amendment to Section 508 of the Rehabilitation Act. www.section508.gov/index.cfm?FuseAction=Content&ID=14, 1998. Visited: January 2007.
13. E. Velleman, C. A. Velasco, M. Snaprud, and D. Burger. D-WAB4 Unified Web Evaluation Methodology (UWEM 1.0). Technical report, WAB Cluster, 2006.
14. M. Vigo, M. Arrue, G. Brajnik, R. Lomuscio, and J. Abascal. Quantitative metrics for measuring web accessibility. In W4A '07: Proceedings of the 2007 International Cross-Disciplinary Workshop on Web Accessibility (W4A), pages 99--107, New York, NY, USA, 2007. ACM Press. www.dimi.uniud.it/giorgio/papers/w4a07.pdf.
15. X. Zeng. Evaluation of Enhancement of Web Content Accessibility for Persons with Disabilities. PhD thesis, University of Pittsburgh, 2004.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • HCI design and evaluation methods

• Social and professional topics

  • Professional topics

    • Management of computing and information systems

      • System management

        • Centralization / decentralization

        • Quality assurance

        • Technology audits