skip to main content
10.1145/1535654.1535662acmotherconferencesArticle/Chapter ViewAbstractPublication Pagesw4aConference Proceedingsconference-collections
research-article

Transition of accessibility evaluation tools to new standards

Published: 20 April 2009 Publication History

Abstract

While automatic tools are not intended to replace human judgement they are crucial in order to develop accessible web sites. The release of WCAG 2.0 entails that the existing plethora of accessibility review tools will have to be updated. This paper presents an evaluation framework for making the transition from WCAG 1.0 to WCAG 2.0 in a less painful way. A framework is thus proposed that allows developers to create new guidelines, and update or reuse the existing ones. A case study to test its feasibility has been carried out by incorporating WCAG 2.0 guidelines into the framework. The results are satisfactory, since 55% of the automatic and 16% of the semi-automatic ones could be expressed using the framework. Therefore, it is demonstrated that even if the framework does not fully support the transition process, at least it makes it less burdensome. Moreover, by analyzing WCAG 2.0 we have learnt how to extend the existing tools in order to provide greater coverage and thus increase their effectiveness.

References

[1]
Chisholm, W., Vanderheiden, G., and Jacobs, I. (Eds.). (1999, May 5). Web Content Accessibility Guidelines 1.0. (W3C Recommendation). Available at http://www.w3.org/TR/WAI-WEBCONTENT/
[2]
Freed, G., Rothberg, M., and Wlodkowski, T. (2003). Making Educational Software and Web Sites Accessible. Available at http://ncam.wgbh.org/cdrom/guideline/
[3]
Kurniawan, S., and Zaphiris, P. (2005). Research-derived web design guidelines for older people. Proceedings of the ACM SIGACCESS Conference on Computers and Accessibility, ASSETS'05, 129--135. ACM Press.
[4]
Rabin, J., and McCathieNevile, C. (Eds.). (2008, July 29). Mobile Web Best Practices (W3C Recommendation). http://www.w3.org/TR/mobile-bp/
[5]
Caldwell, B., Cooper, M., Guarino Reid, L., and Vanderheiden, G. (Eds.). (2008, December 11). Web Content Accessibility Guidelines 2.0 (W3C Recommendation). Available at http://www.w3.org/TR/WCAG20/
[6]
Ivory, M. Y., and Hearst M. A. (2001). The state of art in automating usability evaluations of user interfaces. ACM Computing Surveys 33(4), 470--516. ACM Press.
[7]
Brajnik, G. (2004). Comparing accessibility evaluation tools: a method for tool effectiveness. Universal Access in the Information Society 3(3--4), 252--263. Springer.
[8]
Ivory, M. Y., Mankoff, J., and Le, A. (2003). Using Automated Tools to Improve Web Site Usage by Users with Diverse Abilities. Information Technology and Society 1(3), 195--236. SIQSS.
[9]
Brajnik, G. (2000). Automatic web usability evaluation: what needs to be done? In Proceedings of the 6th Human Factors and the Web Conference.
[10]
Abou-Zahra, S. (Ed.). (2006). Selecting Web Accessibility Evaluation Tools. Available at http://www.w3.org/WAI/eval/selectingtools
[11]
Xiong, J., Farenc, C., and Winckler, M. (2007). Analyzing Tool Support for Inspecting Accessibility Guidelines During the Development Process of Web Sites. WISE Workshops 2007. LNCS 4832, 470--480. Springer.
[12]
Brinck, T., Hermann, D., Minnebo, B., and Hakim, A. (2002). AccessEnable: A Tool for Evaluating Compliance with Accessibility Standards. In Automatically Evaluating the Usability of Web Sites, CHI'02 Workshop.
[13]
Beirekdar, A., Vanderdonckt, J., and Noirhomme-Fraiture, M. (2002). KWARESMI - Knowledge-based Web Automated Evaluation with Reconfigurable Guidelines Optimization. In Proceedings of 9th International Workshop on Design, Specification, and Verification of Interactive Systems DSV-IS 2002. 362--376.
[14]
Beirekdar, A., Vanderdonckt, J., and Noirhomme-Fraiture, M. (2002). A Framework and a Language for Usability Automatic Evaluation of Web Sites by Static Analysis of HTML Source Code. Chapter 29, Conference on Computer-Aided Design of User Interfaces, CADUI'02, 337--348. Kluwer.
[15]
Vanderdonckt, J., and Bereikdar, A. (2005). Automated Web Evaluation by Guideline Review. Journal of Web Engineering. Rinton Press 4(2), 102--117. Rinton Press.
[16]
Leporini, B., Paternò, F., and Scorcia, A. (2006). Flexible tool support for accessibility evaluation. Interacting with Computers 18(5), 869--890. Elsevier.
[17]
Abascal, J., Arrue, M., Fajardo, I., Garay, N., and Tomás, J. (2004). Use of Guidelines to automatically verify web accessibility. Universal Access in the Information Society 3(1), 71--79. Springer.
[18]
Arrue, M., Vigo, M., and Abascal, J. (2008). Including Heterogeneous Web Accessibility Guidelines in the Development Process. Engineering Interactive Systems, EIS'07. LNCS 4940, 620--37. Springer.
[19]
Takata, Y., Nakamura, T., and Seki, H. (2004). Accessibility Verification of WWW Documents by an Automatic Guideline Verification Tool. Proceedings of the 37th Hawaii International Conference on System Sciences.
[20]
Luque, V., Delgado, C., Gaedke, M., and Nussbaumer, M. (2005). WCAG Formalization with W3C Techniques. Web Engineering. LNCS 3579, 615--617. Springer.
[21]
Vanderdonckt, J. (1999). Development milestones towards a tool for working with guidelines. Interacting with Computers 12(2), 81--118. Elsevier.
[22]
Abascal, J., and Nicolle, C. (2001). Why Inclusive Design Guidelines? In J. Abascal, C. Nicolle (Eds.), Inclusive Design Guidelines for HCI, Chapter 1, 3--13. Taylor & Francis.
[23]
Mariage, C., Vanderdonckt, J., and Pribeanu, C. (2005). State of the Art of Web Usability Guidelines. In Proctor, R. and Vu, K. (Eds.). The Handbook of Human Factors in Web Design, Chapter 38, 688--700. Lawrence Erlbaum.
[24]
IBM Accessibility Center: Developer guidelines for Web Accessibility. Available at http://www-306.ibm.com/able/guidelines/web/accessweb.html
[25]
Grammenos, D., Akoumianakis, D., and Stephanidis, C (2000). Integrated support for working with guidelines: the Sherlock guideline management system. Interacting with Computers 12(3), 281--311. Elsevier.
[26]
Leporini, B., Paternò, F., and Scorcia, A. (2006). An Environment for Defining and Handling Guidelines for the Web. ICCHP'06. LNCS 4061, 176--183. Springer.

Cited By

View all
  • (2023)A platform to check website compliance with web accessibility standardsProceedings of the 20th International Web for All Conference10.1145/3587281.3587289(75-78)Online publication date: 30-Apr-2023
  • (2020)Providing Continuous Web Accessibility EvaluationInteractivity and the Future of the Human-Computer Interface10.4018/978-1-7998-2637-8.ch001(1-26)Online publication date: 2020
  • (2017)Ontology Design for Automatic Evaluation of Web User Interface Usability2017 Portland International Conference on Management of Engineering and Technology (PICMET)10.23919/PICMET.2017.8125425(1-8)Online publication date: Jul-2017
  • Show More Cited By

Recommendations

Reviews

Goran Trajkovski

As the Web is evolving, so are our expectations. The accessibility of Web interfaces has been a focal point of standardization efforts by professional and governmental forums. Most of the existing national guidelines are based on the first version of the World Wide Web Consortium's Web accessibility initiative document called Web Content Accessibility Guidelines (WCAG), published in 1999. This document has some loose ends, left to interpretation by Web developers, and has also been criticized for lack of attention to specific user demographics such as the elderly and mobile device users. Multiple tools have been developed in an attempt to control the time, effort, and resources needed to check for compliance to these guidelines in the life cycle of a Web application. With the publication of the WCAG 2.0 in 2008, these tools face the challenge of aligning with the new, clearer, better-specified guidelines in the document. Aizpurua et al. look into the challenges that the accessibility evaluation tools may face in this transition. The tool EvalAccess-developed by the authors' research team-is used to investigate to what degree WCAG 1.0 can be used for the new guidelines. The tool features an evaluation engine separate from the logic of the guidelines. It is depicted as a flexible tool that might not solve all the transitional problems, but makes the automated and semi-automated checking of the guidelines fairly efficient. The study shows that 55 percent of the automatic and 16 percent of the semi-automatic checks of WCAG 2.0 can be performed using this framework, initially developed for WCAG 1.0. Online Computing Reviews Service

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
W4A '09: Proceedings of the 2009 International Cross-Disciplinary Conference on Web Accessibililty (W4A)
April 2009
189 pages
ISBN:9781605585611
DOI:10.1145/1535654
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

  • The Mozilla Foundation
  • Google Inc.
  • Microsoft: Microsoft
  • Zakon Group

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 April 2009

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. automatic evaluation
  2. guidelines
  3. web accessibility

Qualifiers

  • Research-article

Conference

W4A '09
Sponsor:
  • Microsoft

Acceptance Rates

Overall Acceptance Rate 171 of 371 submissions, 46%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)5
  • Downloads (Last 6 weeks)1
Reflects downloads up to 30 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2023)A platform to check website compliance with web accessibility standardsProceedings of the 20th International Web for All Conference10.1145/3587281.3587289(75-78)Online publication date: 30-Apr-2023
  • (2020)Providing Continuous Web Accessibility EvaluationInteractivity and the Future of the Human-Computer Interface10.4018/978-1-7998-2637-8.ch001(1-26)Online publication date: 2020
  • (2017)Ontology Design for Automatic Evaluation of Web User Interface Usability2017 Portland International Conference on Management of Engineering and Technology (PICMET)10.23919/PICMET.2017.8125425(1-8)Online publication date: Jul-2017
  • (2016)A Study on Immediate Automatic Usability Evaluation of Web Application User InterfacesDatabases and Information Systems10.1007/978-3-319-40180-5_18(257-271)Online publication date: 21-Jun-2016
  • (2015)Investigating accessibility on web-based mapsACM SIGAPP Applied Computing Review10.1145/2815169.281517115:2(17-26)Online publication date: 14-Aug-2015
  • (2015)Evaluation of web accessibility on the maps domainProceedings of the 30th Annual ACM Symposium on Applied Computing10.1145/2695664.2695771(157-162)Online publication date: 13-Apr-2015
  • (2013)Benchmarking web accessibility evaluation toolsProceedings of the 10th International Cross-Disciplinary Conference on Web Accessibility10.1145/2461121.2461124(1-10)Online publication date: 13-May-2013
  • (2012)Web accessibility guidelinesWorld Wide Web10.1007/s11280-011-0130-815:1(61-88)Online publication date: 1-Jan-2012
  • (2010)Extraction of Web Site Evaluation Criteria and Automatic EvaluationJournal of Advanced Computational Intelligence and Intelligent Informatics10.20965/jaciii.2010.p039614:4(396-401)Online publication date: 20-May-2010
  • (2010)Web Site Evaluation Criteria Extraction and Evaluation AutomationIEEJ Transactions on Electronics, Information and Systems10.1541/ieejeiss.130.66130:1(66-74)Online publication date: 2010

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media