skip to main content
10.1145/1296843.1296855acmconferencesArticle/Chapter ViewAbstractPublication PagesassetsConference Proceedingsconference-collections
Article

Effects of sampling methods on web accessibility evaluations

Published: 15 October 2007 Publication History

Abstract

Except for trivial cases, any accessibility evaluation has to be based on some method for selecting pages to be analyzed. But this selection process may bias the evaluation. Up to know, not much is known about available selection methods, and about their effectiveness and efficiency.
The paper addresses the following open issues: how to define the quality of the selection process, which processes are better than others, how to measure their difference in quality, which factors may affect quality (type of assessment, size of the page pool, structural features of the web site).
These issues are investigated through an experimental evaluation of thirteen sampling methods applied to 32000 web pages. While some of the conclusions are not surprising (for example, that sample size affect accuracy), others were not expected at all (that minimal sampling size obtains a high accuracy level under certain circumstances).

References

[1]
J. Abascal, M. Arrue, I. Fajardo, N. Garay, and
[2]
M. Arrue, M. Vigo, and J. Abascal. Quantitative metrics for web accessibility evaluation. In Proceedings of the ICWE 2005 Workshop on Web Metrics and Measurement, 2005.
[3]
G. Brajnik. Comparing accessibility evaluation tools: a method for tool effectiveness. Universal Access in the Information Society, 3(3-4):252--263, Oct 2004. www.springerlink.com/openurl.asp?genre= article&id=
[4]
G. Brajnik. Ranking websites through prioritized we accessibility barriers. In Technology and Persons with Disabilities Conference, Los Angeles, March 2007. CSUN, California State University Northridge. www.dimi.uniud.it/giorgio/papers/csun07.pdf.
[5]
G. Brajnik and R. Lomuscio. Samba: a semi-automatic method for measuring barriers of accessibility. In S. Trewin, editor, 9th Int. ACM SIGACCESS Conference on Computers and Accessibility, ASSETS, Tempe, AZ, Oct 2007. ACM Press.
[6]
M. Dunham. Data Mining: Introductory and Advanced Topics. Prentice Hall, 2003.
[7]
EIAO. European Internet Accessibility Observatory. www.eiao.net.
[8]
Google. Google accessible search. labs.google.com/accessible, 2007. Visited: May 2007.
[9]
M. R. Henzinger, A. Heydon, M. Mitzenmacher, and M. Najork. On near-uniform URL sampling. In Proceedings of the 9th international World Wide Web conference on Computer Networks: the International Journal of Computer and Telecommunications Networking, pages 295--308, Amsterdam, The Netherlands, 2000. North-Holland Publishing Co.
[10]
Italian Government. Requisiti tecnici e i diversi livelli per l'accessibilitàgli strumenti informatici. a www.pubbliaccesso.it/normative/DM080705.htm, July 2005. G. U. n. 183 8/8/2005.
[11]
M. King, J. Thatcher, P. Bronstad, and R. Easton. Managing usability for people with disabilities in a large web presence. IBM Systems Journal, 44(3):519--535, 2005.
[12]
A. Mulas and C. Pitton. Analisi sperimentale e proposta di nuove tecniche di campionamento per la valutazione manuale dell'accessibilitàdi siti web. a Master's thesis, University of Udine, Udine, Italy, April 2007.
[13]
N. Ulltveit-Moe, M. Snaprud, A. Nietzio, M. G. Olsen, and C. Thomsen. Early Results from Automa Accessibility Benchmarking of Public European Web Sites from the EIAO. eiao.net/publications. Visited: May 2007.
[14]
E. Velleman, C. A. Velasco, M. Snaprud, and D. Burger. D-WAB4 Unified Web Evaluation Methodology (UWEM 1.0). Technical report, WAB Cluster, 2006.
[15]
M. Vigo, M. Arrue, G. Brajnik, R. Lomuscio, and J. Abascal. Quantitative metrics for measuring web accessibility. In W4A '07: Proceedings of the 2007 International Cross-Disciplinary Workshop on Web Accessibility (W4A), pages 99--107, New York, NY, USA, 2007. ACM Press. www.dimi.uniud.it/giorgio/papers/w4a07.pdf.
[16]
W3C/WAI. Conformance evaluation of web sites for accessibility: Determine the scope of the evaluation. www.w3.org/WAI/eval/conformance.html#scope. Visited: May 2007.

Cited By

View all
  • (2024)AI-Enhanced Web Form Development: Tackling Accessibility Barriers with Generative Technologies2024 IEEE International Conference on Systems, Man, and Cybernetics (SMC)10.1109/SMC54092.2024.10831040(1195-1198)Online publication date: 6-Oct-2024
  • (2023)Web accessibility automatic evaluation tools: to what extent can they be automated?CCF Transactions on Pervasive Computing and Interaction10.1007/s42486-023-00127-85:3(288-320)Online publication date: 14-Mar-2023
  • (2022)A Systematic Review of Web Accessibility MetricsApp and Website Accessibility Developments and Compliance Strategies10.4018/978-1-7998-7848-3.ch004(77-108)Online publication date: 2022
  • Show More Cited By

Index Terms

  1. Effects of sampling methods on web accessibility evaluations

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    Assets '07: Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibility
    October 2007
    282 pages
    ISBN:9781595935731
    DOI:10.1145/1296843
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 15 October 2007

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. accessibility metric
    2. sampling methods
    3. web accessibility

    Qualifiers

    • Article

    Conference

    ASSETS07
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 436 of 1,556 submissions, 28%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)13
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 27 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)AI-Enhanced Web Form Development: Tackling Accessibility Barriers with Generative Technologies2024 IEEE International Conference on Systems, Man, and Cybernetics (SMC)10.1109/SMC54092.2024.10831040(1195-1198)Online publication date: 6-Oct-2024
    • (2023)Web accessibility automatic evaluation tools: to what extent can they be automated?CCF Transactions on Pervasive Computing and Interaction10.1007/s42486-023-00127-85:3(288-320)Online publication date: 14-Mar-2023
    • (2022)A Systematic Review of Web Accessibility MetricsApp and Website Accessibility Developments and Compliance Strategies10.4018/978-1-7998-7848-3.ch004(77-108)Online publication date: 2022
    • (2021)Empirical web accessibility evaluation for blind web usersACM SIGACCESS Accessibility and Computing10.1145/3458055.3458057(1-5)Online publication date: 21-Mar-2021
    • (2020)Comparing accessibility evaluation plug-insProceedings of the 17th International Web for All Conference10.1145/3371300.3383346(1-11)Online publication date: 20-Apr-2020
    • (2018)A Holistic Decision Support Environment for Web AccessibilityComputers Helping People with Special Needs10.1007/978-3-319-94277-3_1(3-7)Online publication date: 26-Jun-2018
    • (2017)Active learning for Web accessibility evaluationProceedings of the 14th International Web for All Conference10.1145/3058555.3058559(1-9)Online publication date: 2-Apr-2017
    • (2017)An optimal sampling method for web accessibility quantitative metric and its online extensionInternet Research10.1108/IntR-07-2016-020527:5(1190-1208)Online publication date: 2-Oct-2017
    • (2016)Accessibility of the Italian institutional web pagesUniversal Access in the Information Society10.1007/s10209-014-0381-015:2(305-312)Online publication date: 1-Jun-2016
    • (2016)Web Page Clustering for More Efficient Website Accessibility EvaluationsComputers Helping People with Special Needs10.1007/978-3-319-41264-1_35(259-266)Online publication date: 6-Jul-2016
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media