skip to main content
10.1145/1168149.1168162acmotherconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
Article

Heuristics for information visualization evaluation

Published: 23 May 2006 Publication History

Abstract

Heuristic evaluation is a well known discount evaluation technique in human-computer interaction (HCI) but has not been utilized in information visualization (InfoVis) to the same extent. While several sets of heuristics have been used or proposed for InfoVis, it is not yet known what kind of heuristics are useful for finding general InfoVis problems. We performed a meta-analysis with the goal of exploring the issues of heuristic evaluation for InfoVis. This meta-analysis concentrates on issues pertaining to the selection and organization of heuristics, and the process itself. For this purpose, we used three sets of previously published heuristics to assess a visual decision support system that is used to examine simulation data. The meta-analysis shows that the evaluation process and results have a high dependency on the heuristics and the types of evaluators chosen. We describe issues related to interpretation, redundancy, and conflict in heuristics. We also provide a discussion of generalizability and categorization of these heuristics.

References

[1]
R. Amar and J. Stasko. A Knowledge Task-Based Framework for Design and Evaluation of Information Visualizations. In Proc. of IEEE Info Vis, pages 143--149, Los Alamitos, USA, 2004. IEEE Press.
[2]
M. Q. W. Baldonado, A. Woodruff, and A. Kuchinsky. Guidelines for Using Multiple Views in Information Visualization. In Proceedings of AVI '00, pages 110--119, New York, USA, 2000. ACM Press.
[3]
J. Bertin. Semiology of Graphics. University of Wisconsin Press, 1983. Translation: William J. Berg.
[4]
N. Bevan, C. Barnum, G. Cockton, J. Nielsen, J. Spool, and D. Wixon. The "Magic Number 5": is it enough for Web Testing? In CHI Extended Abstracts, pages 698--699, New York, USA, 2003. ACM Press.
[5]
S. K. Card, J. D. Mackinlay, and B. Shneiderman. Readings in Information Visualization: Using Vision to Think. Morgan Kaufmann, San Francisco, CA, USA, 1999.
[6]
B. Craft and P. Cairns. Beyond Guidelines: What can we learn from the Information Seeking Mantra? In Proceedings of IV, pages 110--118, Los Alamitos, CA, USA, 2005. IEEE Press.
[7]
R. Dougherty and A. Wade. Vischeck. http://www.vischeck.com/vischeck/vischeckImage.php, 2006. visited: March, 2006.
[8]
A. Fall and J. Fall. SELES: A Spatially Explicit Landscape Event Simulator. In Proc. of GIS and Environmental Modeling, pages 104--112. National Center for Geographic Information and Analysis, 1996.
[9]
A. Fall and J. Fall. A Domain-Specific Language For Models of Landscape Dynamics. Ecological Modelling, 137(1):1--21, February 2001.
[10]
E. Gamma, R. Helma, R. Johnson, and J. Vlissides, editors. Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley, 1994.
[11]
C. G. Healey. On the Use of Perceptual Cues and Data Mining for Effective Visualization of Scientific Datasets. In Proc. of GI, pages 177--184, 1998.
[12]
M. J. Kahn and A. Prail. Formal usability inspections. In Usability Inspection Methods, pages 141--171. John Wiley & Sons, Inc., 1994.
[13]
J. Mackinlay. Automating the design of graphical presentations of relational information. ACM Transactions on Graphics (TOG), 5(2):110--141, 1986.
[14]
J. Mankoff, A. K. Dey, G. Hsieh, J. Kientz, S. Lederer, and M. Ames. Heuristic Evaluation of Ambient Displays. In Proc. of CHI '03, pages 169--176, New York, NY, USA, 2003. ACM Press.
[15]
E. Morse, M. Lewis, and K. A. Olsen. Evaluating Visualizations: Using a Taxonomic Guide. Int. J. Human-Computer Studies, 53:637--662, 2000.
[16]
J. Nielsen. Heuristic Evaluation. In J. Nielsen and R. L. Mack, editors, Usability Inspection Methods, pages 25--62. John Wiley & Sons, Inc., 1994.
[17]
J. Nielsen and R. L. Mack. Usability Inspection Methods. John Wiley & Sons, Inc., 1994.
[18]
L. Schlesier, J. Hughes, A. Fall, and M. S. T. Carpendale. The LuMPB Key: A Multiple View Interface to Explore High Dimensional Mountain Pine Beetle Simulation Data. In Proc. of CMV, pages 31--41. IEEE Computer Society, 2006.
[19]
B. Shneiderman. Designing the User Interface: Strategies for Effective Human-Computer Interaction. Addison-Wesley, Reading, MA, 1987.
[20]
B. Shneiderman. The Eyes Have It: A Task by Data Type Taxonomy for Information Visualizations. In Proc. of the IEEE Symposium on Visual Languages, pages 336--343. IEEE Press, 1996.
[21]
J. Spool and W. Schroeder. Testing Web Sites: Five Users is Nowhere Near Enough. In CHI '01 Extended Abstracts, pages 285--286, New York, NY, USA, 2001. ACM Press.
[22]
B. Tognazzini. First Principles of Interaction Design. http://www.asktog.com/basics/firstPrinciples.html, 2006. visited: March, 2006.
[23]
M. Tory and T. Möller. Evaluating Visualizations: Do Expert Reviews Work. IEEE Computer Graphics and Applications, 25(5):8--11, September/October 2005.
[24]
E. R. Tufte. The Visual Display of Quantitative Information. Graphics Press, Cheshire, CT, 2001.
[25]
C. Ware. Information Visualization: Perception for Design. Morgan Kaufmann, 2nd edition, 2004.
[26]
T. Zuk and M. S. T. Carpendale. Theoretical Analysis of Uncertainty Visualizations. In Proc. SPIE & IS&T Conf. Electronic Imaging, Vol. 6060: Visualization and Data Analysis 2006, 2006. 606007.

Cited By

View all
  • (2024)From Instruction to Insight: Exploring the Functional and Semantic Roles of Text in Interactive DashboardsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.345660131:1(382-392)Online publication date: 10-Sep-2024
  • (2024)Heuristics for Supporting Cooperative Dashboard DesignIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332715830:1(370-380)Online publication date: 1-Jan-2024
  • (2024)A Heuristic Approach for Dual Expert/End-User Evaluation of Guidance in Visual AnalyticsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332715230:1(997-1007)Online publication date: 1-Jan-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
BELIV '06: Proceedings of the 2006 AVI workshop on BEyond time and errors: novel evaluation methods for information visualization
May 2006
89 pages
ISBN:1595935622
DOI:10.1145/1168149
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 May 2006

Permissions

Request permissions for this article.

Check for updates

Qualifiers

  • Article

Conference

AVI06

Acceptance Rates

Overall Acceptance Rate 45 of 64 submissions, 70%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)121
  • Downloads (Last 6 weeks)12
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)From Instruction to Insight: Exploring the Functional and Semantic Roles of Text in Interactive DashboardsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.345660131:1(382-392)Online publication date: 10-Sep-2024
  • (2024)Heuristics for Supporting Cooperative Dashboard DesignIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332715830:1(370-380)Online publication date: 1-Jan-2024
  • (2024)A Heuristic Approach for Dual Expert/End-User Evaluation of Guidance in Visual AnalyticsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332715230:1(997-1007)Online publication date: 1-Jan-2024
  • (2024)Metrics-Based Evaluation and Comparison of Visualization NotationsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332690730:1(425-435)Online publication date: 1-Jan-2024
  • (2023)Strategies for evaluating visual analytics systems: A systematic review and new perspectivesInformation Visualization10.1177/1473871623121256823:1(84-101)Online publication date: 28-Dec-2023
  • (2023)VisLab: Enabling Visualization Designers to Gather Empirically Informed Design FeedbackProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581132(1-18)Online publication date: 19-Apr-2023
  • (2023)Reflections on Designing and Running Visualization Design and Programming Activities in Courses with Many Students2023 IEEE VIS Workshop on Visualization Education, Literacy, and Activities (EduVis)10.1109/EduVis60792.2023.00015(55-64)Online publication date: 22-Oct-2023
  • (2023)Evaluating narrative visualization: a survey of practitionersInternational Journal of Data Science and Analytics10.1007/s41060-023-00394-918:1(19-34)Online publication date: 31-Mar-2023
  • (2023)User Study on a Multi-view Environment to Identify Differences Between Biological TaxonomiesHCI in Business, Government and Organizations10.1007/978-3-031-36049-7_23(300-319)Online publication date: 17-Jul-2023
  • (2022)A Systematic Review on Visualizations for Self-Generated Health Data for Daily ActivitiesInternational Journal of Environmental Research and Public Health10.3390/ijerph19181116619:18(11166)Online publication date: 6-Sep-2022
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media