Uncertainty Visualization Influences how Humans Aggregate Discrepant Information

ABSTRACT

The number of sensors in our surroundings that provide the same information steadily increases. Since sensing is prone to errors, sensors may disagree. For example, a GPS-based tracker on the phone and a sensor on the bike wheel may provide discrepant estimates on traveled distance. This poses a user dilemma, namely how to reconcile the conflicting information into one estimate. We investigated whether visualizing the uncertainty associated with sensor measurements improves the quality of users' inference. We tested four visualizations with increasingly detailed representation of uncertainty. Our study repeatedly presented two sensor measurements with varying degrees of inconsistency to participants who indicated their best guess of the "true" value. We found that uncertainty information improves users' estimates, especially if sensors differ largely in their associated variability. Improvements were larger for information-rich visualizations. Based on our findings, we provide an interactive tool to select the optimal visualization for displaying conflicting information.

References

1. Anthony D. Andre and Henry A. Cutler. 1998. Displaying Uncertainty in Advanced Navigation Systems. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 42, 1 (1998), 31--35.
2. Peter W. Battaglia, Robert A. Jacobs, and Richard N. Aslin. 2003. Bayesian integration of visual and auditory signals for spatial localization. Journal of the Optical Society of America A 20, 7 (Jul 2003), 1391--1397.
3. Ken Brodlie, Rodolfo Allendes Osorio, and Adriano Lopes. 2012. A Review of Uncertainty in Data Visualization. Expanding the Frontiers of Visual Analytics and Visualization (2012), 81--109.
4. David V. Budescu. 2006. Confidence in aggregation of opinions from multiple sources. In Information Sampling and Adaptive Cognition. 327--352.
5. David V. Budescu, Stephen Broomell, and Han-Hui Por. 2009. Improving communication of uncertainty in the reports of the intergovernmental panel on climate change. Psychological science 20, 3 (2009), 299--308.
6. Michael Correll and Michael Gleicher. 2014. Error bars considered harmful: Exploring alternate encodings for mean and error. IEEE Transactions on Visualization and Computer Graphics 20, 12 (2014), 2142--2151.
7. Helen Couclelis. 2003. The Certainty of Uncertainty: GIS and the Limits of Geographic Knowledge. Transactions in GIS 7, 2 (2003), 165--175.
8. Marc O. Ernst. 2010. Decisions Made Better. Science 329, 5995 (2010), 1022--1023.
9. Marc O. Ernst and Heinrich H. Bülthoff. 2004. Merging the senses into a robust percept. Trends in Cognitive Sciences 8, 4 (2004), 162 -- 169.
10. Nivan Ferreira, Danyel Fisher, and Arnd C. Konig. 2014. Sample-oriented Task-driven Visualizations: Allowing Users to Make Better, More Confident Decisions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). 571--580.
11. Nahum Gershon. 1998. Visualization of an imperfect world. IEEE Computer Graphics and Applications 18, 4 (1998), 43--45.
12. Gerd Gigerenzer, Ralph Hertwig, Eva Van Den Broek, Barbara Fasolo, and Konstantinos V. Katsikopoulos. 2005. "A 30% chance of rain tomorrow": How does the public understand probabilistic weather forecasts? Risk Analysis 25, 3 (jun 2005), 623--629.
13. Miriam Greis, Passant El. Agroudy, Hendrik Schuff, Tonja Machulla, and Albrecht Schmidt. 2016. Decision-Making Under Uncertainty: How the Amount of Presented Uncertainty Influences User Behavior. In Proceedings of the 9th Nordic Conference on Human-Computer Interaction (NordiCHI '16). 52:1--52:4.
14. Miriam Greis, Emre Avci, Albrecht Schmidt, and Tonja Machulla. 2017a. Increasing Users' Confidence in Uncertain Data by Aggregating Data from Multiple Sources. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, 828--840.
15. Miriam Greis, Jessica Hullman, Michael Correll, Matthew Kay, and Orit Shaer. 2017b. Designing for Uncertainty in HCI: When Does Uncertainty Help?. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '17). 593--600.
16. Miriam Greis, Thorsten Ohler, Niels Henze, and Albrecht Schmidt. 2015. Investigating Representation Alternatives for Communicating Uncertainty to Non-experts. In Human-Computer Interaction - INTERACT 2015. Vol. 9299. 256--263.
17. Miriam Greis, Hendrik Schuff, Marius Kleiner, Niels Henze, and Albrecht Schmidt. 2017. Input Controls for Entering Uncertain Data. Proceedings of the ACM on Human-Computer Interaction 1, 1 (jun 2017), 1--17.
18. Harald Ibrekk and M. Granger Morgan. 1987. Graphical Communication of Uncertain Quantities to Nontechnical People. Risk Analysis 7, 4 (1987), 519--529.
19. Robert A. Jacobs. 1999. Optimal integration of texture and motion cues to depth. Vision Research 39, 21 (1999), 3621 -- 3629.
20. Susan L. Joslyn and Jared E. LeClerc. 2012. Uncertainty forecasts improve weather-related decisions and attenuate the effects of forecast error. Journal of experimental psychology. Applied 18, 1 (mar 2012), 126--40.
21. Malte F. Jung, David Sirkin, Turgut M. Gür, and Martin Steinert. 2015. Displayed Uncertainty Improves Driving Experience and Behavior: The Case of Range Anxiety in an Electric Car. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '15). 2201--2210.
22. Matthew Kay, Tara Kola, Jessica R. Hullman, and Sean A. Munson. 2016. When (ish) is My Bus?: User-centered Visualizations of Uncertainty in Everyday, Mobile Predictive Systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '16). 5092--5103.
23. Matthew Kay, Dan Morris, Mc Schraefel, and Julie A. Kientz. 2013. There's No Such Thing as Gaining a Pound: Reconsidering the Bathroom Scale User Interface. In Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '13). 401--410.
24. Isaac M. Lipkus, Greg Samsa, and Barbara K. Rimer. 2001. General performance on a numeracy scale among highly educated samples. Medical decision making : an international journal of the Society for Medical Decision Making 21, 1 (2001), 37--44.
25. Alan M. MacEachren, Anthony Robinson, and Susan Hopper. 2005. Visualizing geospatial information uncertainty: What we know and what we need to know. Cartography and Geographic Information Science 32, 3 (jan 2005), 139--160.
26. Rebecca E. Morss, Julie L. Demuth, and Jeffrey K. Lazo. 2008. Communicating Uncertainty in Weather Forecasts: A Survey of the U.S. Public. Weather and Forecasting 23, 5 (oct 2008), 974--991.
27. Alex T. Pang, Craig M. Wittenbrink, and Suresh K. Lodha. 1997. Approaches to uncertainty visualization. The Visual Computer 13, 8 (1997), 370--390.
28. Kristin Potter, Joe Kniss, Richard Riesenfeld, and Chris R. Johnson. 2010. Visualizing Summary Statistics and Uncertainty. Computer Graphics Forum 29, 3 (June 2010), 823--832.
29. Kristin Potter, Paul Rosen, and Chris R. Johnson. 2012. From quantification to visualization: A taxonomy of uncertainty visualization approaches. In IFIP Advances in Information and Communication Technology, Vol. 377 AICT. 226--247.
30. Maria H. Ramos, Schalk J. Van Andel, and Florian Pappenberger. 2013. Do probabilistic forecasts lead to better decisions? Hydrology and Earth System Sciences 17, 6 (June 2013), 2219--2232.
31. Gordan Ristovski, Tobias Preusser, Horst K. Hahn, and Lars Linsen. 2014. Uncertainty in medical visualization: Towards a taxonomy. Computers and Graphics (Pergamon) 39, 1 (2014), 60--73.
32. Mark S. Roulston, Gary E. Bolton, Andrew N. Kleit, and Addison L. Sears-Collins. 2006. A Laboratory Study of the Benefits of Including Uncertainty Information in Weather Forecasts. Weather and Forecasting 21, 1 (2006), 116--122.
33. Mark S. Roulston and Todd R. Kaplan. 2009. A laboratory-based study of understanding of uncertainty in 5-day site-specific temperature forecasts. Meteorological Applications 16, 2 (2009), 237--244.
34. Enrico Rukzio, John Hamard, Chie Noda, and Alexander De Luca. 2006. Visualization of Uncertainty in Context Aware Mobile Applications. In Proceedings of the 8th Conference on Human-computer Interaction with Mobile Devices and Services (MobileHCI '06). 247--250.
35. Orit Shaer, Oded Nov, Lauren Westendorf, and Madeleine Ball. 2017. Communicating Personal Genomic Information to Non-experts: A New Frontier for Human-Computer Interaction. Foundations and Trends in Human-Computer Interaction 11, 1 (2017), 1--62.
36. Meredith Skeels, Bongshin Lee, Greg Smith, and George Robertson. 2008. Revealing uncertainty for information visualization. In Proceedings of the working conference on Advanced visual interfaces - AVI '08, Vol. 9. 376.
37. Susanne Tak and Alexander Toet. 2014. Color and Uncertainty: It is not always Black and White. Eurographics Conference on Visualization (2014), 55--59.
38. J. Taylor. 1997. Introduction to Error Analysis, the Study of Uncertainties in Physical Measurements, 2nd Edition. University Science Books.
39. Judi Thomson, Elizabeth Hetzler, Alan MacEachren, Mark Gahegan, and Misha Pavel. 2005. A typology for visualizing uncertainty. In IS&T/SPIE Electronic Imaging, Vol. 5669. 146.
40. Amos Tversky and Daniel Kahneman. 1975. Judgment under Uncertainty: Heuristics and Biases. In Utility, Probability, and Human Decision Making. Springer Netherlands, Dordrecht, 141--162.
41. Thomas S. Wallsten, David V. Budescu, Amnon Rapoport, Rami Zwick, and Barbara Forsyth. 1986. Measuring the Vague Meanings of Probability Terms. Journal of Experimental Psychology: General 115, 4 (1986), 348--365.
42. Jacob O. Wobbrock, Leah Findlater, Darren Gergle, and James J. Higgins. 2011. The Aligned Rank Transform for Nonparametric Factorial Analyses Using Only Anova Procedures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). 143--146.
43. Torre Zuk and Sheelagh Carpendale. 2006. Theoretical analysis of uncertainty visualizations. Electronic Imaging 2006 6060, March (jan 2006), 1--14.

Index Terms

• Human-centered computing