Alexandra Fernandes
ABSTRACT
In this paper we describe a pilot study where we explore the potential of self-rated confidence judgments in the context of control room human performance evaluation. Response time, accuracy, and confidence data were obtained in a simplified task in a simulator environment of a nuclear control room. Licensed operators were presented with blocks of questions regarding specific images portraying recognizable displays with relevant process information -- after answering the primary task they were requested to evaluate their confidence in the response. The images included either benchmark (e.g. numerical values) or innovative (e.g. numerical values plus graphs and figures) design features. Overall results show that: a) there is not a complete correspondence between response accuracy and confidence ratings; b) participants took longer to reply to the confidence ratings when answers to the primary task were incorrect; c) level of confidence seems to be discriminative, with the innovative displays showcasing higher levels of certainty.
- Donald A. Curtis, Samuel L. Lind, Christy K. Boscardin, Mark Dellinges. 2013. Does student confidence on multiple-choice question assessments provide useful information? Medical Education, 47(6), 578--584. http://doi.org/10.1111/medu.12147Google Scholar
Cross Ref
- Hildebrantdt, M. & Eitrheim, M.H.R. (2015). A micro-task method for assessing performance effects of innovative interface elements. Proceedings of the 59th Human Factors and Ergonomics Society, Los Angeles, 26-30 October, 2015, pp. 1759--1763.Google Scholar
- Hildebrantdt, M. & Fernandes, A. (accepted). Micro Task Evaluation of Analog vs. Digital NPP Control Room Interfaces. Proceedings of the Human Factors and Ergonomics Society International Conference, Washington DC, 19-23 September, 2016.Google Scholar
- Joshua Klayman, Jack, B. Soll, Claudia Gonzalez-Vallejo, Sema Barlas. 1999. Overconfidence: It depends on How, What, and Whom You Ask. Organizational Behavior and Human Decision Processes, 79(3), 216--247.Google ScholarCross Ref
- Roger Ratcliff, Jeffrey J. Starns. 2013. Modeling confidence judgments, response times, and multiple choices in decision making: Recognition memory and motion discrimination. Psychological Review, 120(3), 697--719. http://doi.org/10.1037/a0033152Google ScholarCross Ref
- Mark T. Spence, Merrie Brucks. 1997. The Moderating Effects of Problem Characteristics on Experts' and Novices' Judgments. Journal of Marketing Research, 34(2), 233. http://doi.org/10.2307/3151861Google Scholar
- Thomas A. Ulrich, Steffen Werner, Ronald L. Boring. 2015. Studying Situation Awareness on a Shoestring Budget: An Example of an Inexpensive Simulation Environment for Theoretical Research. In Proceedings of the Human Factors and Ergonomics Society 59th Annual Meeting, 1--6. http://doi.org/10.1177/1541931215591329Google ScholarCross Ref
- Nick Yeung, Christopher Summerfield. 2012. Metacognition in human decision-making: confidence and error monitoring. Philosophical Transactions of the Royal Society B: Biological Sciences, 367(1594), 1310--1321. http://doi.org/10.1098/rstb.2011.0416Google ScholarCross Ref
Recommendations
Measuring (and enhancing?) student confidence with confidence scores
FIE '00: Proceedings of the 30th Annual Frontiers in Education - Volume 01The important skill of building confidence in one's analysis through sanity and cross-checking is often poorly acquired by engineering students. An introductory circuit analysis class presents an ideal opportunity in which to emphasize and measure this ...
Development of a team workload assessment technique for the main control room of advanced nuclear power plants
For al main control room (MCR) in advanced nuclear power plants (ANPPs), a team of three operators has been adopted to enhance the safety of all the operating conditions, including general, abnormal, and emergent. Therefore, to evaluate the workload of ...
Unraveling Confidence: Examining Confidence Scores as Proxy for OCR Quality
Document Analysis and Recognition - ICDAR 2023AbstractWhile performing Optical Character Recognition (OCR), most engines provide confidence scores. These scores give an indication on how certain an engine is that a word or character is correctly determined. The practical application of this score is ...
Comments