Van Du Nguyen
ABSTRACT
Recently, research on the Wisdom of Crowd (WoC) has been widely expanded by supporting interval values as an additional representation of underlying predictions. Accordingly, instead of giving single values, ones can express their predictions on a given cognition problem in the form of interval values1. For such a representation, many methods have been proposed for aggregating underlying predictions based on their midpoints. In this case, of course, the outputs of the proposed methods are single values. In some situations, however, the aggregated prediction in the form of interval value can be better representation of underlying predictions. In the current study, we present a comparison of the use of different approaches for aggregating individual predictions including Interval Aggregation and MidPoint Aggregation. Experimental studies have been conducted to determine how do different aggregation methods influence the quality of the obtained collective prediction.
- O. Arazy, N. Halfon, and D. Malkinson. 2015. Forecasting Rain Events -Meteorological Models or Collective Intelligence? In Proc. of EGU General Assembly Conference 17 (Vienna, Austria, 2015), 15611--15614Google Scholar
- J.S. Armstrong. 2001. Combining forecasts. Principles of forecasting. Springer (2001), 417--439Google Scholar
- J. A. Baars, and C. F. Mass. 2005. Performance of National Weather Service forecasts compared to operational, consensus, and weighted model output statistics. Weather and Forecasting 20 (2005), 1034--1047Google ScholarCross Ref
- Clemen, R. T. 1989. Combining forecasts: A review and annotated bibliography. International Journal of Forecasting 5 (1989), 559--583Google ScholarCross Ref
- F. Galton. 1907. Vox populi (The wisdom of crowds). Nature 75 (1907), 450--451Google ScholarCross Ref
- M. Glaser, T. Langer, and M. Weber. 2013. True Overconfidence in Interval Estimates: Evidence Based on a New Measure of Miscalibration. Journal of Behavioral Decision Making 26 (2013), 405--417Google ScholarCross Ref
- R. H. Kurvers, S. M. Herzog, R. Hertwig, J. Krause, P. A. Carney, A. Bogart, G. Argenziano, I. Zalaudek, and M. Wolf. 2016. Boosting medical diagnostics by pooling independent judgments. Proceedings of the National Academy of Sciences (2016), 8777--8782Google Scholar
- R. H. Kurvers, J. Krause, G. Argenziano, I. Zalaudek, and M. Wolf. 2015. Detection accuracy of collective intelligence assessments for skin cancer diagnosis. JAMA dermatology 151 (2015), 1346--1353Google Scholar
- M. Lang, N. Bharadwaj, and C. A. Di Benedetto. 2016. How crowdsourcing improves prediction of market-oriented outcomes. Journal of Business Research 69 (2016), 4168--4176Google ScholarCross Ref
- A. Lyon, B. C. Wintle, and M. Burgman. 2015. Collective wisdom: Methods of confidence interval aggregation. Journal of Business Research 68 (2015), 1759--1767Google ScholarCross Ref
- N. T. Nguyen (ed.). 2008. Advanced Methods for Inconsistent Knowledge Management. Springer-Verlag, London Google ScholarDigital Library
- N. T. Nguyen. 2001. Consensus-based Timestamps in Distributed Temporal Databases. The Computer Journal 44 (2001), 398--409Google ScholarCross Ref
- N. T. Nguyen. 2005. Processing Inconsistency of Knowledge on Semantic Level. Journal of Universal Computer Science 11(2005), 285--302Google Scholar
- M. Nofer, and O. Hinz. 2014. Are crowds on the internet wiser than experts? The case of a stock prediction community. Journal of Business Economics 84 (2014), 303--338Google ScholarCross Ref
- S. Park, and D. V. Budescu. 2015. Aggregating multiple probability intervals to improve calibration. Judgment and Decision Making 10 (2015), 130--143Google Scholar
- J. B. Soll, and J. Klayman. 2004. Overconfidence in interval estimates. Journal of Experimental Psychology: Learning, Memory, and Cognition 30 (2004), 299--314Google ScholarCross Ref
- A. Speirs-Bridge, F. Fidler, M. McBride, L. Flander, G. Cumming, and M. Burgman. 2010. Reducing Overconfidence in the Interval Judgments of Experts. Risk Analysis 30 (2010), 512--523Google ScholarCross Ref
- J. Surowiecki (ed.). 2005. The wisdom of crowds. Doubleday/Anchor, New York Google ScholarDigital Library
- M., Zgrzywa. 2007. Consensus Determining with Dependencies of Attributes with Interval Values. Journal of Universal Computer Science 13 (2007), 329--344Google Scholar
Index Terms
- A Comparative Study of Methods for Collective Prediction Determination Using Interval Estimates
Recommendations
Towards Analyzing the Impact of Diversity and Cardinality on the Quality of Collective Prediction Using Interval Estimates
Advances and Trends in Artificial Intelligence. From Theory to PracticeAbstractRecently, many research results have indicated that diversity is the most important characteristic of crowd-based applications. However, it is a point estimates-based finding in which single values are used as the representation of individual ...
"Potential Interval of Root" of Nonlinear Equation: Labeling Algorithm
MIWAI 2014: Proceedings of the 8th International Workshop on Multi-disciplinary Trends in Artificial Intelligence - Volume 8875Novel Sequence Generating Algorithm (SGA) and Potential Interval Qualifier Algorithm (PIQA) are designed to classify potential interval estimates of a given nonlinear equation into intervals possessing roots and intervals containing extrema. Using ...
The Wisdom of Multiple Guesses
EC '15: Proceedings of the Sixteenth ACM Conference on Economics and ComputationThe "wisdom of crowds" dictates that aggregate predictions from a large crowd can be surprisingly accurate, rivaling predictions by experts. Crowds, meanwhile, are highly heterogeneous in their expertise. In this work, we study how the heterogeneous ...
Comments