ABSTRACT
In this paper we describe the usefulness of statistical validation techniques for human factors survey research. We need to investigate a diversity of validity aspects when creating metrics in human factors research, and we argue that the statistical tests used in other fields to get support for reliability and construct validity in surveys, should also be applied to human factors research in software engineering more often. We also show briefly how such methods can be applied (Test-Retest, Cronbach's α, and Exploratory Factor Analysis).
- D. J. Bartholomew. Stochastic models for social processes. Wiley, London, 1967.Google Scholar
- A. Bosu, J. Carver, R. Guadagno, B. Bassett, D. McCallum, and L. Hochstein. Peer impressions in open source organizations: A survey. Journal of Systems and Software, 94:4--15, 2014.Google ScholarCross Ref
- G. E. P. Box and N. R. Draper. Empirical Model-building and Response Surface. John Wiley & Sons, Inc., New York, NY, USA, 1986. Google ScholarDigital Library
- J. Chen, J. Xiao, Q. Wang, L. J. Osterweil, and M. Li. Perspectives on refactoring planning and practice: an empirical study. Empirical Software Engineering, pages 1--40, 2015. Google ScholarDigital Library
- L. Cronbach. Coefficient alpha and the internal structure of tests. Psychometrika, 16(3):297--334, 1951.Google ScholarCross Ref
- L. Fabrigar and D. Wegener. Exploratory Factor Analysis. Series in understanding statistics. OUP USA, 2012.Google Scholar
- D. M. Fernández and S. Wagner. Naming the pain in requirements engineering: A design for a global family of surveys and first results from germany. Information and Software Technology, 57:616--643, 2015.Google ScholarCross Ref
- M. T. Frohlich and R. Westbrook. Arcs of integration: an international study of supply chain strategies. Journal of operations management, 19(2):185--200, 2001.Google Scholar
- B. Glaser and A. Strauss. The discovery of grounded theory: Strategies for qualitative research. Aldine Transaction (a division of Transaction Publishers), New Brunswick, N.J., 2006.Google Scholar
- L. Gren, R. Torkar, and R. Feldt. The prospects of a quantitative measurement of agility: A validation study on an agile maturity model. Journal of Systems and Software, 107:38--49, 2015. Google ScholarDigital Library
- N. Hanebutte, C. S. Taylor, and R. R. Dumke. Techniques of successful application of factor analysis in software measurement. Empirical Software Engineering, 8(1):43--57, 2003. Google ScholarDigital Library
- J. Iivari and N. Iivari. The relationship between organizational culture and the deployment of agile methods. Information and Software Technology, 53(5):509--520, 2011. Google ScholarDigital Library
- I. Izquierdo, J. Olea, and F. J. Abad. Exploratory factor analysis in validation studies: Uses and recommendations. Psicothema, 26(3):395--400, 2014.Google Scholar
- B. A. Kitchenham, S. L. Pfleeger, L. M. Pickard, P. W. Jones, D. C. Hoaglin, K. El Emam, and J. Rosenberg. Preliminary guidelines for empirical research in software engineering. IEEE Transactions on Software Engineering, 28(8):721--734, 2002. Google ScholarDigital Library
- R. Kurzweil. How to create a mind: the secret of human thought revealed. Penguin Books, New York, N.Y., 2013. Google ScholarDigital Library
- R. C. MacCallum, K. F. Widaman, S. Zhang, and S. Hong. Sample size in factor analysis. Psychological methods, 4:84--99, 1999.Google ScholarCross Ref
- S. Messick. Validity of psychological assessment: validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American psychologist, 50(9):741, 1995.Google ScholarCross Ref
- M. B. Miles and A. M. Huberman. Qualitative data analysis: a sourcebook of new methods. Sage, Beverly Hills, 1984.Google Scholar
- P. Ranganath. Elevating teams from 'doing' agile to 'being' and 'living' agile. In Agile Conference (AGILE), 2011, pages 187--194, Aug 2011. Google ScholarDigital Library
- E. Schein. Organizational culture and leadership. Jossey-Bass, San Francisco, 4 edition, 2010.Google Scholar
- P. Serrador and J. K. Pinto. Does Agile work? -- A quantitative analysis of agile project success. International Journal of Project Management, 33(5):1040--1051, July 2015.Google ScholarCross Ref
- C. So and W. Scholl. Perceptive agile measurement: New instruments for quantitative studies in the pursuit of the social-psychological effect of agile practices. In Agile Processes in Software Engineering and Extreme Programming, pages 83--93. Springer, 2009.Google ScholarCross Ref
- B. Thompson and L. G. Daniel. Factor analytic evidence for the construct validity of scores: A historical overview and some guidelines. Educational and psychological measurement, 56(2):197--208, 1996.Google Scholar
- C. Tolfo and R. Wazlawick. The influence of organizational culture on the adoption of extreme programming. Journal of systems and software, 81(11):1955--1967, 2008. Google ScholarDigital Library
- C. Tolfo, R. Wazlawick, M. Ferreira, and F. Forcellini. Agile methods and organizational culture: Reflections about cultural levels. Journal of Software Maintenance and Evolution: Research and Practice, 23(6):423--441, 2011. Google ScholarDigital Library
- E. Whitworth and R. Biddle. The social nature of agile teams. In Agile Conference (AGILE), 2007, pages 26--36. IEEE, 2007. Google ScholarDigital Library
- C. Wohlin. Are individual differences in software development performance possible to capture using a quantitative survey? Empirical Software Engineering, 9(3):211--228, 2004. Google ScholarDigital Library
Index Terms
Useful statistical methods for human factors research in software engineering: a discussion on validation with quantitative data
Recommendations
Human factors related challenges in software engineering: an industrial perspective
CHASE '15: Proceedings of the Eighth International Workshop on Cooperative and Human Aspects of Software EngineeringIt is increasingly recognised that successful Software Engineering not only depends on technical or process issues, but requires attention to human factors. Researchers include such aspects which has led to both new theories and refined methods. However,...
Human Factors Related Challenges in Software Engineering -- An Industrial Perspective
CHASE '15: Proceedings of the 2015 IEEE/ACM 8th International Workshop on Cooperative and Human Aspects of Software EngineeringIt is increasingly recognised that successful Software Engineering not only depends on technical or process issues, but requires attention to human factors. Researchers include such aspects which has led to both new theories and refined methods. However,...
Human Factors in Visualization Research
Abstract--Visualization can provide valuable assistance for data analysis and decision making tasks. However, how people perceive and interact with a visualization tool can strongly influence their understanding of the data as well as the system's ...
Comments