skip to main content
10.1145/2897586.2897588acmconferencesArticle/Chapter ViewAbstractPublication PagesicseConference Proceedingsconference-collections
short-paper

Useful statistical methods for human factors research in software engineering: a discussion on validation with quantitative data

Published:14 May 2016Publication History

ABSTRACT

In this paper we describe the usefulness of statistical validation techniques for human factors survey research. We need to investigate a diversity of validity aspects when creating metrics in human factors research, and we argue that the statistical tests used in other fields to get support for reliability and construct validity in surveys, should also be applied to human factors research in software engineering more often. We also show briefly how such methods can be applied (Test-Retest, Cronbach's α, and Exploratory Factor Analysis).

References

  1. D. J. Bartholomew. Stochastic models for social processes. Wiley, London, 1967.Google ScholarGoogle Scholar
  2. A. Bosu, J. Carver, R. Guadagno, B. Bassett, D. McCallum, and L. Hochstein. Peer impressions in open source organizations: A survey. Journal of Systems and Software, 94:4--15, 2014.Google ScholarGoogle ScholarCross RefCross Ref
  3. G. E. P. Box and N. R. Draper. Empirical Model-building and Response Surface. John Wiley & Sons, Inc., New York, NY, USA, 1986. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. J. Chen, J. Xiao, Q. Wang, L. J. Osterweil, and M. Li. Perspectives on refactoring planning and practice: an empirical study. Empirical Software Engineering, pages 1--40, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. L. Cronbach. Coefficient alpha and the internal structure of tests. Psychometrika, 16(3):297--334, 1951.Google ScholarGoogle ScholarCross RefCross Ref
  6. L. Fabrigar and D. Wegener. Exploratory Factor Analysis. Series in understanding statistics. OUP USA, 2012.Google ScholarGoogle Scholar
  7. D. M. Fernández and S. Wagner. Naming the pain in requirements engineering: A design for a global family of surveys and first results from germany. Information and Software Technology, 57:616--643, 2015.Google ScholarGoogle ScholarCross RefCross Ref
  8. M. T. Frohlich and R. Westbrook. Arcs of integration: an international study of supply chain strategies. Journal of operations management, 19(2):185--200, 2001.Google ScholarGoogle Scholar
  9. B. Glaser and A. Strauss. The discovery of grounded theory: Strategies for qualitative research. Aldine Transaction (a division of Transaction Publishers), New Brunswick, N.J., 2006.Google ScholarGoogle Scholar
  10. L. Gren, R. Torkar, and R. Feldt. The prospects of a quantitative measurement of agility: A validation study on an agile maturity model. Journal of Systems and Software, 107:38--49, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. N. Hanebutte, C. S. Taylor, and R. R. Dumke. Techniques of successful application of factor analysis in software measurement. Empirical Software Engineering, 8(1):43--57, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. J. Iivari and N. Iivari. The relationship between organizational culture and the deployment of agile methods. Information and Software Technology, 53(5):509--520, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. I. Izquierdo, J. Olea, and F. J. Abad. Exploratory factor analysis in validation studies: Uses and recommendations. Psicothema, 26(3):395--400, 2014.Google ScholarGoogle Scholar
  14. B. A. Kitchenham, S. L. Pfleeger, L. M. Pickard, P. W. Jones, D. C. Hoaglin, K. El Emam, and J. Rosenberg. Preliminary guidelines for empirical research in software engineering. IEEE Transactions on Software Engineering, 28(8):721--734, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. R. Kurzweil. How to create a mind: the secret of human thought revealed. Penguin Books, New York, N.Y., 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. R. C. MacCallum, K. F. Widaman, S. Zhang, and S. Hong. Sample size in factor analysis. Psychological methods, 4:84--99, 1999.Google ScholarGoogle ScholarCross RefCross Ref
  17. S. Messick. Validity of psychological assessment: validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American psychologist, 50(9):741, 1995.Google ScholarGoogle ScholarCross RefCross Ref
  18. M. B. Miles and A. M. Huberman. Qualitative data analysis: a sourcebook of new methods. Sage, Beverly Hills, 1984.Google ScholarGoogle Scholar
  19. P. Ranganath. Elevating teams from 'doing' agile to 'being' and 'living' agile. In Agile Conference (AGILE), 2011, pages 187--194, Aug 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. E. Schein. Organizational culture and leadership. Jossey-Bass, San Francisco, 4 edition, 2010.Google ScholarGoogle Scholar
  21. P. Serrador and J. K. Pinto. Does Agile work? -- A quantitative analysis of agile project success. International Journal of Project Management, 33(5):1040--1051, July 2015.Google ScholarGoogle ScholarCross RefCross Ref
  22. C. So and W. Scholl. Perceptive agile measurement: New instruments for quantitative studies in the pursuit of the social-psychological effect of agile practices. In Agile Processes in Software Engineering and Extreme Programming, pages 83--93. Springer, 2009.Google ScholarGoogle ScholarCross RefCross Ref
  23. B. Thompson and L. G. Daniel. Factor analytic evidence for the construct validity of scores: A historical overview and some guidelines. Educational and psychological measurement, 56(2):197--208, 1996.Google ScholarGoogle Scholar
  24. C. Tolfo and R. Wazlawick. The influence of organizational culture on the adoption of extreme programming. Journal of systems and software, 81(11):1955--1967, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. C. Tolfo, R. Wazlawick, M. Ferreira, and F. Forcellini. Agile methods and organizational culture: Reflections about cultural levels. Journal of Software Maintenance and Evolution: Research and Practice, 23(6):423--441, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. E. Whitworth and R. Biddle. The social nature of agile teams. In Agile Conference (AGILE), 2007, pages 26--36. IEEE, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. C. Wohlin. Are individual differences in software development performance possible to capture using a quantitative survey? Empirical Software Engineering, 9(3):211--228, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Useful statistical methods for human factors research in software engineering: a discussion on validation with quantitative data

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in
      • Published in

        cover image ACM Conferences
        CHASE '16: Proceedings of the 9th International Workshop on Cooperative and Human Aspects of Software Engineering
        May 2016
        142 pages
        ISBN:9781450341554
        DOI:10.1145/2897586

        Copyright © 2016 ACM

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 14 May 2016

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • short-paper

        Acceptance Rates

        Overall Acceptance Rate47of70submissions,67%

        Upcoming Conference

        ICSE 2024

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader