skip to main content
10.1145/2501604.2501617acmotherconferencesArticle/Chapter ViewAbstractPublication PagessoupsConference Proceedingsconference-collections
research-article

On the ecological validity of a password study

Published: 24 July 2013 Publication History

Abstract

The ecological validity of password studies is a complex topic and difficult to quantify. Most researchers who conduct password user studies try to address the issue in their study design. However, the methods researchers use to try to improve ecological validity vary and some methods even contradict each other. One reason for this is that the very nature of the problem of ecological validity of password studies is hard to study, due to the lack of ground truth. In this paper, we present a study on the ecological validity of password studies designed specifically to shed light on this issue. We were able to compare the behavior of 645 study participants with their real world password choices. We conducted both online and laboratory studies, under priming and non-priming conditions, to be able to evaluate the effects of these different forms of password studies. While our study is able to investigate only one specific password environment used by a limited population and thus cannot answer all questions about ecological validity, it does represent a first important step in judging the impact of ecological validity on password studies.

References

[1]
A. Adams and M. A. Sasse. Users are not the enemy. Communications of the ACM, 42(12):40--46, 1999.
[2]
J. Bonneau. The science of guessing: analyzing an anonymized corpus of 70 million passwords. In Proc. IEEE S&P, 2012.
[3]
C. Bravo-Lillo, L. Cranor, J. Downs, S. Komanduri, S. Schechter, and M. Sleeper. Operating System Framed in Case of Mistaken Identity: Measuring The Success of Web-based Spoofing Attacks on OS Password-entry Dialogs. In Proc. ACM CCS, 2012.
[4]
M. Buhrmester, T. Kwang, and S. D. Gosling. Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data? Perspectives on Psychological Science, 6(1):3--5, Feb. 2011.
[5]
S. Chiasson, R. Biddle, and P. C. Van Oorschot. A second look at the usability of click-based graphical passwords. In Proc. SOUPS. ACM, July 2007.
[6]
A. Forget, S. Chiasson, P. C. Van Oorschot, and R. Biddle. Improving text passwords through persuasion. In Proc. SOUPS. ACM, July 2008.
[7]
S. Gaw and E. W. Felten. Password management strategies for online accounts. In Proc. SOUPS. ACM, 2006.
[8]
S. M. T. Haque, M. Wright, and S. Scielzo. A study of user password strategy for multiple accounts. In Proc. CODASPY. ACM, 2013.
[9]
M. Just and D. Aspinall. Personal choice and challenge questions: a security and usability assessment. Proceedings of the 5th Symposium on Usable Privacy and Security, page 8, 2009.
[10]
P. G. Kelley, S. Komanduri, M. L. Mazurek, R. Shay, T. Vidas, L. Bauer, N. Christin, L. F. Cranor, and J. Lopez. Guess again (and again and again): Measuring password strength by simulating password-cracking algorithms. In Proc. IEEE S&P, pages 523--537, 2012.
[11]
S. Komanduri, R. Shay, P. G. Kelley, M. L. Mazurek, L. Bauer, N. Christin, L. F. Cranor, and S. Egelman. Of passwords and people: measuring the effect of password-composition policies. In Proc. CHI. ACM, 2011.
[12]
S. E. Schechter, R. Dhamija, A. Ozment, and I. Fischer. The emperor's new security indicators. In Proceedings of the 2007 IEEE Symposium on Security and Privacy, SP '07, pages 51--65, Washington, DC, USA, 2007. IEEE Computer Society.
[13]
R. Shay, P. G. Kelley, S. Komanduri, M. L. Mazurek, B. Ur, T. Vidas, L. Bauer, N. Christin, and L. F. Cranor. Correct horse battery staple: Exploring the usability of system-assigned passphrases. In Proc. SOUPS, page 7, 2012.
[14]
R. Shay, S. Komanduri, P. G. Kelley, P. G. Leon, M. L. Mazurek, L. Bauer, N. Christin, and L. F. Cranor. Encountering stronger password requirements: user attitudes and behaviors. In Proc. SOUPS, volume 10, 2010.
[15]
B. Ur, P. G. Kelley, S. Komanduri, J. Lee, M. Maass, M. Mazurek, T. Passaro, R. Shay, T. Vidas, and L. Bauer. How does your password measure up? The effect of strength meters on password creation. In Proc. USENIX, 2012.
[16]
M. Weir, S. Aggarwal, M. Collins, and H. Stern. Testing metrics for password creation policies by attacking large sets of revealed passwords. In Proc. ACM CCS, pages 162--175, 2010.
[17]
M. Weir, S. Aggarwal, B. de Medeiros, and B. Glodek. Password Cracking Using Probabilistic Context-Free Grammars. In Proc. IEEE S&P, pages 391--405, 2009.
[18]
N. H. Zakaria, D. Griffiths, S. Brostoff, and J. Yan. Shoulder surfing defence for recall-based graphical passwords. In Proc. SOUPS, page 6, 2011.

Cited By

View all
  • (2024)Leveraging the Power of Storytelling to Encourage and Empower Children towards Strong PasswordsProceedings of the ACM on Human-Computer Interaction10.1145/36870438:CSCW2(1-27)Online publication date: 8-Nov-2024
  • (2024)Priming through Persuasion: Towards Secure Password BehaviorProceedings of the ACM on Human-Computer Interaction10.1145/36373878:CSCW1(1-27)Online publication date: 26-Apr-2024
  • (2024)Hidden in Onboarding: Cyber Hygiene Training and AssessmentHCI for Cybersecurity, Privacy and Trust10.1007/978-3-031-61379-1_4(53-63)Online publication date: 29-Jun-2024
  • Show More Cited By

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
SOUPS '13: Proceedings of the Ninth Symposium on Usable Privacy and Security
July 2013
241 pages
ISBN:9781450323192
DOI:10.1145/2501604
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

  • Carnegie Mellon University: Carnegie Mellon University

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 July 2013

Check for updates

Author Tags

  1. ecological validity
  2. passwords
  3. usable security

Qualifiers

  • Research-article

Conference

SOUPS '13
Sponsor:
  • Carnegie Mellon University
SOUPS '13: Symposium On Usable Privacy and Security
July 24 - 26, 2013
Newcastle, United Kingdom

Acceptance Rates

Overall Acceptance Rate 15 of 49 submissions, 31%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)71
  • Downloads (Last 6 weeks)7
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Leveraging the Power of Storytelling to Encourage and Empower Children towards Strong PasswordsProceedings of the ACM on Human-Computer Interaction10.1145/36870438:CSCW2(1-27)Online publication date: 8-Nov-2024
  • (2024)Priming through Persuasion: Towards Secure Password BehaviorProceedings of the ACM on Human-Computer Interaction10.1145/36373878:CSCW1(1-27)Online publication date: 26-Apr-2024
  • (2024)Hidden in Onboarding: Cyber Hygiene Training and AssessmentHCI for Cybersecurity, Privacy and Trust10.1007/978-3-031-61379-1_4(53-63)Online publication date: 29-Jun-2024
  • (2023)Prospects for improving password selectionProceedings of the Nineteenth USENIX Conference on Usable Privacy and Security10.5555/3632186.3632201(263-282)Online publication date: 7-Aug-2023
  • (2023)Can Password Meter be More Effective Towards User Attention, Engagement, and Attachment?: A Study of Metaphor-based DesignsCompanion Publication of the 2023 Conference on Computer Supported Cooperative Work and Social Computing10.1145/3584931.3606983(164-171)Online publication date: 14-Oct-2023
  • (2023)Story-based authentication for mobile devices using semantically-linked imagesInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2022.102967171:COnline publication date: 1-Mar-2023
  • (2022)Can Humans Detect Malicious Always-Listening Assistants? A Framework for Crowdsourcing Test DrivesProceedings of the ACM on Human-Computer Interaction10.1145/35556136:CSCW2(1-28)Online publication date: 11-Nov-2022
  • (2022)Can I Borrow Your ATM? Using Virtual Reality for (Simulated) In Situ Authentication Research2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)10.1109/VR51125.2022.00049(301-310)Online publication date: Mar-2022
  • (2022)MPO: MQTT-Based Privacy Orchestrator for Smart Home Users2022 IEEE 46th Annual Computers, Software, and Applications Conference (COMPSAC)10.1109/COMPSAC54236.2022.00152(988-993)Online publication date: Jun-2022
  • (2022)Enhancing the user authentication process with colour memory cuesBehaviour & Information Technology10.1080/0144929X.2022.209147442:10(1548-1567)Online publication date: 15-Jul-2022
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media