skip to main content
10.1145/2810103.2813704acmconferencesArticle/Chapter ViewAbstractPublication PagesccsConference Proceedingsconference-collections
research-article
Public Access

An Empirical Study of Web Vulnerability Discovery Ecosystems

Authors Info & Claims
Published:12 October 2015Publication History

ABSTRACT

In recent years, many organizations have established bounty programs that attract white hat hackers who contribute vulnerability reports of web systems. In this paper, we collect publicly available data of two representative web vulnerability discovery ecosystems (Wooyun and HackerOne) and study their characteristics, trajectory, and impact. We find that both ecosystems include large and continuously growing white hat communities which have provided significant contributions to organizations from a wide range of business sectors. We also analyze vulnerability trends, response and resolve behaviors, and reward structures of participating organizations. Our analysis based on the HackerOne dataset reveals that a considerable number of organizations exhibit decreasing trends for reported web vulnerabilities. We further conduct a regression study which shows that monetary incentives have a significantly positive correlation with the number of vulnerabilities reported. Finally, we make recommendations aimed at increasing participation by white hats and organizations in such ecosystems.

References

  1. OWASP 2013 Top 10. www.owasp.org/index.php/Top_10_2013-Top_10.Google ScholarGoogle Scholar
  2. Updates on vulnerability handling process. www.wooyun.org/notice.php?action=view&id=28, 2013.Google ScholarGoogle Scholar
  3. Banks reluctant to use 'white hat' hackers to spot security flaws. NPR, 2014.Google ScholarGoogle Scholar
  4. Bug bounty highlights and updates. Facebook, 2014.Google ScholarGoogle Scholar
  5. How Bugcrowd uses crowdsourcing to uncover security flaws faster than the bad guys do (Interview). VentureBeat, 2014.Google ScholarGoogle Scholar
  6. Website security statistics report. White Hat Security, 2014.Google ScholarGoogle Scholar
  7. CSUS student hunts for computer bugs as a 'white hat'. www.sacbee.com/news/business/article5014716.html, 2015.Google ScholarGoogle Scholar
  8. Improving signal over 10,000 bugs. https://hackerone.com/blog, 2015.Google ScholarGoogle Scholar
  9. LinkedIn's private bug bounty program: Reducing vulnerabilities by leveraging expert crowds. security.linkedin.com, 2015.Google ScholarGoogle Scholar
  10. Small business website statistics. www.statisticbrain.com/small-business-website-statistics/, 2015.Google ScholarGoogle Scholar
  11. Start with security: A guide for business. FTC, 2015.Google ScholarGoogle Scholar
  12. The state of bug bounty. BugCrowd, 2015.Google ScholarGoogle Scholar
  13. A. Algarni and Y. Malaiya. Software vulnerability markets: Discoverers and buyers. International Journal of Computer, Information Science and Engineering, 8(3):71--81, 2014.Google ScholarGoogle Scholar
  14. R. Böhme. A comparison of market approaches to software vulnerability disclosure. In Emerging Trends in Information and Communication Security. 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. P. Chen, N. Nikiforakis, L. Desmet, and C. Huygens. Security analysis of the Chinese web: How well is it protected? In Workshop on Cyber Security Analytics, Intelligence and Automation, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. A. Doupé, M. Cova, and G. Vigna. Why Johnny can't pentest: An analysis of black-box web vulnerability scanners. In Detection of Intrusions and Malware, and Vulnerability Assessment, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. A. Edmundson, B. Holtkamp, E. Rivera, M. Finifter, A. Mettler, and D. Wagner. An empirical study on the effectiveness of security code review. In Engineering Secure Software and Systems, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. S. Egelman, C. Herley, and P. van Oorschot. Markets for zero-day exploits: Ethics and implications. In New Security Paradigms Workshop, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. M. Fang and M. Hafiz. Discovering buffer overflow vulnerabilities in the wild: An empirical study. In 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. M. Finifter, D. Akhawe, and D. Wagner. An empirical study of vulnerability rewards programs. In USENIX Security Symposium, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. S. Frei, D. Schatzmann, B. Plattner, and B. Trammell. Modeling the security ecosystem - The dynamics of (in)security. In Economics of Information Security and Privacy, 2009.Google ScholarGoogle Scholar
  22. J. Grossklags, N. Christin, and J. Chuang. Secure or insure? A game-theoretic analysis of information security games. In 17th International Conference on World Wide Web, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. B. Johnson, R. Böhme, and J. Grossklags. Security games with market insurance. In Decision and Game Theory for Security, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. K. Kannan and R. Telang. Market for software vulnerabilities? Think again. Management Science, 51(5):726--740, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. A. Laszka and J. Grossklags. Should cyber-insurance providers invest in software security? In 20th European Symposium on Research in Computer Security, 2015.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. A. Lotka. The frequency distribution of scientific productivity. Journal of Washington Academy Sciences, 16(12):317--323, 1926.Google ScholarGoogle Scholar
  27. R. McGeehan and L. Honeywell. Bounty launch lessons. medium.com/@magoo/bounty-launch-lessons-c7c3be3f5b, 2015.Google ScholarGoogle Scholar
  28. E. Messmer. Hacker group defies U.S. law, defends exposing McAfee website vulnerabilities. Network World, 2011.Google ScholarGoogle Scholar
  29. K. Moussouris. You need to speak up for internet security. Right now. Wired, 2015.Google ScholarGoogle Scholar
  30. A. Ozment. Bug auctions: Vulnerability markets reconsidered. In Workshop on the Economics of Information Security, 2004.Google ScholarGoogle Scholar
  31. A. Ozment. The likelihood of vulnerability rediscovery and the social utility of vulnerability hunting. In Workshop on the Econ. of Information Security, 2005.Google ScholarGoogle Scholar
  32. A. Ozment and S. Schechter. Milk or wine: Does software security improve with age? In USENIX Security Symposium, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. S. Preibusch and J. Bonneau. The password game: Negative externalities from weak password practices. In International Conference on Decision and Game Theory for Security, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. E. Rescorla. Is finding security holes a good idea? IEEE Security & Privacy, 3(1):14--19, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. G. Schryen. Is open source security a myth? Communications of the ACM, 54(5):130--140, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. M. Shahzad, M. Shafiq, and A. Liu. A large scale exploratory analysis of software vulnerability life cycles. In International Conference on Software Engineering, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. T. Van Goethem, F. Piessens, W. Joosen, and N. Nikiforakis. Clubbing seals: Exploring the ecosystem of third-party security seals. In ACM Conference on Computer and Communications Security, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. M. Zhao, J. Grossklags, and K. Chen. An exploratory study of white hat behaviors in a web vulnerability disclosure program. In 2014 ACM CCS Workshop on Security Information Workers, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. An Empirical Study of Web Vulnerability Discovery Ecosystems

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      CCS '15: Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security
      October 2015
      1750 pages
      ISBN:9781450338325
      DOI:10.1145/2810103

      Copyright © 2015 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 12 October 2015

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

      Acceptance Rates

      CCS '15 Paper Acceptance Rate128of660submissions,19%Overall Acceptance Rate1,261of6,999submissions,18%

      Upcoming Conference

      CCS '24
      ACM SIGSAC Conference on Computer and Communications Security
      October 14 - 18, 2024
      Salt Lake City , UT , USA

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader