ABSTRACT
In recent years, many organizations have established bounty programs that attract white hat hackers who contribute vulnerability reports of web systems. In this paper, we collect publicly available data of two representative web vulnerability discovery ecosystems (Wooyun and HackerOne) and study their characteristics, trajectory, and impact. We find that both ecosystems include large and continuously growing white hat communities which have provided significant contributions to organizations from a wide range of business sectors. We also analyze vulnerability trends, response and resolve behaviors, and reward structures of participating organizations. Our analysis based on the HackerOne dataset reveals that a considerable number of organizations exhibit decreasing trends for reported web vulnerabilities. We further conduct a regression study which shows that monetary incentives have a significantly positive correlation with the number of vulnerabilities reported. Finally, we make recommendations aimed at increasing participation by white hats and organizations in such ecosystems.
- OWASP 2013 Top 10. www.owasp.org/index.php/Top_10_2013-Top_10.Google Scholar
- Updates on vulnerability handling process. www.wooyun.org/notice.php?action=view&id=28, 2013.Google Scholar
- Banks reluctant to use 'white hat' hackers to spot security flaws. NPR, 2014.Google Scholar
- Bug bounty highlights and updates. Facebook, 2014.Google Scholar
- How Bugcrowd uses crowdsourcing to uncover security flaws faster than the bad guys do (Interview). VentureBeat, 2014.Google Scholar
- Website security statistics report. White Hat Security, 2014.Google Scholar
- CSUS student hunts for computer bugs as a 'white hat'. www.sacbee.com/news/business/article5014716.html, 2015.Google Scholar
- Improving signal over 10,000 bugs. https://hackerone.com/blog, 2015.Google Scholar
- LinkedIn's private bug bounty program: Reducing vulnerabilities by leveraging expert crowds. security.linkedin.com, 2015.Google Scholar
- Small business website statistics. www.statisticbrain.com/small-business-website-statistics/, 2015.Google Scholar
- Start with security: A guide for business. FTC, 2015.Google Scholar
- The state of bug bounty. BugCrowd, 2015.Google Scholar
- A. Algarni and Y. Malaiya. Software vulnerability markets: Discoverers and buyers. International Journal of Computer, Information Science and Engineering, 8(3):71--81, 2014.Google Scholar
- R. Böhme. A comparison of market approaches to software vulnerability disclosure. In Emerging Trends in Information and Communication Security. 2006. Google ScholarDigital Library
- P. Chen, N. Nikiforakis, L. Desmet, and C. Huygens. Security analysis of the Chinese web: How well is it protected? In Workshop on Cyber Security Analytics, Intelligence and Automation, 2014. Google ScholarDigital Library
- A. Doupé, M. Cova, and G. Vigna. Why Johnny can't pentest: An analysis of black-box web vulnerability scanners. In Detection of Intrusions and Malware, and Vulnerability Assessment, 2010. Google ScholarDigital Library
- A. Edmundson, B. Holtkamp, E. Rivera, M. Finifter, A. Mettler, and D. Wagner. An empirical study on the effectiveness of security code review. In Engineering Secure Software and Systems, 2013. Google ScholarDigital Library
- S. Egelman, C. Herley, and P. van Oorschot. Markets for zero-day exploits: Ethics and implications. In New Security Paradigms Workshop, 2013. Google ScholarDigital Library
- M. Fang and M. Hafiz. Discovering buffer overflow vulnerabilities in the wild: An empirical study. In 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, 2014. Google ScholarDigital Library
- M. Finifter, D. Akhawe, and D. Wagner. An empirical study of vulnerability rewards programs. In USENIX Security Symposium, 2013. Google ScholarDigital Library
- S. Frei, D. Schatzmann, B. Plattner, and B. Trammell. Modeling the security ecosystem - The dynamics of (in)security. In Economics of Information Security and Privacy, 2009.Google Scholar
- J. Grossklags, N. Christin, and J. Chuang. Secure or insure? A game-theoretic analysis of information security games. In 17th International Conference on World Wide Web, 2008. Google ScholarDigital Library
- B. Johnson, R. Böhme, and J. Grossklags. Security games with market insurance. In Decision and Game Theory for Security, 2011. Google ScholarDigital Library
- K. Kannan and R. Telang. Market for software vulnerabilities? Think again. Management Science, 51(5):726--740, 2005. Google ScholarDigital Library
- A. Laszka and J. Grossklags. Should cyber-insurance providers invest in software security? In 20th European Symposium on Research in Computer Security, 2015.Google ScholarDigital Library
- A. Lotka. The frequency distribution of scientific productivity. Journal of Washington Academy Sciences, 16(12):317--323, 1926.Google Scholar
- R. McGeehan and L. Honeywell. Bounty launch lessons. medium.com/@magoo/bounty-launch-lessons-c7c3be3f5b, 2015.Google Scholar
- E. Messmer. Hacker group defies U.S. law, defends exposing McAfee website vulnerabilities. Network World, 2011.Google Scholar
- K. Moussouris. You need to speak up for internet security. Right now. Wired, 2015.Google Scholar
- A. Ozment. Bug auctions: Vulnerability markets reconsidered. In Workshop on the Economics of Information Security, 2004.Google Scholar
- A. Ozment. The likelihood of vulnerability rediscovery and the social utility of vulnerability hunting. In Workshop on the Econ. of Information Security, 2005.Google Scholar
- A. Ozment and S. Schechter. Milk or wine: Does software security improve with age? In USENIX Security Symposium, 2006. Google ScholarDigital Library
- S. Preibusch and J. Bonneau. The password game: Negative externalities from weak password practices. In International Conference on Decision and Game Theory for Security, 2010. Google ScholarDigital Library
- E. Rescorla. Is finding security holes a good idea? IEEE Security & Privacy, 3(1):14--19, 2005. Google ScholarDigital Library
- G. Schryen. Is open source security a myth? Communications of the ACM, 54(5):130--140, 2011. Google ScholarDigital Library
- M. Shahzad, M. Shafiq, and A. Liu. A large scale exploratory analysis of software vulnerability life cycles. In International Conference on Software Engineering, 2012. Google ScholarDigital Library
- T. Van Goethem, F. Piessens, W. Joosen, and N. Nikiforakis. Clubbing seals: Exploring the ecosystem of third-party security seals. In ACM Conference on Computer and Communications Security, 2014. Google ScholarDigital Library
- M. Zhao, J. Grossklags, and K. Chen. An exploratory study of white hat behaviors in a web vulnerability disclosure program. In 2014 ACM CCS Workshop on Security Information Workers, 2014. Google ScholarDigital Library
Index Terms
An Empirical Study of Web Vulnerability Discovery Ecosystems
Recommendations
An Exploratory Study of White Hat Behaviors in a Web Vulnerability Disclosure Program
SIW '14: Proceedings of the 2014 ACM Workshop on Security Information WorkersWhite hats are making significant contributions to cybersecurity by submitting vulnerability discovery reports to public vulnerability disclosure programs and company-initiated vulnerability reward programs. In this paper, we study white hat behaviors ...
The Benefits of Vulnerability Discovery and Bug Bounty Programs: Case Studies of Chromium and Firefox
WWW '23: Proceedings of the ACM Web Conference 2023Recently, bug-bounty programs have gained popularity and become a significant part of the security culture of many organizations. Bug-bounty programs enable organizations to enhance their security posture by harnessing the diverse expertise of crowds of ...
New Hurdles for Vulnerability Disclosure
Vulnerability disclosure is an important part of information security. In recent years, vulnerabilities in specific Web sites and SCADA implementations have created new hurdles for vulnerability disclosure. These aspects of information security have ...
Comments