Designing ethical phishing experiments: a study of (ROT13) rOnl query features

ABSTRACT

We study how to design experiments to measure the success rates of phishing attacks that are ethical and accurate, which are two requirements of contradictory forces. Namely, an ethical experiment must not expose the participants to any risk; it should be possible to locally verify by the participants or representatives thereof that this was the case. At the same time, an experiment is accurate if it is possible to argue why its success rate is not an upper or lower bound of that of a real attack -- this may be difficult if the ethics considerations make the user perception of the experiment different from the user perception of the attack. We introduce several experimental techniques allowing us to achieve a balance between these two requirements, and demonstrate how to apply these, using a context aware phishing experiment on a popular online auction site which we call "rOnl". Our experiments exhibit a measured average yield of 11% per collection of unique users. This study was authorized by the Human Subjects Committee at Indiana University (Study #05-10306).

REFERENCES

• 1: Mailfrontier phishing IQ test. http://survey.mailfrontier.com/survey/quiztest.html.
• 2: Know your enemy : Phishing. behind the scenes of phishing attacks. http://www.honeynet.org/papers/phishing/, 2005.
• 3: Simson L. Garfinkel , Robert C. Miller, Johnny 2: a user test of key continuity management with S/MIME and Outlook Express, Proceedings of the 2005 symposium on Usable privacy and security, p.13-24, July 06-08, 2005, Pittsburgh, Pennsylvania [doi>10.1145/1073001.1073003]
• 4: Jakobsson, M. Modeling and preventing phishing attacks. In Financial Cryptography (2005).
• 5: Lester, A. WWW::Mechanize - handy web browsing in a perl object. http://search.cpan.org/ petdance/WWW-Mechanize-1.16/lib/WWW/Mechanize.p%m, 2005.
• 6: Litan, A. Phishing attack victims likely targets for identity theft. FT-22-8873, Gartner Research (2004).
• 7: M. Jakobsson, T. Jagatic, S. S. Phishing for clues. www.browser-recon.info.
• 8: T. Jagatic, N. Johnson, M. J., and Menczer, F. Social phishing. 2006.

INDEX TERMS

Primary Classification:
  K. Computing Milieux
  K.4 COMPUTERS AND SOCIETY
      K.4.4 Electronic Commerce
          Subjects: Security

Additional Classification:
  K. Computing Milieux
  K.4 COMPUTERS AND SOCIETY
      K.4.1 Public Policy Issues
          Subjects: Ethics

General Terms:
Experimentation, Human Factors, Legal Aspects, Security

Keywords:
accurate, ethical, experiment, phishing, security