skip to main content
10.1145/319709.319710acmconferencesArticle/Chapter ViewAbstractPublication PagesccsConference Proceedingsconference-collections
Article
Free Access

The base-rate fallacy and its implications for the difficulty of intrusion detection

Authors Info & Claims
Published:01 November 1999Publication History

ABSTRACT

Many different demands can be made of intrusion detection systems. An important requirement is that it be effective i.e. that it should detect a substantial percentage of intrusions into the supervised system, while still keeping the false alarm rate at an acceptable level.

This paper aims to demonstrate that, for a reasonable set of assumptions, the false alarm rate is the limiting factor for the performance of an intrusion detection system. This is due to the base-rate fallacy phenomenon, that in order to achieve substantial values of the Bayesian detection rate, P(Intrusion|Alarm), we have to achieve—a perhaps unattainably low—false alarm rate.

A selection of reports of intrusion detection performance are reviewed, and the conclusion is reached that there are indications that at least some types of intrusion detection have far to go before they can attain such low false alarm rates.

References

  1. 1.J. P. Anderson. Computer security threat monitoring and surveillance. Technical Report Contract 79F26400, James P. Anderson Co., Box 42, Fort Washington, PA, 19034, USA, Feb. 26, revised Apr. 15~ 1980.Google ScholarGoogle Scholar
  2. 2.S. Axelsson. Research in Intrusion-Detection systems: A Survey. Technical Report 98-17, Department of Computer Engineering Chedmers University of Technology, SE-412 96 GSteborg, Sweden, Dec. 1998. URL: htt p://www, ce. c h a liners .se / staff/sax.Google ScholarGoogle Scholar
  3. 3.S. Axelsson, U. Lindqvist, U. Gustafson, and E. Jonsson. An approach to UNIX security logging. In Proceedings of the ~lst National Information Systems Security Conference, pages 62-75, Crystal City, Arlington, VA~ USAj Oct. 5-8, 1998. NIST.Google ScholarGoogle Scholar
  4. 4.D. E. Denning. An intrusion-detection model. IEEE ff~ansactions on Software Engineemng, Vol. SE-13(No. 2):222-232, Feb. 1987. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. 5.D. E. Denning and P. G. Neumann. Requirements and model for IDES--A real-time intrusion detection system. Technical report, Computer Science Laboratory, SRI International~ Menlo Park, CA, USA, 1985.Google ScholarGoogle Scholar
  6. 6.L. Halme and B. Kahn. Building a security monitor with adaptive user work profiles. In Procee&ngs of the 11th Nat,onal Computer Security Conference, Washington DC, Oct. 1988.Google ScholarGoogle Scholar
  7. 7.P. Helman and G. Liepins. Statistical foundations of audit trail analysis for the detection of computer misuse. IEEE Transactions on Software tEngineering~ 19(9):886--901, Sept. 1993. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. 8.C. Ko, M. Ruschitzl~, and K. Levitt. Execution monitoring of security-critical programs in distributed systems: A specification-based approach. In Proceedings of the 1#97 IEEE Symposium on Security and Privacy, pages 175-187, Oakland, CA, USA, May 1997. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. 9.T. Lane and C. E. Brodie. Temporal sequence learning and data reduction for anomaly detection, in 5th A CM Conference on Computer 8j Communications Security, pages 150-158, San Francisco, California, USA, Nov. 3- 5, 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. 10.W. Lee. A data mining framework for building intrusion detection models. In IEEE Symposium on Security and Pmvacy, pages 120-132, Berkeley~ California, May 1999.Google ScholarGoogle Scholar
  11. 11.R. P. Lippmann, I. Graf, S. L. Garfinkel, A. S. Gotton, K. R. Kendall, D. J. McClung, D. J. Weber, S. E. Webster, D. Wyschogrod, and M. A. Zissma~u. The 1998 DARPA/AFRL off-line intrusion detection evaluation. Presented to The First Intl. Workshop on Recent Advances in Intrusion Detection (RAID-98), Lovain-hu- Neuve, Belgium, No printed proceedings, Sept. 14-16, 1998.Google ScholarGoogle Scholar
  12. 12.T. F. Lunt. Automated audit trail analysis and intrusion detection: A survey. In Proceedings of the 11th Natzonat Computer Security Conference, pages 65-73, Baltimore, Marylazad, Oct. 17-20 1988. NIST.Google ScholarGoogle Scholar
  13. 13.R. Matthews. Base-rate errors and rain forecasts. Nature, 382(6594):766, Aug. 29 1996.Google ScholarGoogle ScholarCross RefCross Ref
  14. 14.R. Matthews. Decision-theoretic limits on earthquake prediction. Geophys. Jr. Int., 131(3):526--529, Dec. 1997.Google ScholarGoogle ScholarCross RefCross Ref
  15. 15.R. A. Maxion. Measuring intrusion-detection systems. Presented to The First Intl. Workshop on Recent Advances in Intrusion Detection (RAID-98), Lovain-la- Neuve, Belgium, No pmnted proceedings, Sept. 14-16, 1998.Google ScholarGoogle Scholar
  16. 16.G. McGuire Pierce. Destruction by demolition, incendiaries and sabotage. Field training manual, Fleet Marine Force, US Marine Corps, 1943-1948. Reprinted: Paladin Press, PO 1507, Boulder CO, USA.Google ScholarGoogle Scholar
  17. 17.S. J. Russel and P. Norvig. Artificial IntelhgenceLA Modern Approach, chapter 14, pages 426-435. Prentice Hall Series in Artificial Intelligence. Prentice Hall International, Inc., London, UK, first edition, 1995. Exercise 14.3. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. 18.M. M. Sebring, E. Shellhouse, M. E. Hanna, mad R. A. Whitehurst. Expert systems in intrusion detection: A case study. In Proceedings of the 11th National Computer Securzty Conference, pages 74-81, Baltimore, Maryland, Oct. 17-20, 1988. NIST.Google ScholarGoogle Scholar
  19. 19.It. S. Vaccaro and G. E. Liepins. Detection of anomalous computer session activity. In Proceedzngs of the 1989 IEEE Symposium on Securely and Privacy, pages 280-289, Oakland, California, May 1-3, 1989.Google ScholarGoogle ScholarCross RefCross Ref
  20. 20.H. L. Van Trees. Detection, Estimation, and Modulation Theory, Part I, Detection, Estimation, and Linear Modulation Theory. john Wiley and Sons, inc., 1968. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. 21.C. Warrender, S. Forrest, and B. Perlmutter. Detecting intrusions using system calls: Alternative data models. In IEEE Symposium on Security and Privacy, pages 133-145~ Berkeley~ California, May 1999.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. The base-rate fallacy and its implications for the difficulty of intrusion detection

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in
        • Published in

          cover image ACM Conferences
          CCS '99: Proceedings of the 6th ACM conference on Computer and communications security
          November 1999
          160 pages
          ISBN:1581131488
          DOI:10.1145/319709

          Copyright © 1999 ACM

          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 1 November 1999

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • Article

          Acceptance Rates

          Overall Acceptance Rate1,261of6,999submissions,18%

          Upcoming Conference

          CCS '24
          ACM SIGSAC Conference on Computer and Communications Security
          October 14 - 18, 2024
          Salt Lake City , UT , USA

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader