ABSTRACT
Many different demands can be made of intrusion detection systems. An important requirement is that it be effective i.e. that it should detect a substantial percentage of intrusions into the supervised system, while still keeping the false alarm rate at an acceptable level.
This paper aims to demonstrate that, for a reasonable set of assumptions, the false alarm rate is the limiting factor for the performance of an intrusion detection system. This is due to the base-rate fallacy phenomenon, that in order to achieve substantial values of the Bayesian detection rate, P(Intrusion|Alarm), we have to achieve—a perhaps unattainably low—false alarm rate.
A selection of reports of intrusion detection performance are reviewed, and the conclusion is reached that there are indications that at least some types of intrusion detection have far to go before they can attain such low false alarm rates.
- 1.J. P. Anderson. Computer security threat monitoring and surveillance. Technical Report Contract 79F26400, James P. Anderson Co., Box 42, Fort Washington, PA, 19034, USA, Feb. 26, revised Apr. 15~ 1980.Google Scholar
- 2.S. Axelsson. Research in Intrusion-Detection systems: A Survey. Technical Report 98-17, Department of Computer Engineering Chedmers University of Technology, SE-412 96 GSteborg, Sweden, Dec. 1998. URL: htt p://www, ce. c h a liners .se / staff/sax.Google Scholar
- 3.S. Axelsson, U. Lindqvist, U. Gustafson, and E. Jonsson. An approach to UNIX security logging. In Proceedings of the ~lst National Information Systems Security Conference, pages 62-75, Crystal City, Arlington, VA~ USAj Oct. 5-8, 1998. NIST.Google Scholar
- 4.D. E. Denning. An intrusion-detection model. IEEE ff~ansactions on Software Engineemng, Vol. SE-13(No. 2):222-232, Feb. 1987. Google ScholarDigital Library
- 5.D. E. Denning and P. G. Neumann. Requirements and model for IDES--A real-time intrusion detection system. Technical report, Computer Science Laboratory, SRI International~ Menlo Park, CA, USA, 1985.Google Scholar
- 6.L. Halme and B. Kahn. Building a security monitor with adaptive user work profiles. In Procee&ngs of the 11th Nat,onal Computer Security Conference, Washington DC, Oct. 1988.Google Scholar
- 7.P. Helman and G. Liepins. Statistical foundations of audit trail analysis for the detection of computer misuse. IEEE Transactions on Software tEngineering~ 19(9):886--901, Sept. 1993. Google ScholarDigital Library
- 8.C. Ko, M. Ruschitzl~, and K. Levitt. Execution monitoring of security-critical programs in distributed systems: A specification-based approach. In Proceedings of the 1#97 IEEE Symposium on Security and Privacy, pages 175-187, Oakland, CA, USA, May 1997. Google ScholarDigital Library
- 9.T. Lane and C. E. Brodie. Temporal sequence learning and data reduction for anomaly detection, in 5th A CM Conference on Computer 8j Communications Security, pages 150-158, San Francisco, California, USA, Nov. 3- 5, 1998. Google ScholarDigital Library
- 10.W. Lee. A data mining framework for building intrusion detection models. In IEEE Symposium on Security and Pmvacy, pages 120-132, Berkeley~ California, May 1999.Google Scholar
- 11.R. P. Lippmann, I. Graf, S. L. Garfinkel, A. S. Gotton, K. R. Kendall, D. J. McClung, D. J. Weber, S. E. Webster, D. Wyschogrod, and M. A. Zissma~u. The 1998 DARPA/AFRL off-line intrusion detection evaluation. Presented to The First Intl. Workshop on Recent Advances in Intrusion Detection (RAID-98), Lovain-hu- Neuve, Belgium, No printed proceedings, Sept. 14-16, 1998.Google Scholar
- 12.T. F. Lunt. Automated audit trail analysis and intrusion detection: A survey. In Proceedings of the 11th Natzonat Computer Security Conference, pages 65-73, Baltimore, Marylazad, Oct. 17-20 1988. NIST.Google Scholar
- 13.R. Matthews. Base-rate errors and rain forecasts. Nature, 382(6594):766, Aug. 29 1996.Google ScholarCross Ref
- 14.R. Matthews. Decision-theoretic limits on earthquake prediction. Geophys. Jr. Int., 131(3):526--529, Dec. 1997.Google ScholarCross Ref
- 15.R. A. Maxion. Measuring intrusion-detection systems. Presented to The First Intl. Workshop on Recent Advances in Intrusion Detection (RAID-98), Lovain-la- Neuve, Belgium, No pmnted proceedings, Sept. 14-16, 1998.Google Scholar
- 16.G. McGuire Pierce. Destruction by demolition, incendiaries and sabotage. Field training manual, Fleet Marine Force, US Marine Corps, 1943-1948. Reprinted: Paladin Press, PO 1507, Boulder CO, USA.Google Scholar
- 17.S. J. Russel and P. Norvig. Artificial IntelhgenceLA Modern Approach, chapter 14, pages 426-435. Prentice Hall Series in Artificial Intelligence. Prentice Hall International, Inc., London, UK, first edition, 1995. Exercise 14.3. Google ScholarDigital Library
- 18.M. M. Sebring, E. Shellhouse, M. E. Hanna, mad R. A. Whitehurst. Expert systems in intrusion detection: A case study. In Proceedings of the 11th National Computer Securzty Conference, pages 74-81, Baltimore, Maryland, Oct. 17-20, 1988. NIST.Google Scholar
- 19.It. S. Vaccaro and G. E. Liepins. Detection of anomalous computer session activity. In Proceedzngs of the 1989 IEEE Symposium on Securely and Privacy, pages 280-289, Oakland, California, May 1-3, 1989.Google ScholarCross Ref
- 20.H. L. Van Trees. Detection, Estimation, and Modulation Theory, Part I, Detection, Estimation, and Linear Modulation Theory. john Wiley and Sons, inc., 1968. Google ScholarDigital Library
- 21.C. Warrender, S. Forrest, and B. Perlmutter. Detecting intrusions using system calls: Alternative data models. In IEEE Symposium on Security and Privacy, pages 133-145~ Berkeley~ California, May 1999.Google ScholarCross Ref
Index Terms
The base-rate fallacy and its implications for the difficulty of intrusion detection
Recommendations
The base-rate fallacy and the difficulty of intrusion detection
Many different demands can be made of intrusion detection systems. An important requirement is that an intrusion detection system be effective; that is, it should detect a substantial percentage of intrusions into the supervised system, while still ...
Data base support for intrusion detection with honeynets
TELE-INFO'07: Proceedings of the 6th WSEAS Int. Conference on Telecommunications and InformaticsAs computer attacks are becoming more and more difficult to identify the need for better and more efficient intrusion detection systems increases. The main problem with current intrusion detection systems is high rate of false alarms. In this paper we ...
Comments