Article

Scalable network-based buffer overflow attack detection

Authors:

Tzi-cker ChiuehAuthors Info & Claims

ANCS '06: Proceedings of the 2006 ACM/IEEE symposium on Architecture for networking and communications systems

Pages 163 - 172

https://doi.org/10.1145/1185347.1185370

Published: 03 December 2006 Publication History

Abstract

Buffer overflow attack is the main attack method that most if not all existing malicious worms use to propagate themselves from machine to machine. Although a great deal of research has been invested in defense mechanisms against buffer overflow attack, most of them require modifications to the network applications and/or the platforms that host them. Being an extension work of CTCP, this paper presents a network-based low performance overhead buffer overflow attack detection system called Nebula ¹ NEtwork-based BUffer overfLow Attack detection, which can detect both known and zero-day buffer overflow attacks based solely on the packets observed without requiring any modifications to the end hosts. Moreover, instead of deriving a specific signature for each individual buffer overflow attack instance, Nebula uses a generalized signature that can capture all known variants of buffer overflow attacks while reducing the number of false positives to a negligible level. In addition, Nebula is built on a centralized TCP/IP architecture that effectively defeats all existing NIDS evasion techniques. Finally, Nebula incorporates a payload type identification mechanism that reduces further the false positive rate and scales the proposed buffer overflow attack detection scheme to gigabit network links.

References

[1]

Prashant Pradhan, Tzi-cker Chiueh, Anindya Neogi, "Aggregate TCP Congestion Control Using Multiple Network Probing," ICDCS 2000.

Digital Library

[2]

Tzi-cker Chiueh and Fu-Hau Hsu, "RAD: A Compiler Time Solution to Buffer Overflow Attacks," Proceeding of ICDCS 2001, Arizon USA, April 2001

Digital Library

[3]

C. Cowan, C. Pu, D. Maier, J. Walpole, P. Bakke, S. Beattie, A. Grier, P. Wagle, Q. Zhang, and H. Hinton, "StackGuard: Automatic Adaptive Detection and Prevention of Buffer-Overflow Attacks," in Proceedings of 7th USENIX Security Conference, San Antonio, Texas, Jan. 1998

Digital Library

[4]

Sandeep Bhatkar, Daniel C. DuVarney, and R. Sekar, "Address Obfuscation: An Efficient Approach to Combat a Broad Range of Memory Error Exploits," 12th USENIX Security Symposium, Washington, DC, August 2003.

Digital Library

[5]

Fu-Hau Hsu and Tzi-cker Chiueh, "CTCP: A Transparent Centralized TCP/IP Architecture for Network Security," Annual Computer Security Application Conference (ACSAC 2004), Tucson, Arizona, Dec., 2004.

Digital Library

[6]

D. Ditzel and R. McLellan., "Register Allocation for Free: The C Machine Stack Cache," Proc. of the Symp. on Architectural Support for Programming Languages and Operating Systems, pp. 48--56, March 1982.

Digital Library

[7]

Sangyeun Cho, Pen-Chung Yew, Gyungho Lee, "Decoupling local variable accesses in a wide-issue superscalar processor," Pro. of the 26th annual international symposium on Computer architecture, Georgia, United States, 1999.

Digital Library

[8]

Sandeep Grover, "Buffer Overflow Attacks and Their Countermeasures," Linux Journal, March 10, 2003

[9]

Ethereal: A Network Protocol Analyzer, www.ethereal.com

[10]

FastTrack Description, http://www.p2pwatchdog.com/packet_fasttrack.html

[11]

Manish Prasad, Tzi-cker Chiueh, "A Binary Rewriting Defense against Stack based Buffer Overflow Attacks," Usenix Annual Technical Conference, General Track, San Antonio, TX, June 2003

[12]

Fyodor, "Exploit world! Master Index for ALL Exploits," http://www.insecure.org/sploits_all.html

[13]

A. Pasupulati, J. Coit, K. Levitt, S.F. Wu, S.H. Li, R.C. Kuo, and K.P. Fan, "Buttercup: On Network-based Detection of Polymorphic Buffer Overflow Vulnerabilities," Network Operations and Management Symposium 2004(NOMS 2004).

[14]

Thomas Toth, Christopher Kruegel, "Accurate Buffer Overflow Detection via Abstract Payload Execution," Distributed Systems Group, Technical University Vienna, Austria, RAID 2002.

Digital Library

[15]

Stig Andersson, Andrew Clark, and George Mohay," Network-Based Buffer Overflow Detection by Exploit Code Analysis," AUSCERT 2004

[16]

M. Bernaschi, E. Gabrielli, and L. V. Mancini, "Operating system enhancements to prevent the misuse of system calls," Proceedings of the 7th ACM conference on Computer and Communications Security, 2000, Athens, Greece.

Digital Library

[17]

Matthew Smart, G. Robert Malan, Farnam Jahanian, "Defeating TCP/IP Stack Fingerprinting," USENIX Security Symposium, Aug. 2000.

Digital Library

[18]

Mark Handley, Vern Paxson, and Christian Kreibich, "Network Intrusion Detection: Evasion, Traffic Normalization, and End-to-End Protocol Semantics," Proc. USENIX Security Symposium 2001.

Digital Library

[19]

Vendicator, "Stack Shield," http://www.angelfire.com/sk/stackshield/

[20]

David Wagner, Jeffrey S. Foster, Eric A. Brewer, and Alexander Aiken, "A First Step Towards Automated Detection of Buffer Overrun Vulnerabilities," NDSS 2000.

[21]

CacheLogic, http://www.cachelogic.com/research/slide1.php

[22]

C. Kruegel, T. Toth, and E. Kirda, "Service Specific Anomaly Detection for Network Intrusion Detection," In Symposium on Applied Computing (SAC), Spain, March 2002.

Digital Library

[23]

Ke Wang and S. J. Stolfo, "Anomalous Payload-based Network Intrusion Detection,"Recent Advance in Intrusion Detection (RAID), Sept. 2005.

[24]

DilDog, "The Tao of Windows Buffer Overflow," http://www.cultdeadcow.com/cDc_files/cDc-351/index.html

Cited By

Yang GLiu XTang C(2022)Horus: An Effective and Reliable Framework for Code-Reuse Exploits Detection in Data StreamElectronics10.3390/electronics1120336311:20(3363)Online publication date: 18-Oct-2022
https://doi.org/10.3390/electronics11203363
Bronte RShahriar HHaddad H(2016)A Signature-Based Intrusion Detection System for Web Applications based on Genetic AlgorithmProceedings of the 9th International Conference on Security of Information and Networks10.1145/2947626.2951964(32-39)Online publication date: 20-Jul-2016
https://dl.acm.org/doi/10.1145/2947626.2951964
Hsu FChen LLin C(2015)Defeat scanning worms in cyber warfareSecurity and Communication Networks10.1002/sec.10198:5(715-726)Online publication date: 25-Mar-2015
https://dl.acm.org/doi/10.1002/sec.1019
Show More Cited By

Index Terms

Scalable network-based buffer overflow attack detection
1. Security and privacy
  1. Network security

Recommendations

SigFree: A Signature-Free Buffer Overflow Attack Blocker

We propose SigFree, an online signature-free out-of-the-box application-layer method for blocking code-injection buffer overflow attack messages targeting at various Internet services such as web service. Motivated by the observation that buffer ...
Building intrusion pattern miner for Snort network intrusion detection system

In this paper, we enhance the functionalities of Snort network-based intrusion detection system to automatically generate patterns of misuse from attack data, and the ability of detecting sequential intrusion behaviors. To that, we implement an ...
Detecting Return-to-libc Buffer Overflow Attacks Using Network Intrusion Detection Systems
ICDS '10: Proceedings of the 2010 Fourth International Conference on Digital Society

There has been a significant amount of research recently into methods of protecting systems from buffer overflow attacks by detecting stack injected shell code. The majority of the research focuses on developing algorithms or signatures for detecting ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ANCS '06: Proceedings of the 2006 ACM/IEEE symposium on Architecture for networking and communications systems

December 2006

202 pages

ISBN:1595935800

DOI:10.1145/1185347

General Chair:
Laxmi N. Bhuyan
University of California, Riverside, USA
,
Program Chairs:
Michel Dubois
University of Southern California, USA
,
Will Eatherton
Cisco, USA

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 December 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ANCS06

Sponsor:

ANCS06: Symposium on Architecture for Networking and Communications Systems

December 3 - 5, 2006

California, San Jose, USA

Acceptance Rates

Overall Acceptance Rate 88 of 314 submissions, 28%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
1,088
Total Downloads

Downloads (Last 12 months)16
Downloads (Last 6 weeks)0

Reflects downloads up to 18 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yang GLiu XTang C(2022)Horus: An Effective and Reliable Framework for Code-Reuse Exploits Detection in Data StreamElectronics10.3390/electronics1120336311:20(3363)Online publication date: 18-Oct-2022
https://doi.org/10.3390/electronics11203363
Bronte RShahriar HHaddad H(2016)A Signature-Based Intrusion Detection System for Web Applications based on Genetic AlgorithmProceedings of the 9th International Conference on Security of Information and Networks10.1145/2947626.2951964(32-39)Online publication date: 20-Jul-2016
https://dl.acm.org/doi/10.1145/2947626.2951964
Hsu FChen LLin C(2015)Defeat scanning worms in cyber warfareSecurity and Communication Networks10.1002/sec.10198:5(715-726)Online publication date: 25-Mar-2015
https://dl.acm.org/doi/10.1002/sec.1019
da Silva FMoura DGaldino J(2014)Security Issues in Tactical Software-Defined RadiosAdvancing Embedded Systems and Real-Time Communications with Emerging Technologies10.4018/978-1-4666-6034-2.ch002(22-53)Online publication date: 2014
https://doi.org/10.4018/978-1-4666-6034-2.ch002
Peng Wu Wolf T(2014)Stack protection in packet processing systems2014 International Conference on Computing, Networking and Communications (ICNC)10.1109/ICCNC.2014.6785304(53-57)Online publication date: Feb-2014
https://doi.org/10.1109/ICCNC.2014.6785304
da Silva FMoura DGaldino J(2012)Classes of Attacks for Tactical Software Defined RadiosInternational Journal of Embedded and Real-Time Communication Systems10.4018/jertcs.20121001043:4(57-82)Online publication date: 1-Oct-2012
https://doi.org/10.4018/jertcs.2012100104
Drozd MBarabas MGregr MChmelar P(2011)Buffer overflow attacks data acquisitionProceedings of the 6th IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems10.1109/IDAACS.2011.6072875(775-779)Online publication date: Sep-2011
https://doi.org/10.1109/IDAACS.2011.6072875
Chan YAkmayeva GShoniregun C(2010)A novel approach against the system buffer overflowInternational Journal of Internet Technology and Secured Transactions10.1504/IJITST.2010.0314712:1/2(32-58)Online publication date: 1-Feb-2010
https://dl.acm.org/doi/10.1504/IJITST.2010.031471
Chen GCheung WChu S(2009)Third Party E-Service and Third Party Service-Oriented Architecture2009 International Conference on E-Business and Information System Security10.1109/EBISS.2009.5138135(1-5)Online publication date: May-2009
https://doi.org/10.1109/EBISS.2009.5138135
Chen TZhang XLiu Z(2009)A Hybrid Detection Approach for Zero-Day Polymorphic Shellcodes2009 International Conference on E-Business and Information System Security10.1109/EBISS.2009.5137874(1-5)Online publication date: May-2009
https://doi.org/10.1109/EBISS.2009.5137874

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten