skip to main content
10.1145/2490428.2490454acmconferencesArticle/Chapter ViewAbstractPublication PagessecuritConference Proceedingsconference-collections
poster

Obfuscated malware detection using API call dependency

Published:17 August 2012Publication History

ABSTRACT

Malwares pose a grave threat to security of a network and host systems. Many events such as Distributed Denial-of-Service attacks, spam emails etc., often have malwares as their root cause. So a great deal of research is being invested in detection and removal of malwares. Thus many malware detection systems or antivirus softwares have come up. But the drawback of these antivirus softwares is they rely upon signature matching approach for malware detection which can be easily defeated using simple code obfuscation techniques. This has given rise to a new generation of metamorphic and polymorphic malwares. In this paper we proposed the approach of monitoring interdependent system calls to detect obfuscated malicious programs. We took some sample malwares and some common obfuscation techniques. We tested these obfuscated malwares against open source antivirus ClamAV and our detection model. The results obtained have been elaborated further in the paper. Again how our algorithm is sound against many drawbacks of the API call monitoring approach such as API call reordering, garbage API call insertion etc., are also described.

References

  1. http://www.symantec.com/connect/blogs/w32stuxnet-installation-details.Google ScholarGoogle Scholar
  2. http://www.clamav.net/.Google ScholarGoogle Scholar
  3. http://www.datarescue.com/idabase/.Google ScholarGoogle Scholar
  4. http://www.ollydbg.de/.Google ScholarGoogle Scholar
  5. https://www.virtualbox.org/.Google ScholarGoogle Scholar
  6. http://vx.netlux.org/.Google ScholarGoogle Scholar
  7. http://msdn.microsoft.com/en-us/library/ms123401.aspx.Google ScholarGoogle Scholar
  8. D. Bruschi, L. Martignoni, and M. Monga. Detecting self-mutating malware using control flow graph matching. In PROCEEDINGS OF THE CONFERENCE ON DETECTION OF INTRUSIONS AND MALWARE AND VULNERABILITY ASSESSMENT (DIMVA), IEEE COMPUTER SOCIETY, pages 129--143, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. M. Christodorescu and S. Jha. Static analysis of executables to detect malicious patterns. In In Proceedings of the 12th USENIX Security Symposium, pages 169--186, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. M. Christodorescu, S. Jha, and C. Kruegel. Mining specifications of malicious behavior. In In Proceedings of the 6th joint meeting of the European Software Engineering Conference and the ACM SIGSOFT International Symposium on Foundations of Software Engineering, pages 5--14, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. M. Christodorescu, S. Jha, S. A. Seshia, D. X. Song, and R. E. Bryant. Semantics-aware malware detection. In IN IEEE SYMPOSIUM ON SECURITY AND PRIVACY, pages 32--46, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. C. Kolbitsch, P. M. Comparetti, C. Kruegel, E. Kirda, X. Zhou, and X. Wang. Effective and efficient malware detection at the end host.Google ScholarGoogle Scholar
  13. A. Lakhotia and E. U. Kumar. Abstract stack graph to detect obfuscated calls in binaries. In In Proc. 4th. IEEE International Workshop on Source Code Analysis and Manipulation, pages 17--26. IEEE Computer Society, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. J. Lee, K. Jeong, and H. Lee. Detecting metamorphic malwares using code graphs. In Proceedings of the 2010 ACM Symposium on Applied Computing, SAC '10, pages 1970--1977, New York, NY, USA, 2010. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. L. O. M. Nicolas Falliere and E. Chien. W32.stuxnet dossier. Technical report, Symantec, 2011.Google ScholarGoogle Scholar
  16. K. Rozinov. Reverse code engineering: An in-depth analysis of the bagle virus. Technical report, Bell Labs, 2004.Google ScholarGoogle Scholar
  17. P. Szűr and P. Ferrie. Hunting for metamorphic. In In Virus Bulletin Conference, pages 123--144, 2001.Google ScholarGoogle Scholar
  18. I. You and K. Yim. Malware obfuscation techniques: A brief survey. In Broadband, Wireless Computing, Communication and Applications (BWCCA), 2010 International Conference on, pages 297--300, nov. 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. z0mbie. http://z0mbie.host.sk.Google ScholarGoogle Scholar

Index Terms

  1. Obfuscated malware detection using API call dependency

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in
        • Published in

          cover image ACM Conferences
          SecurIT '12: Proceedings of the First International Conference on Security of Internet of Things
          August 2012
          266 pages
          ISBN:9781450318228
          DOI:10.1145/2490428

          Copyright © 2012 ACM

          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 17 August 2012

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • poster

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader