Testing malware detectors

ABSTRACT

In today's interconnected world, malware, such as worms and viruses, can cause havoc. A malware detector (commonly known as virus scanner) attempts to identify malware. In spite of the importance of malware detectors, there is a dearth of testing techniques for evaluating them. We present a technique based on program obfuscation for generating tests for malware detectors. Our technique is geared towards evaluating the resilience of malware detectors to various obfuscation transformations commonly used by hackers to disguise malware. We also demonstrate that a hacker can leverage a malware detector's weakness in handling obfuscation transformations and can extract the signature used by a detector for a specific malware. We evaluate three widely-used commercial virus scanners using our techniques and discover that the resilience of these scanners to various obfuscations is very poor.

REFERENCES

• 1: Dana Angluin, Learning regular sets from queries and counterexamples, Information and Computation, v.75 n.2, p.87-106, November 1, 1987 [doi>10.1016/0890-5401(87)90052-6]
• 2: K. Brunnstein. "Heureka-2" AntiVirus Tests. Virus Test Center, University of Hamburg, Computer Science Department, Mar. 2002. Published online at http://agn-www.informatik.uni-hamburg.de/vtc/en0203.htm. Last accessed: 16 Jan. 2004.
• 3: T. Y. Chen , Y. T. Yu, On the Relationship Between Partition and Random Testing, IEEE Transactions on Software Engineering, v.20 n.12, p.977-980, December 1994 [doi>10.1109/32.368132 ]
• 4: Stanley Chow , Yuan Gu , Harold Johnson , Vladimir A. Zakharov, An Approach to the Obfuscation of Control-Flow of Sequential Computer Programs, Proceedings of the 4th International Conference on Information Security, p.144-155, October 01-03, 2001
• 5: C. Collberg, C. Thomborson, and D. Low. A taxonomy of obfuscating transformations. Technical Report 148, Department of Computer Science, University of Auckland, New Zealand, July 1997.
• 6: Christian Collberg , Clark Thomborson , Douglas Low, Breaking Abstractions and Unstructuring Data Structures, Proceedings of the 1998 International Conference on Computer Languages, p.28, May 14-16, 1998
• 7: Christian Collberg , Clark Thomborson , Douglas Low, Manufacturing cheap, resilient, and stealthy opaque constructs, Proceedings of the 25th ACM SIGPLAN-SIGACT symposium on Principles of programming languages, p.184-196, January 19-21, 1998, San Diego, California, United States [doi>10.1145/268946.268962]
• 8: Dennis W. Cooper, Adaptive testing, Proceedings of the 2nd international conference on Software engineering, p.102-105, October 13-15, 1976, San Francisco, California, United States
• 9: T. Detristan, T. Ulenspiegel, Y. Malcom, and M. S. von Underduk. Polymorphic shellcode engine using spectrum analysis. Phrack, 11(61), Aug. 2003. Published online at http://www.phrack.org. Last accessed: 16 Jan. 2004.
• 10: J. W. Duran and S. C. Ntafos. An evaluation of random testing. IEEE Transactions on Software Engineering, 10(7):438--444, July 1984.
• 11: J. E. Forrester and B. P. Miller. An empirical study of the robustness of Windows NT applications using random testing. In Proceedings of the 4th USENIX Windows Systems Symposium, pages 59--68, Seattle, WA, USA, Aug. 2000.
• 12: Phyllis Frankl , Dick Hamlet , Bev Littlewood , Lorenzo Strigini, Choosing a testing method to deliver reliability, Proceedings of the 19th international conference on Software engineering, p.68-78, May 17-23, 1997, Boston, Massachusetts, United States [doi>10.1145/253228.253244]
• 13: S. Gordon and R. Ford. Real world anti-virus product reviews and evaluations -- the current state of affairs. In Proceedings of the 19th National Information Systems Security Conference (NISSC'96), pages 526--538, Baltimore, MD, USA, Oct. 1996. National Institute of Standards and Technology (NIST).
• 14: Dick Hamlet , Ross Taylor, Partition Testing Does Not Inspire Confidence (Program Testing), IEEE Transactions on Software Engineering, v.16 n.12, p.1402-1411, December 1990 [doi>10.1109/32.62448 ]
• 15: Ralf Hildebrandt , Andreas Zeller, Simplifying failure-inducing input, Proceedings of the 2000 ACM SIGSOFT international symposium on Software testing and analysis, p.135-145, August 21-24, 2000, Portland, Oregon, United States
• 16: Andreas Zeller , Ralf Hildebrandt, Simplifying and Isolating Failure-Inducing Input, IEEE Transactions on Software Engineering, v.28 n.2, p.183-200, February 2002 [doi>10.1109/32.988498 ]
• 17: ICSA Labs. Anti-virus product certification. Published online at http://www.icsalabs.com/html/communities/antivirus/certification.shtml. Last accessed: 16 Jan. 2004.
• 18: E. Kaspersky. Virus List Encyclopedia, chapter Ways of Infection: Viruses without an Entry Point. Kaspersky Labs, 2002.
• 19: LURHQ Threat Intelligence Group. Sobig.a and the spam you received today. Technical report, LURHQ, 2003. Published online at http://www.lurhq.com/sobig.html. Last accessed on 16 Jan. 2004.
• 20: LURHQ Threat Intelligence Group. Sobig.e -Evolution of the worm. Technical report, LURHQ, 2003. Published online at http://www.lurhq.com/sobig-e.html. Last accessed on 16 Jan. 2004.
• 21: LURHQ Threat Intelligence Group. Sobig.f examined. Technical report, LURHQ, 2003. Published online at http://www.lurhq.com/sobig-f.html. Last accessed on 16 Jan. 2004.
• 22: A. Marinescu. Russian doll. Virus Bulletin, pages 7--9, Aug. 2003.
• 23: A. Marx. A guideline to anti-malware-software testing. In Proceedings of the 9th Annual European Institute for Computer Antivirus Research Conference (EICAR'00), 2000.
• 24: A. Marx. Retrospective testing -- how good heuristics really work. In Proceedings of the 2002 Virus Bulletin Conference (VB2002), New Orleans, LA, USA, Sept. 2002. Virus Bulletin.
• 25: McAfee AVERT. Virus information library. Published online at http://us.mcafee.com/virusInfo/default.asp. Last accessed: 16 Jan. 2004.
• 26: Gary McGraw , Greg Morrisett, Attacking Malicious Code: A Report to the Infosec Research Council, IEEE Software, v.17 n.5, p.33-41, September 2000 [doi>10.1109/52.877857 ]
• 27: Barton P. Miller , Louis Fredriksen , Bryan So, An empirical study of the reliability of UNIX utilities, Communications of the ACM, v.33 n.12, p.32-44, Dec. 1990 [doi>10.1145/96267.96279]
• 28: B. P. Miller, D. Koski, C. P. Lee, V. Maganty, R. Murthy, A. Natarajan, and J. Steidl. Fuzz revisited: A re-examination of the reliability of UNIX utilities and services. Technical Report 1268, University of Wisconsin, Madison, Computer Sciences Department, Madison, WI, USA, Apr. 1995.
• 29: Glenford J. Myers, Art of Software Testing, John Wiley & Sons, Inc., New York, NY, 1979
• 30: Simeon Ntafos, On random and partition testing, Proceedings of the 1998 ACM SIGSOFT international symposium on Software testing and analysis, p.42-48, March 02-04, 1998, Clearwater Beach, Florida, United States
• 31: Simeon C. Ntafos, On Comparisons of Random, Partition, and Proportional Partition Testing, IEEE Transactions on Software Engineering, v.27 n.10, p.949-960, October 2001 [doi>10.1109/32.962563 ]
• 32: Symantec Antivirus Research Center. Expanded threat list and virus encyclopedia. Published online at http://securityresponse.symantec.com/avcenter/venc/data/cih.html. Last accessed: 16 Jan. 2004.
• 33: P. Ször and P. Ferrie. Hunting for metamorphic. In Proceedings of 2001 Virus Bulletin Conference (VB2001), pages 123--144, September 2001.
• 34: TESO. Burneye ELF encryption program. Published online at http://teso.scene.at. Last accessed: 15 Jan. 2004.
• 35: The WildList Organization International. Frequently asked questions. Published online at http://www.wildlist.org/faq.htm. Last accessed: 16 Jan. 2004.
• 36: Virus Bulletin. VB 100% Award. Published online at http://www.virusbtn.com/vb100/about/100use.xml. Last accessed: 16 Jan. 2004.
• 37: Chenxi Wang , John Knight, A security architecture for survivability mechanisms, 2001
• 38: West Coast Labs. Anti-virus Checkmark level 2. Published online at http://www.check-mark.com/checkmark/pdf/Checkmark_AV1.pdf. Last accessed: 16 Jan. 2004.
• 39: West Coast Labs. Anti-virus Checkmark level 2. Published online at http://www.check-mark.com/checkmark/pdf/Checkmark_AV2.pdf. Last accessed: 16 Jan. 2004.
• 40: Elaine J. Weyuker , Bingchiang Jeng, Analyzing Partition Testing Strategies, IEEE Transactions on Software Engineering, v.17 n.7, p.703-711, July 1991 [doi>10.1109/32.83906 ]
• 41: G. Wroblewski. General method of program code obfuscation. PhD thesis, Institute of Engineering Cybernetics, Wroclaw University of Technology, Wroclaw, Poland, 2002.
• 42: z0mbie. Automated reverse engineering: Mistfall engine. Published online at http://z0mbie.host.sk/autorev.txt. Last accessed: 16 Jan. 2004.
• 43: z0mbie. z0mbie's homepage. Published online at http://z0mbie.host.sk. Last accessed: 16 Jan. 2004.

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging
  D.4 OPERATING SYSTEMS
      D.4.6 Security and Protection

General Terms:
Algorithms, Experimentation, Measurement, Security

Keywords:
adaptive testing, anti-virus, malware, obfuscation