Secure and practical defense against code-injection attacks using software dynamic translation

ABSTRACT

One of the most common forms of security attacks involves exploiting a vulnerability to inject malicious code into an executing application and then cause the injected code to be executed. A theoretically strong approach to defending against any type of code-injection attack is to create and use a process-specific instruction set that is created by a randomization algorithm. Code injected by an attacker who does not know the randomization key will be invalid for the randomized processor effectively thwarting the attack. This paper describes a secure and efficient implementation of instruction-set randomization (ISR) using software dynamic translation. The paper makes three contributions beyond previous work on ISR. First, we describe an implementation that uses a strong cipher algorithm--the Advanced Encryption Standard (AES), to perform randomization. AES is generally believed to be impervious to known attack methodologies. Second, we demonstrate that ISR using AES can be implemented practically and efficiently (considering both execution time and code size overheads) without requiring special hardware support. The third contribution is that our approach detects malicious code before it is executed. Previous approaches relied on probabilistic arguments that execution of non-randomized foreign code would eventually cause a fault or runtime exception.

REFERENCES

• 1: ABADI, M., BUDIU, M., ERLINGSSON, U., AND LIGATTI, J. Control-flow integrity. In Microsoft Research Technical Report MSF-TR-05-18 (2005).
• 2: BARRANTES, E. G., ACKLEY, D. H., FORREST, S., AND STEFANOVIC, D. Randomized instruction set emulation. ACM Transactions on Information System Security. 8, 1 (2005), 3--40.
• 3: BARRANTES, E. G., ACKLEY, D. H., PALMER, T. S., STEFANOVIC, D., AND ZOVI, D. D. Randomized instruction set emulation to disrupt binary code injection attacks. In CCS '03: Proceedings of the 10th ACM Conference on Computer and Communications Security (New York, NY, USA, 2003), ACM Press, pp. 281--289.
• 4: BUS, B. D., SUTTER, B. D., PUT, L. V., CHANET, D., AND BOSSCHERE, K. D. Link-time optimization of ARM binaries. ACM SIGPLAN Notices 39, 7 (July 2004), 211--220.
• 5: CHEN, S., XU, J., SEZER, E., GAURIAR, P., AND IYER, R. Non-control-data attacks are realistic threats. In Proceedings of the 14th USENIX Security Symposium (Berkeley, CA, USA, 2005), USENIX Association,pp. 177--192.
• 6: COWAN, C., BARRINGER, M., BEATTIE, S., AND KROAH-HARTMAN, G. Formatguard: Automatic protection from printf format string vulnerabilities. In Proceedings of the 10th USENIX Security Symposium (August 2001).
• 7: COWAN, C., BEATTIE, S., JOHANSEN, J., AND WAGLE, P. PointGuard: Protecting pointers from buffer overflow vulnerabilities. In Proceedings of the 12th USENIX Security Symposium (Aug. 2003), USENIX, pp. 91--104.
• 8: COWAN, C., PU, C., MAIER, D., HINTON, H., BAKKE, P., BEATTIE, S., GRIER, A., WAGLE, P., AND ZHANG, Q. Stackguard: Automatic adaptive detection and prevention of buffer-overflow attacks. In Proceedings of the 1998 USENIX Security Symposium (January 1998).
• 9: KC, G. S., KEROMYTIS, A. D., AND PREVELAKIS, V. Countering code-injection attacks with instruction-set randomization. In CCS '03: Proceedings of the 10th ACM Conference on Computer and Communications Security (New York, NY, USA, 2003), ACM Press, pp. 272--280.
• 10: KIRIANSKY, V., BRUENING, D., AND AMARASINGHE, S. P. Secure execution via program shepherding. In Proceedings of the 11th USENIX Security Symposium (Berkeley, CA, USA, 2002), USENIX Association, pp. 191--206.
• 11: KIROVSKI, D., DRINIC, M., AND POTKONJAK, M. Enabling trusted software integrity. In ASPLOS-X: Proceedings of the 10th International Conference on Architectural Support for Programming Languages and Operating Systems (New York, NY, USA, 2002), ACM Press, pp. 108--120.
• 12: KUMAR, N., AND CHILDERS, B. Flexible instrumentation for software dynamic translation. In Workshop on Exploring the Trace Space, International Conference on Supercomputing (2003).
• 13: KUPERMAN, B. A., BRODLEY, C. E., OZDOGANOGLU, H., VIJAYKUMAR, T. N., AND JALOTE, A. Detection and prevention of stack buffer overflow attacks. Communications of the ACM 48, 11 (2005), 50--56.
• 14: LAWTON, K. P. Bochs: A portable pc emulator for Unix/X. Linux J. 1996, 29es (1996), 7.
• 15: LUK, C.-K., COHN, R., MUTH, R., PATIL, H., KLAUSER, A., LOWNEY, G., WALLACE, S., REDDI, V. J., AND HAZELWOOD, K. Pin: building customized program analysis tools with dynamic instrumentation. In PLDI '05: Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation (New York, NY, USA, 2005), ACM Press, pp. 190--200.
• 16: MILENKOVIC, M., MILENKOVIC, A., AND JOVANOV, E. Using instruction block signatures to counter code injection attacks. SIGARCH Computer Architecture News 33, 1 (2005), 108--117.
• 17: NETHERCOTE, N. Dynamic binary analysis and instrumentation. Tech. Rep. UCAM-CL-TR-606, University of Cambridge, Computer Laboratory, Nov. 2004.
• 18: THE COMMITTEE ON NATIONAL SECURITY SYSTEMS, T. C. National policy on the use of the advanced encryption standard (AES) to protect national security systems and national security information. Tech. rep., National Security Agency, 2003.
• 19: PINCUS, J., AND BAKER, B. Beyond stack smashing: Recent advances in exploiting buffer overruns. IEEE Security & Privacy 2, 4 (July/August 2004), 20--27.
• 20: PRASAD, M., AND CHIUEH, T. A binary rewriting defense against stack-based buffer overflow attacks. In Proceedings of the 2003 USENIX Annual Technical Conference (June 2003), pp. 211--224.
• 21: SCOTT, K., AND DAVIDSON, J. Strata: A software dynamic translation infrastructure. In IEEE Workshop on Binary Translation (September 2001).
• 22: SCOTT, K., AND DAVIDSON, J. W. Safe virtual execution using software dynamic translation. In Proceedings of the 18th Annual Computer Security Applications Conference (Las Vegas, NV, December 2002), pp. 209--218.
• 23: SCOTT, K., KUMAR, N., VELUSAMY, S., CHILDERS, B. R., DAVIDSON, J. W., AND SOFFA, M. L. Retargetable and reconfigurable software dynamic translation. In International Symposium on Code Generation and Optimization (San Francisco, CA, Mar. 2003), IEEE Computer Society, pp. 36--47.
• 24: SHOGAN, S., AND CHILDERS, B. Compact binaries with code compression in a software dynamic translator. In Design Automation and Test in Europe (2004).
• 25: SOVAREL, N., EVANS, D., AND PAUL, N. Where's the feeb? the effectiveness of instruction set randomization. In Proceedings of the 14th USENIX Security Conference (2005).
• 26: THE PAX TEAM. http://pax.grsecurity.net.
• 27: THIMBLEBY, H. Can viruses ever be useful? Computers and Security 10, 2 (1991), 111--114.

INDEX TERMS

Primary Classification:
  D. Software
  D.3 PROGRAMMING LANGUAGES
      D.3.4 Processors
          Subjects: Interpreters

Additional Classification:
  D. Software
  D.3 PROGRAMMING LANGUAGES
      D.3.4 Processors
          Subjects: Run-time environments
  D.4 OPERATING SYSTEMS
      D.4.6 Security and Protection
          Subjects: Invasive software (e.g., viruses, worms, Trojan horses)

General Terms:
Security

Keywords:
software dynamic translation, virtual execution