Clean Application Compartmentalization with SOAAP

ABSTRACT

Application compartmentalization, a vulnerability mitigation technique employed in programs such as OpenSSH and the Chromium web browser, decomposes software into isolated components to limit privileges leaked or otherwise available to attackers. However, compartmentalizing applications -- and maintaining that compartmentalization -- is hindered by ad hoc methodologies and significantly increased programming effort. In practice, programmers stumble through (rather than overtly reason about) compartmentalization spaces of possible decompositions, unknowingly trading off correctness, security, complexity, and performance. We present a new conceptual framework embodied in an LLVM-based tool: the Security-Oriented Analysis of Application Programs (SOAAP) that allows programmers to reason about compartmentalization using source-code annotations (compartmentalization hypotheses). We demonstrate considerable benefit when creating new compartmentalizations for complex applications, and analyze existing compartmentalized applications to discover design faults and maintenance issues arising from application evolution.

References

1. Apple WebKit Vulnerability Statistics. http://www.cvedetails.com/product/10007/Apple-Webkit.html.
2. CVS commit: pkgsrc/security/openssh, Aug. 2015. http://mail-index.netbsd.org/pkgsrc-changes/2015/08/14/msg128305.html.
3. Anderson, J. Computer security technology planning study. Tech. Rep. ESD-TR-73--51, U.S. Air Force Electronic Systems Division, October 1972.
4. Anderson, R., Bond, M., Clulow, J., and Skorobogatov, S. Cryptographic processors-a survey. Proceedings of the IEEE 94, 2 (Feb 2006), 357--369.
5. Belay, A., Bittau, A., Mashtizadeh, A., Terei, D., Mazières, D., and Kozyrakis, C. Dune: safe user-level access to privileged CPU features. In Proceedings of the 10th Conference on Operating Systems Design and Implementation (2012), USENIX.
6. Beurdouche, B., Bhargavan, K., Delignat-Lavaud, A., Fournet, C., Kohlweiss, M., Pironti, A., Strub, P.-Y., and Zinzindohoue, J. K. A Messy State of the Union: Taming the Composite State Machines of TLS. In Proceedings of the IEEE Symposium on Security and Privacy (2015).
7. Bittau, A., Marchenko, P., Handley, M., and Karp, B. Wedge: Splitting Applications into Reduced-Privilege Compartments. In Proceedings of the 5th USENIX Symposium on Networked Systems Design and Implementation (2008), USENIX.
8. Boebert, W., and Kain, R. A practical alternative to hierarchical integrity policies. In Proceedings of the Eighth DoD/NBS Computer Security Initiative Conference (1985).
9. Brumley, D., and Song, D. Privtrans: automatically partitioning programs for privilege separation. In Proceedings of the 13th USENIX Security Symposium (2004), USENIX.
10. Dean, J., Grove, D., and Chambers, C. Optimization of object-oriented programs using static class hierarchy analysis. In Proceedings of the 9th European Conference on Object-Oriented Programming (1995), ECOOP '95, Springer-Verlag.
11. Dennis, J. B., and Van Horn, E. C. Programming semantics for multiprogrammed computations. Commun. ACM 9, 3 (1966), 143--155.
12. Dolan, S., Williams, N., Tolnay, D., Lapresta, S., Langford, W., and Gordan, A.progjq: a lightweight and flexible command-line JSON processor. http://stedolan.github.io/jq/.
13. Gong, L., Mueller, M., Prafullchandra, H., and Schemers, R. Going beyond the sandbox: An overview of the new security architecture in the Java Development Kit 1.2. In Proceedings of the Symposium on Internet Technologies and Systems (1997), USENIX.
14. Gudka, K., Watson, R. N. M., Anderson, J., Chisnall, D., Davis, B., Laurie, B., Madhavapeddy, A., Marinos, I., Murdoch, S. J., Neumann, P. G., and Richardson, A. Clean Application Compartmentalization with SOAAP (extended version). Tech. Rep. UCAM-CL-TR-873, University of Cambridge Computer Laboratory, Sept. 2015.
15. Hamilton, G., and Kougiouris, P. The Spring Nucleus: A microkernel for objects. In Proceedings of the 1993 Summer Usenix Conference (1993), USENIX.
16. Harris, W. R., Farley, B., Jha, S., and Reps, T. Secure Programming as a Parity Game. Tech. Rep. 1694, University of Wisconsin Madison, July 2011.
17. Jurczyk, M., and Coldwind, G. FFmpeg and a thousand fixes. Google Online Security Blog, January 2014. http://googleonlinesecurity.blogspot.com/2014/01/ffmpeg-and-thousand-fixes.html.
18. Kamp, P., and Watson, R. N. M. Jails: Confining the omnipotent root. In Proceedings of the 2nd International SANE Conference (2000).
19. Karger, P. A. Limiting the damage potential of discretionary trojan horses. In Proceedings of the IEEE Symposium on Security and Privacy (1987), IEEE.
20. Kilpatrick, D. Privman: A Library for Partitioning Applications. In Proceedings of USENIX Annual Technical Conference (2003), USENIX.
21. Klein, G., Andronick, J., Elphinstone, K., Heiser, G., Cock, D., Derrin, P., Elkaduwe, D., Engelhardt, K., Kolanski, R., Norrish, M., Sewell, T., Tuch, H., and Winwood, S. seL4: Formal verification of an operating-system kernel. Commun. ACM 53 (June 2009), 107--115.
22. Lattner, C., and Adve, V. LLVM: A compilation framework for lifelong program analysis & transformation. In Proceedings of the International Symposium on Code Generation and Optimization (2004), CGO '04, IEEE.
23. Levin, R., Cohen, E., Corwin, W., Pollack, F., and Wulf, W. Policy/mechanism separation in Hydra. In SOSP '75: Proceedings of the fifth ACM Symposium on Operating Systems Principles (1975).
24. Lipner, S. B., Wulf, W. A., Schell, R. R., Popek, G. J., Neumann, P. G., Weissman, C., and Linden, T. A. Security kernels. In AFIPS '74: Proceedings of the May 6--10, 1974, National Computer Conference and Exposition (1974), ACM.
25. Loscocco, P., and Smalley, S. Integrating flexible support for security policies into the linux operating system. In Proceedings of the USENIX Annual Technical Conference (2001), USENIX.
26. McKusick, M. K., Neville-Neil, G. V., and Watson, R. N. M. The Design and Implementation of the FreeBSD Operating System. Pearson, 2014.
27. Mettler, A., Wagner, D., and Close, T. Joe-E: A Security-Oriented Subset of Java. In Proceedings of the Network and Distributed System Security Symposium (2010), NDSS'10.
28. Miller, M. S. Robust composition: towards a unified approach to access control and concurrency control. PhD thesis, Johns Hopkins University, Baltimore, MD, USA, 2006.
29. MITRE. CVE-2012-0652.
30. MITRE. CVE-2014-0160.
31. MITRE. CVE-2014--4877.
32. MITRE. CVE-2014--6271.
33. Murray, D. G., and Hand, S. Privilege separation made easy: trusting small libraries not big processes. In Proceedings of the 1st European Workshop on System Security (2008), EUROSEC '08, ACM.
34. Myers, A. C., and Liskov, B. A decentralized model for information flow control. SIGOPS Oper. Syst. Rev. 31 (October 1997), 129--142.
35. NIST. libpoppler CVEs. https://web.nvd.nist.gov/view/vuln/search-results?cpe_vendor=cpe:/:freedesktop&cpe_product=cpe:/::poppler.
36. Provos, N., Friedl, M., and Honeyman, P. Preventing privilege escalation. In Proceedings of the 12th USENIX Security Symposium (2003), USENIX.
37. Reis, C., and Gribble, S. D. Isolating web programs in modern browser architectures. In EuroSys '09: Proceedings of the 4th ACM European Conference on Computer Systems (2009), ACM.
38. Saltzer, J. H., and Schroeder, M. D. The protection of information in computer systems. Proceedings of the IEEE 63, 9 (September 1975), 1278--1308.
39. Teresa Johnson, X. D. L. ThinLTO: A Fine-Grained Demand-Driven Infrastructure. In EuroLLVM (2015).
40. Wagner, D., and Tribble, D. A security analysis of the combex darpabrowser architecture, March 2002.
41. Watson, R. N. M. A decade of OS access-control extensibility. Commun. ACM 56, 2 (Feb. 2013).
42. Watson, R. N. M., Anderson, J., Laurie, B., and Kennaway, K. Capsicum: Practical capabilities for UNIX. In Proceedings of the 19th USENIX Security Symposium (2010), USENIX.
43. Watson, R. N. M., Woodruff, J., Neumann, P. G., Moore, S. W., Anderson, J., Chisnall, D., Dave, N., Davis, B., Gudka, K., Laurie, B., Murdoch, S. J., Norton, R., Roe, M., Son, S., and Vadera, M. CHERI: A Hybrid Capability-System Architecture for Scalable Software Compartmentalization. In Proceedings of the 36th IEEE Symposium on Security and Privacy (2015).

Index Terms

• Security and privacy

  • Systems security

    • Operating systems security