Engineering with logic: HOL specification and symbolic-evaluation testing for TCP implementations

Abstract

The TCP/IP protocols and Sockets API underlie much of modern computation, but their semantics have historically been very complex and ill-defined. The real standard is the de facto one of the common implementations, including, for example, the 15,000--20,000 lines of C in the BSD implementation. Dealing rigorously with the behaviour of such bodies of code is challenging.We have recently developed a post-hoc specification of TCP, UDP, and Sockets that is rigorous, detailed, readable, has broad coverage, and is remarkably accurate. In this paper we describe the novel techniques that were required.Working within a general-purpose proof assistant (HOL), we developed language idioms (within higher-order logic) in which to write the specification: operational semantics with nondeterminism, time, system calls, monadic relational programming, etc. We followed an experimental semantics approach, validating the specification against several thousand traces captured from three implementations (FreeBSD, Linux, and WinXP). Many differences between these were identified, and a number of bugs. Validation was done using a special-purpose symbolic model checker programmed above HOL.We suggest that similar logic engineering techniques could be applied to future critical software infrastructure at design time, leading to cleaner designs and (via specification-based testing using a similar checker) more predictable implementations.

References

1. R. Alur and B.-Y. Wang. Verifying network protocol implementations by symbolic refinement checking. In Proc. CAV '01, LNCS 2102, pages 169--181, 2001.]]
2. K. Bhargavan, S. Chandra, P. J. McCann, and C. A. Gunter. What packets may come: automata for network monitoring. In Proc. POPL, pages 206--219, 2001.]]
3. K. Bhargavan, D. Obradovic, and C. A. Gunter. Formal verification of standards for distance vector routing protocols. J. ACM, 49(4):538--576, 2002.]]
4. E. Biagioni. A structured TCP in Standard ML. In Proc. SIGCOMM '94, pages 36--45, 1994.]]
5. S. Bishop, M. Fairbairn, M. Norrish, P. Sewell, M. Smith, and K. Wansbrough. Rigorous specification and conformance testing techniques for network protocols, as applied to TCP, UDP, and Sockets. In Proc. SIGCOMM 2005 (Philadelphia), Aug. 2005.]]
6. S. Bishop, M. Fairbairn, M. Norrish, P. Sewell, M. Smith, and K. Wansbrough. TCP, UDP, and Sockets: rigorous and experimentally-validated behavioural specification. Volume 1: Overview. Technical Report UCAM-CL-TR-624, Computer Laboratory, University of Cambridge, Mar. 2005. 88pp. Available at http://www.cl.cam.ac.uk/users/pes20/Netsem/.]]
7. S. Bishop, M. Fairbairn, M. Norrish, P. Sewell, M. Smith, and K. Wansbrough. TCP, UDP, and Sockets: rigorous and experimentally-validated behavioural specification. Volume 2: The specification. Technical Report UCAM-CL-TR-625, Computer Laboratory, University of Cambridge, Mar. 2005. 386pp. Available at http://www.cl.cam.ac.uk/users/pes20/Netsem/.]]
8. C. Castelluccia, W. Dabbous, and S. O'Malley. Generating efficient protocol code from an abstract specification. IEEE/ACM Trans. Netw., 5(4):514--524, 1997. Full version of a paper in SIGCOMM '96.]]
9. M. Compton. Stenning's protocol implemented in UDP and verified in Isabelle. In Proc. 11th CATS, Computing: The Australasian Theory Symposium, pages 21--30, 2005.]]
10. M. Gordon, R. Milner, and C. P. Wadsworth. Edinburgh LCF, LNCS 78. 1979.]]
11. M. J. C. Gordon and T. Melham, editors. Introduction to HOL: a theorem proving environment. Cambridge University Press, 1993.]]
12. The HOL 4 system, Kananaskis-3 release. hol.sourceforge.net.]]
13. IEEE and The Open Group. IEEE Std 1003.1™-2001 Standard for Information Technology --- Portable Operating System Interface (POSIX®). Dec. 2001. Issue 6. Available http://www.opengroup.org/onlinepubs/007904975/toc.htm.]]
14. The Isabelle proof assistant. http://isabelle.in.tum.de/.]]
15. E. Kohler, M. F. Kaashoek, and D. R. Montgomery. A readable TCP in the Prolac protocol language. In Proc. SIGGCOMM '99, pages 3--13, August 1999.]]
16. N. Lynch and F. Vaandrager. Forward and backward simulations -- Part II: Timing-based systems. Information and Computation, 128(1):1--25, July 1996.]]
17. M. Musuvathi and D. Engler. Model checking large network protocol implementations. In Proc.NSDI: 1st Symposium on Networked Systems Design and Implementation, pages 155--168, 2004.]]
18. M. Norrish, P. Sewell, and K. Wansbrough. Rigour is good for you, and feasible: reflections on formal treatments of C and UDP sockets. In Proceedings of the 10th ACM SIGOPS European Workshop (Saint-Emilion), pages 49--53, Sept. 2002.]]
19. V. Paxson. Automated packet trace analysis of TCP implementations. In Proc. SIGCOMM '97, pages 167--179, 1997.]]
20. A. Serjantov, P. Sewell, and K. Wansbrough. The UDP calculus: Rigorous semantics for real networking. Technical Report 515, Computer Laboratory, University of Cambridge, July 2001.]]
21. A. Serjantov, P. Sewell, and K. Wansbrough. The UDP calculus: Rigorous semantics for real networking. In Proc. TACS 2001: Fourth International Symposium on Theoretical Aspects of Computer Software, Tohoku University, Sendai, Oct. 2001.]]
22. M. A. Smith and K. K. Ramakrishnan. Formal specification and verification of safety and performance of TCP selective acknowledgment. IEEE/ACM Trans. Netw., 10(2):193--207, 2002.]]
23. M. A. S. Smith. Formal verification of communication protocols. In Proc. FORTE IX/PSTV XVI, pages 129--144, 1996.]]
24. W. R. Stevens. TCP/IP Illustrated Vol. 1: The Protocols. 1994.]]
25. W. R. Stevens. UNIX Network Programming Vol. 1: Networking APIs: Sockets and XTI. Second edition, 1998.]]
26. K. Wansbrough, M. Norrish, P. Sewell, and A. Serjantov. Timing UDP: mechanized semantics for sockets, threads and failures. In Proc. ESOP, LNCS 2305, pages 278--294, Apr. 2002.]]
27. G. K. Wright and W. R. Stevens. TCP state transition diagram. InphTCP/IP Illustrated, Volume 2: The Implementation http://www.kohala.com/start/pocketguide1.ps.]]
28. G. R. Wright and W. R. Stevens. TCP/IP Illustrated Vol. 2: The Implementation. 1995.]]
29. W. Yi. CCS + time = an interleaving model for real time systems. In Proc. ICALP, pages 217--228, 1991.]]

Index Terms

• General and reference

  • Document types

    • Computing standards, RFCs and guidelines

• Networks

  • Network protocols

• Theory of computation

  • Logic

  • Semantics and reasoning

    • Program reasoning