Integrated system interoperability testing with applications to VoIP

Abstract

This work has been motivated by the need to test interoperability of systems carrying voice calls over the IP network. The voice over IP (VoIP) systems must be integrated and interoperate with the existing public switched telephone network (PSTN) before they are widely adopted. Standards have been developed to address this problem, but unfortunately different standards bodies and commercial consortiums have defined different standards. Furthermore, the prevailing VoIP standard such as H.323 is incomplete, complex, and presents the implementers with "vendors latitudes." As a result, there is no guarantee that the integrated VoIP systems would interoperate properly even if the implementations are all H.323-compliant. Thus interoperability testing has become indispensable.

We want to test all the system interoperations by exercising all the required patterns of "interoperating behaviors." On the other hand, test execution in real environment is expensive, and we want to minimize the number of tests while maintaining the coverage.

We present a general method for automatic generation of test cases, which cover all the required system interoperations and contain a minimal number of tests. We study data structures and efficient test generation algorithms, which take time proportional to the total test case size. Finally, we report experimental results on VoIP systems.

References

1. {1} T. Chow, "Testing software design modeled by finite-state machines," IEEE Trans. Software Eng., vol. 4, pp. 178-187, Mar. 1978.
2. {2} N. A. Lynch, Distributed Algorithms. San Mateo, CA: Morgan Kaufmann, 1996.
3. {3} D. Sindu, "Protocol testing: The first ten years, the next ten years," Protocol Specification, Testing and Verification, pp. 44-68, 1990.
4. {4} S. Kang, J. Shin, and M. Kim, "Interoperability test suite derivation for communication protocols," Comput. Networks, vol. 32, no. 3, pp. 347-364, 2000.
5. {5} R. Hao, "Protocol conformance and interoperability testing based on formal methods," Ph.D. dissertation, Tsinghua University, Beijing, China, 1997.
6. {6} J. Gadre, C. Rohrer, C. Summers, and S. Symington, "A COS study of OSI interoperability," Comput. Standards Interfaces, vol. 9, no. 3, pp. 217-237, 1990.
7. {7} G. Bonnes, "IBM OSI interoperability verification services," presented at the IFIP TC6 WG6.1 3rd Int. Workshop Protocol Test System, McLean, VA, 1990.
8. {8} G. S. Vermeer and H. Blik, "Interoperability testing: Basis for the acceptance of communicating systems," in Protocol Test Systems, VI(C-19). Amsterdam, The Netherlands: Elsevier, 1994.
9. {9} O. Rafiq and R. Castanet, "From conformance testing to interoperability testing," in Proc. 3rd Int. Workshop Protocol Test Systems, 1990, pp. 371-385.
10. {10} S. Kang and M. Kim, "Test sequence generation for adaptive interoperability testing," in Proc. Protocol Test Systems, VIII, 1995, pp. 187-200.
11. {11} S. Kang and M. Kim, "Interoperability test suite derivation for symmetric communication protocols," in Proc. FORTE/PSTV'97, 1997.
12. {12} N. Arakawa and T. Soneoka, "A test case generation method for concurrent programs," in Protocol Test Systems, IV, R. H. J. Kroon and E. Brinksma, Eds. Amsterdam, The Netherlands: Elsevier, 1992, pp. 95-106.
13. {13} G. Luo, G. Bochmann, and A. Petrenko, "Test selection based on communicating nondeterministic finite-state machines using a generalized wp-method," IEEE Trans. Software Eng., vol. 20, pp. 149-162, Feb. 1994.
14. {14} R. Castanet and O. Kone, "Deriving coordinated testers for interoperability," in Protocol Test Systems, VI(C-19), R. H. J. Kroon and O. Rafiq, Eds. Amsterdam, The Netherlands: Elsevier, 1994, pp. 331-345.
15. {15} C. Viho, S. Barbin, and L. Tanguy, "Toward a formal framework for interoperability testing," in Proc. FORTE'01, 2001, pp. 51-68.
16. {16} D. Lee and M. Yannakakis, "Principles and methods of testing finite state machines--A survey," Proc. IEEE, vol. 84, pp. 1089-1123, Aug. 1996.
17. {17} G. J. Holzmann, Design and Validation of Computer Protocols. Englewood Cliffs, NJ: Prentice-Hall, 1991.
18. {18} A. Aho, J. E. Hopcroft, and J. D. Ullman, The Design and Analysis of Computer Algorithms. Reading, MA: Addison-Wesley, 1974.
19. {19} A. V. Aho, A. T. Dahbura, D. Lee, and M. U. Uyar, "An optimization technique for protocol conformance test generation based on UIO sequences and rural chinese postman tours," IEEE Trans. Commun., vol. 39, pp. 1604-15, Nov. 1991.
20. {20} "Switching Systems Generic Requirements: Call Processing," LATA, Bellcore, Tech. Rep. GR-505-CORE, 1997.
21. {21} "H.323 Packet-based multimedia communications systems," ITU-T, 2000.
22. {22} "H.225.0 Call signalling protocols and media stream packetization for packet-based multimedia communication systems," ITU-T, 2001.
23. {23} (2001) Standards-Based IP Telephony Interoperability Profile Proposal. IMTC. {Online}. Available: www.imtc.org

Index Terms

• Computer systems organization

  • Dependable and fault-tolerant systems and networks

    • Availability

    • Maintainability and maintenance

    • Reliability

• General and reference

  • Cross-computing tools and techniques

    • Reliability

• Networks

  • Network protocols

• Software and its engineering

  • Software creation and management

    • Software development techniques

      • Object oriented development

  • Software notations and tools

    • Development frameworks and environments

      • Object oriented frameworks

  • Software organization and properties

    • Software system structures

      • Distributed systems organizing principles