SCEMIT: a systemc error and mutation injection tool

ABSTRACT

As high-level models in C and SystemC are increasingly used for verification and even design (through high-level synthesis) of electronic systems, there is a growing need for compatible error injection tools to facilitate further development of coverage metrics and automated diagnosis. This paper introduces SCEMIT, a tool for the automated injection of errors into C/C++/SystemC models. A selection of 'mutation' style errors are supported, and injection is performed though a plugin interface in the GCC compiler, which minimizes the impact of SCEMIT on existing simulation flows. Experimental injected error detection results are presented for the set of OSCI SystemC Example Models as well as the CHStone C High-Level-Synthesis benchmark set. Aside from demonstrating compatibility with these models, the results show the value of high-level error injection as a coverage measure compared to conventional code coverage measures.

References

1. G. Martin and G. Smith. High-level synthesis: Past, present, and future. IEEE Des. Test, 26(4):18--25, 2009.
2. M. E. Delamaro and J. C. Maldonado. Proteum/im 2.0: An integrated mutation testing environment. pages 91--101, 2001.
3. M. Hampton and S. Petithomme. Leveraging a commercial mutation analysis tool for research. In TAICPART-MUTATION '07: Proceedings of the Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION, pages 203--209, Washington, DC, USA, 2007. IEEE Computer Society.
4. N. Bombieri, F. Fummi, and G. Pravadelli. A mutation model for the systemc tlm 2.0 communication interfaces. In DATE '08: Proceedings of the conference on Design, automation and test in Europe, pages 396--401, New York, NY, USA, 2008. ACM.
5. G. Beltrame, C. Bolchini, L. Fossati, A. Miele, and D. Sciuto. Resp: a non-intrusive transaction-level reflective mpsoc simulation platform for design space exploration. In ASP-DAC '08: Proceedings of the 2008 Asia and South Pacific Design Automation Conference, pages 673--678, Los Alamitos, CA, USA, 2008. IEEE Computer Society Press.
6. Gcc, the gnu compiler collection. http://gcc.gnu.org/.
7. Open systemc initiative (osci). http://www.systemc.org/.
8. T. A. Budd, R. A. DeMillo, R. J. Lipton, and F. G. Sayward. Theoretical and empirical studies on using program mutation to test the functional correctness of programs. In POPL '80: Proceedings of the 7th ACM SIGPLAN-SIGACT symposium on Principles of programming languages, pages 220--233, New York, NY, USA, 1980. ACM.
9. T. Lv, J. Fan, and X. Li. An efficient observability evaluation algorithm based on factored use-def chains. In Test Symposium, 2003. ATS 2003. 12th Asian, pages 161--166, Nov. 2003.
10. F. Fallah, S. Devadas, and K. Keutzer. Occom: efficient computation of observability-based code coverage metrics for functional verification. In DAC '98: Proceedings of the 35th annual Design Automation Conference, pages 152--157, New York, NY, USA, 1998. ACM.
11. A. J. Offutt, A. Lee, G. Rothermel, R. H. Untch, and C. Zapf. An experimental determination of sufficient mutant operators. ACM Trans. Softw. Eng. Methodol., 5(2):99--118, 1996.
12. H. Agarwal et. al. Design of mutant operators for the c programming language. Technical report, Purdue University, 1989.
13. Plextest. http://www.itregister.com.au/products/plextest.htm.
14. Y. Hara, H. Tomiyama, S. Honda, H. Takada, and K. Ishii. Chstone: A benchmark program suite for practical c-based high-level synthesis. In Circuits and Systems, 2008. ISCAS 2008. IEEE International Symposium on, pages 1192--1195, May 2008.
15. R. Hierons, M. Harman, and S. Danicic. Using program slicing to assist in the detection of equivalent mutants. Software Testing, Verification and Reliability, 9:233--262, 1999.
16. A. J. Offutt and J. Pan. Automatically detecting equivalent mutants and infeasible paths. Software Testing, Veri and Reliability, 7:165--192, 1997.

Index Terms

• Hardware

  • Hardware validation

    • Functional verification