End-user software visualizations for fault localization

ABSTRACT

End-user programming has become the most common form of programming today. However, despite this growth, there has been little investigation into bringing the benefits of software visualization to end-user programmers. Evidence from the spreadsheet paradigm, probably the most widely used end-user environment, reveals that end users' programs often contain faults. We would like to integrate software visualization into these end-user environments to help end users deal with the reliability issues in their programs. Towards this end, we have devised several fault localization visualization techniques for spreadsheets. This paper describes these techniques and reports the results of a formative study---using tests created by end users---to investigate how these fault localization techniques compare. Our results reveal some strengths and weaknesses of each technique, and provide insights into the cost-effectiveness of each technique for the interactive world of end-user spreadsheet development.

REFERENCES

• 1: J. W. Atwood , M. M. Burnett , R. A. Walpole , E. M. Wilcox , S. Yang, Steering Programs Via Time Travel, Proceedings of the 1996 IEEE Symposium on Visual Languages, p.4, September 03-06, 1996
• 2: Barry W. Boehm , Clark , Horowitz , Brown , Reifer , Chulani , Ray Madachy , Bert Steece, Software Cost Estimation with Cocomo II with Cdrom, Prentice Hall PTR, Upper Saddle River, NJ, 2000
• 3: Margaret Burnett , John Atwood , Rebecca Walpole Djang , James Reichwein , Herkimer Gottfried , Sherry Yang, Forms/3: A first-order visual language to explore the boundaries of the spreadsheet paradigm, Journal of Functional Programming, v.11 n.2, p.155-206, March 2001
• 4: Margaret M. Burnett , Herkimer J. Gottfried, Graphical definitions: expanding spreadsheet languages through direct manipulation and gestures, ACM Transactions on Computer-Human Interaction (TOCHI), v.5 n.1, p.1-33, March 1998 [doi>10.1145/274444.274445]
• 5: T. Y. Chen , Y. Y. Cheung, On program dicing, Journal of Software Maintenance: Research and Practice, v.9 n.1, p.33-46, Jan.-Feb. 1997 [doi>10.1002/(SICI)1096-908X(199701)9:1<33::AID-SMR143>3.3.CO;2-W ]
• 6: DUESTERWALD, E., GUPTA, R., AND SOFFA, M.L. Rigorous data flow testing through output influences. 2nd Irvine Software Symp., Mar. 1992.
• 7: EICK, S. Maintenance of large systems. Software Visualization: Programming as a Multimedia Experience (J. Stasko, J. Domingue, M. Brown, and B. Price, eds.), MIT Press, Cambridge, MA, 1998, 315--328.
• 8: Marc Fisher , Mingming Cao , Gregg Rothermel , Curtis R. Cook , Margaret M. Burnett, Automated test case generation for spreadsheets, Proceedings of the 24th International Conference on Software Engineering, May 19-25, 2002, Orlando, Florida [doi>10.1145/581339.581359]
• 9: Marc Fisher II , Dalai Jin , Gregg Rothermel , Margaret Burnett, Test Reuse in the Spreadsheet Paradigm, Proceedings of the 13th International Symposium on Software Reliability Engineering (ISSRE'02), p.257, November 12-15, 2002
• 10: HEATH, M., MALONY, A., AND ROVER, D. Visualization for parallel performance evaluation and optimization. Software Visualization: Programming as a Multimedia Experience (J. Stasko, J. Domingue, M. Brown, and B. Price, eds.), MIT Press, Cambridge, MA, 1998, 347--365.
• 11: HEGER, N., CYPHER, A., AND SMITH, D. Cocoa at the visual programming challenge 1997. J. Visual Languages and Computing 9, 2, Apr. 1998, 151--169.
• 12: James A. Jones , Mary Jean Harrold , John Stasko, Visualization of test information to assist fault localization, Proceedings of the 24th International Conference on Software Engineering, May 19-25, 2002, Orlando, Florida [doi>10.1145/581339.581397]
• 13: KIMELMAN, D., ROSENBURG, B., AND ROTH, T. Visualization of dynamics in real world software systems. Software Visualization: Programming as a Multimedia Experience (J. Stasko, J. Domingue, M. Brown, and B. Price, eds.), MIT Press, Cambridge, MA, 1998, 293--314.
• 14: LASKI, J. AND KOREL, B. A data flow oriented program testing strategy. IEEE Trans. Soft. Eng. 9, 3, May 1993, 347--354.
• 15: LIEBERMAN, H. AND FRY, C. ZStep 95: A reversible, animated source code stepper. Software Visualization: Programming as a Multimedia Experience (J. Stasko, J. Domingue, M. Brown, and B. Price, eds.), MIT Press, Cambridge, MA, 1998, 277--292.
• 16: NOTKIN, D., ELLISON, R., KAISER, G., KANT, E., HABERMANN, A., AMBRIOLA, V., AND MONTANEGERO, C. Special issue on the GANDALF project. J. Systems and Software 5, 2, May 1985.
• 17: NTAFOS, S.C. On required element testing. IEEE Trans. Soft. Eng. 10, 6, Nov. 1984.
• 18: PAN, H., AND SPAFFORD, E. Toward automatic localization of software faults. 10th Pacific Northwest Software Quality Conference, Oct. 1992.
• 19: Raymond R. Panko, What we know about spreadsheet errors, Journal of End User Computing, v.10 n.2, p.15-21, Spring 1998
• 20: Sandra Rapps , Elaine J. Weyuker, Selecting software test data using data flow information, IEEE Transactions on Software Engineering, v.11 n.4, p.367-375, April 1985 [doi>10.1109/TSE.1985.232226]
• 21: James Reichwein , Gregg Rothermel , Margaret Burnett, Slicing spreadsheets: an integrated methodology for spreadsheet testing and debugging, Proceedings of the 2nd conference on Domain-specific languages, p.25-38, October 03-06, 1999, Austin, Texas, United States
• 22: Steven P. Reiss, Graphical program development with PECAN program development systems, Proceedings of the first ACM SIGSOFT/SIGPLAN software engineering symposium on Practical software development environments, p.30-41, April 1984
• 23: REISS, S. Visualization for software engineering---programming environments. Software Visualization: Programming as a Multimedia Experience (J. Stasko, J. Domingue, M. Brown, and B. Price, eds.), MIT Press, Cambridge, MA, 1998, 259--276.
• 24: Gregg Rothermel , Lixin Li , Christopher DuPuis , Margaret Burnett, What you see is what you test: a methodology for testing form-based visual programs, Proceedings of the 20th international conference on Software engineering, p.198-207, April 19-25, 1998, Kyoto, Japan
• 25: Tim Teitelbaum , Thomas Reps, The Cornell program synthesizer: a syntax-directed programming environment, Communications of the ACM, v.24 n.9, p.563-573, Sept. 1981 [doi>10.1145/358746.358755]
• 26: TELCORDIA TECHNOLOGIES, xSlice: A tool for program debugging. xsuds.argreenhouse.com/html-man/coverpage.html, July 1998.
• 27: TIP, F. A survey of program slicing techniques. J. Programming Languages 3, 3, 1995, 121--189.

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging

Additional Classification:
  D. Software
  D.1 PROGRAMMING TECHNIQUES
      D.1.7 Visual Programming
  D.2 SOFTWARE ENGINEERING
      D.2.6 Programming Environments
          Subjects: Interactive environments

  H. Information Systems
  H.4 INFORMATION SYSTEMS APPLICATIONS
      H.4.1 Office Automation
          Subjects: Spreadsheets

Keywords:
end-user programming, end-user software engineering, end-user software visualization, fault localization, spreadsheets