Multi-environment software testing on the grid

ABSTRACT

We propose a solution to improve the confidence on the correctness of applications designed to be executed in heterogeneous environments, like a grid. Our solution is motivated by the observation that the traditional ways to qualify test processes are based on code coverage metrics. We believe that this approach is not adequate when dealing with applications that can (and do) fail when interacting with heterogeneous execution environments. Besides code coverage, tests must also cover possible environments. As a solution we propose the utilization of InGriD to describe and deploy test environments and GridUnit to coordinate and monitor the execution of test sets. By combining these two solutions we provide a cost effective way to introduce environmental coverage to our test suites, which is complementary and orthogonal to traditional code coverage metrics. As a case study, we have shown how our solution could be applied to help testing a grid application called MyPhotoGrid, which uses the grid to parallelize the generation of large photograph albums.

REFERENCES

• 1: A. Brown, D. A. Patterson, To Err is Human, First Workshop on Evaluating and Architecting System Dependability (EASY '01)
• 2: A. Lastovetsky, Parallel Testing of Distributed Software, Information and Software Technology 47(10), pp.657--662, Elsevier, 2005
• 3: A. Memon , A. Porter , C. Yilmaz , A. Nagarajan , D. Schmidt , B. Natarajan, Skoll: Distributed Continuous Quality Assurance, Proceedings of the 26th International Conference on Software Engineering, p.459-468, May 23-28, 2004
• 4: Alexandre Duarte , Francisco Brasileiro , Walfredo Cirne , Jose Alencar Filho, Collaborative Fault Diagnosis in Grids through Automated Tests, Proceedings of the 20th International Conference on Advanced Information Networking and Applications - Volume 1 (AINA'06), p.69-74, April 18-20, 2006 [doi>10.1109/AINA.2006.127]
• 5: Alexandre Duarte , Walfredo Cirne , Francisco Brasileiro , Patricia Machado, GridUnit: software testing on the grid, Proceeding of the 28th international conference on Software engineering, May 20-28, 2006, Shanghai, China [doi>10.1145/1134285.1134410]
• 6: D. Paranhos, W. Cirne and F. Brasileiro, Trading Information for Cycles: Using Replication to Schedule Bag of Tasks Applications on Computational Grids, in: Proceedings of the Euro-Par 2003: International Conference on Parallel and Distributed Computing, 2003.
• 7: E. F. Miller, Program Testing Techniques, COMPSAC¿77 IEEE Computer Society, 1977.
• 8: E. Gamma, K. Beck. JUnit: A cook's tour. Java Report, 4(5):27--38, May 1999
• 9: E. Starkloff, Designing a Parallel, Distributed Test System, Proceedings of the IEEE AUTOTESTCON, 2000.
• 10: Fran Berman , Geoffrey Fox , Anthony J. G. Hey , Tony Hey, Grid Computing: Making the Global Infrastructure a Reality, John Wiley & Sons, Inc., New York, NY, 2003
• 11: G. M. Kapfhammer, Automatically and Transparently Distributing the Execution of Regression Test Suites, in: Proceedings of the 18th International Conference on Testing Computer Software, 2001.
• 12: Ian Foster , Carl Kesselman, The Grid 2: Blueprint for a New Computing Infrastructure, Morgan Kaufmann Publishers Inc., San Francisco, CA, 2003
• 13: M. Ben-Ari. The bug that destroyed a rocket. Journal of Computer Science Education, 13(2):15--16, 1999.
• 14: Nazareno Andrade , Francisco Brasileiro , Walfredo Cirne , Miranda Mowbray, Discouraging Free Riding in a Peer-to-Peer CPU-Sharing Grid, Proceedings of the 13th IEEE International Symposium on High Performance Distributed Computing (HPDC'04), p.129-137, June 04-06, 2004 [doi>10.1109/HPDC.2004.9]
• 15: P. Anderson, A. Scobie, LCFG: The Next Generation. UKUUG Winter Conference, 2002
• 16: P. Anderson, P. Goldsack and J. Paterson, SmartFrog meets LCFG: Autonomous Reconfiguration with Central Policy Control, in: Proceedings of the 2002 Large Installations Systems Administration, 2002.
• 17: P. Mellor, CAD: Computer-Aided Disaster, High. Integr. Syst., 1(2):101--156, 1994.
• 18: Raissa Medeiros , Walfredo Cirne , Francisco Brasileiro , Jacques Sauvé, Faults in Grids: Why are they so bad and What can be done about it?, Proceedings of the Fourth International Workshop on Grid Computing, p.18, November 17-17, 2003
• 19: The Globus Alliance, Globus. http://www.globus.org, 2005.
• 20: W. Cirne, F. Brasileiro, N. Andrade, R. Santos, A. Andrade, R. Novaes and M. Mowbray, Labs of the World, Unite!!!, Accepted for publication by JoGC. http://www.ourgrid.org/

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging
          Subjects: Testing tools (e.g., data generators, coverage testing)

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging
          Subjects: Distributed debugging

General Terms:
Performance, Reliability, Verification

Keywords:
JUnit, computational grid, distributed testing, unit testing