Using a functional size measurement procedure to evaluate the quality of models in MDD environments

Abstract

Models are key artifacts in Model-Driven Development (MDD) methods. To produce high-quality software by using MDD methods, quality assurance of models is of paramount importance. To evaluate the quality of models, defect detection is considered a suitable approach and is usually applied using reading techniques. However, these reading techniques have limitations and constraints, and new techniques are required to improve the efficiency at finding as many defects as possible. This article presents a case study that has been carried out to evaluate the use of a Functional Size Measurement (FSM) procedure in the detection of defects in models of an MDD environment. To do this, we compare the defects and the defect types found by an inspection group with the defects and the defect types found by the FSM procedure. The results indicate that the FSM is useful since it finds all the defects related to a specific defect type, it finds different defect types than an inspection group, and it finds defects related to the correctness and the consistency of the models.

References

1. Abran, A., Desharnais, J., Lesterhuis, A., Londeix, B., Meli, R., Morris, P., Oligny, S., O'Neil, M., Rollo, T., Rule, G., Santillo, L., Symons, C., and Toivonen, H. 2007. The COSMIC functional size measurement method - Version 3.0. GELOG. www.gelog.etsmtl.ca.
2. Atkinson, C. 1998. Adapting the fusion process to support the unified modeling language. Object. Mag., Nov., 32--39.
3. Bellur, U. and Vallieswaran, V. 2006. On OO design consistency in iterative development. In Proceedings of the 3^rd International Conference on Information Technology: New Generations (ITNG'06). IEEE, 46--51.
4. Benbasat, I., Goldstein, D., and Mead, M. 1987. The case research strategy in studies of information systems. MIS Q. 11, 369--386.
5. Berenbach, B. 2004. The evaluation of large, complex uml analysis and design models. In Proceedings of the 26^th International Conference on Software Engineering (ICSE'04). IEEE Computer Society, 232--241.
6. Berkenkötter, K. 2008. Reliable uml models and profiles. Electron. Not. Theor. Comput. Sci. 217, 203--220.
7. Care-Technologies 2011. Web site. http://www.care-t.com/.
8. Coleman, D., Arnold, P., Godoff, S., Dollin, C., Gilchrist, H., Hayes, F., and Jeremaes, P. 1994. Object-Oriented Development: The Fusion Method. Prentice-Hall, Englewood Cliffs, NJ.
9. Conradi, R., Mohagheghi, P., Arif, T., Hegde, L. C., Bunde, G. A., and Pedersen, A. 2003. Object-oriented reading techniques for inspection of uml models -- An industrial experiment. In Proceedings of the 17^th European Conference on Object-Oriented Programming (ECOOP'03). Springer, 483--501.
10. Davenport, T. H. and Prusak, L. 1998. Working Knowledge: How Organisations Manage What They Know. Business School Press, Boston, MA.
11. Egyed, A. 2006. Instant consistency checking for the uml. In Proceedings of the 28^th International Conference on Software Engineering (ICSE'06). ACM Press, New York, 381--390.
12. España, S., González, A., and Pastor, O. 2009. Communication analysis: A requirements engineering method for information systems. In Proceedings of the 21^st International Conference on Advanced Information Systems Engineering (CAiSE'09). P. van Eck, J. Gordijn and R. Wieringa, Eds., Springer, 530--545.
13. Fink, T., Koch, M., and Pauls, K. 2006. An mda approach to access control specifications using mof and uml profiles. Electron. Not. Theor. Comput. Sci. 142, 161--179.
14. France, R. B., Ghosh, S., Dinh-Trong, T., and Solberg, A. 2006. Model-driven development using uml 2.0: Promises and pitfalls. IEEE Comput. 39, 59--66.
15. Giachetti, G., Albert, M., Marín, B., and Pastor, O. 2010. Linking uml and mdd through uml profiles: A practical approach based on the uml association. J. Universal Comput. Sci. 16, 2353--2373
16. Giachetti, G., Marín, B., and Pastor, O. 2009. Using uml as a domain-specific modeling language: A proposal for automatic generation of uml profiles. In Proceedings of the 21^st International Conference on Advanced Information Systems Engineering (CAiSE'09). P. van Eck, J. Gordijn and R. Wieringa, Eds., Springer, 110--124.
17. Gomaa, H. 2000. Designing Concurrent, Distributed, and Real-Time Applications with UML. Addison-Wesley.
18. Gomaa, H. and Wijesekera, D. 2003. Consistency in multiple-view uml models: A case study. In Proceedings of the Workshop on Consistency Problems in UML-Based Software Development II. IEEE, 1--8.
19. Gómez, J., Insfrán, E., Pelechano, V., and Pastor, O. 1998. The execution model: A component based architecture to generate software components from conceptual models. In Proceedings of the Workshop on Component-based Information Systems Engineering.
20. Hailpern, B. and Tarr, P. 2006. Model-driven develpment: The good, the bad, and the ugly. IBM Syst. J. 45, 451--461.
21. IEEE. 1990. IEEE 610 standard computer dictionary. A compilation of IEEE standard computer glossaries. http://standards.ieee.org/findstds/standard/610-1990.html.
22. IEEE. 2009. IEEE 1044 standard classification for software anomalies. https://standards.ieee.org/findstds/standard/1044-2009.html.
23. Insfrán, E., Pastor, O., and Wieringa, R. 2002. Requirements engineering-based conceptual modelling. J. Requir. Engin. 7, 2, 61--72.
24. ISO. 1998. ISO/IEC 14143-1 -- Information technology -- Software measurement -- Functional size measurement -- Part 1: Definition of concepts. http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=22774.
25. ISO. 2004. International vocabulary of basic and general terms in metrology (VIM), I.O.f. standardization Ed., Geneva, Switzerland. http://143.54.70.55/medterm/ISO_VEM.pdf.
26. ISO/IEC. 2001. ISO/IEC 9126-1, Software engineering -- Product quality -- Part 1: Quality model. http://www.iso.org/iso/iso_catalogue/catalo-gue_tc/catalogue_detail.htm?csnumber=22749.
27. ISO/IEC. 2003. ISO/IEC 19761, Software engineering -- COSMIC-FFP -- A functional size measurement method. http://www.iso.org/iso/iso_catalogue/catalogue_ics/catalogue_detail_ics.htm?csnumber=33899.
28. Kitchenham, B. 2007. Guidelines for performing systematic literature reviews in software engineering. http://www.uio.no/studier/emner/matnat/ifi/INF5500/h11/undervisningsmateriale/Systematic%20reviews.pdf.
29. Kuzniarz, L. 2003. Inconsistencies in student designs. In Proceedings of the Workshop on Consistency Problems in UML-based Software Development II. IEEE, 9--17.
30. Laitenberger, O., Atkinson, C., Schlich, M., and Emam, K. E. 2000. An experimental comparison of reading techniques for defect detection in uml design documents. J. Syst. Softw. 53, 183--204.
31. Lange, C. and Chaudron, M. 2004. An empirical assessment of completeness in uml designs. In Proceedings of the 8^th Conference on Empirical Assessment in Software Engineering (EASE'04). IEEE, 111--121.
32. Lange, C., Wijins, M., and Chaudron, M. 2007. Metric view evolution: UML-based views for monitoring model evolution and quality. In Proceedings of the 11^th European Conference on Software Maintenance and Reengineering (CSMR'07). IEEE, 327--328.
33. Leung, F. and Bolloju, N. 2005. Analyzing the quality of domain models developed by novice systems analysts. In Proceedings of the 38^th Hawaii International Conference on System Sciences. IEEE, 1--7.
34. Lindland, O. I., Sindre, G., and Solvberg, A. 1994. Understanding quality in conceptual modeling. IEEE Softw. 11, 42--49.
35. Marín, B. 2010. OOmCFP measurement guide. Tech. rep. DSIC-II/05/10, Unviersidad Politécnica de Valencia, Valencia, Spain.
36. Marín, B. 2011. Functional size measurement and model verification for software model-driven developments: A cosmic-based approach. Tech. rep., Departamento de Sistemas Informáticos y Computación Unviersidad Politécnica de Valencia. 333.
37. Marín, B., Condori-Fernández, N., and Pastor, O. 2008a. Design of a functional size measurement procedure for a model-driven software development method. In Proceedings of the 3^rd Workshop on Quality in Modeling of MODELS. J.-L. Sourrouille, M. Staron, L. Kuzniarz, P. Mohagheghi and L. Pareto, Eds., 1--15.
38. Marín, B., Condori-fernández, N., Pastor, O., and Abran, A. 2008b. Measuring the functional size of conceptual models in an mda environment. In Proceedings of the 20^th International Conference on Advanced Information Systems Engineering (CAiSE'08). Z. Bellahsene, C. Woo, E. Hunt, X. Franch, and R. Coletta, Eds., 33--36.
39. Marín, B., Giachetti, G., and Pastor, O. 2008c. Automating the measurement of functional size of conceptual models in an mda environment. In Proceedings of the 9^th International Conference on Product-Focused Software Process Improvement (PROFES'08). Springer, 215--229.
40. Marín, B., Giachetti, G., and Pastor, O. 2009. Applying a functional size measurement procedure for defect detection in mdd environments In Proceedings of the 16^th European Conference on Software Process Improvement (EuroSPI'09). R. V. O'Connor, Ed., Springer, 57--68.
41. Marín, B., Giachetti, G., Pastor, O., and Vos, T. E. J. 2010a. A tool for automatic defect detection in models used in model-driven engineering. In Proceedings of the 7^th International Conference on the Quality of Information and Communications Technology (QUATIC'10). IEEE, 242--247.
42. Marín, B., Pastor, O., and Abran, A. 2010b. Towards an accurate functional size measurement procedure for conceptual models in an mda environment. Data Knowl. Engin. 69, 472--490.
43. Mohagheghi, P. and Aagedal, J. 2007. Evaluating quality in model-driven engineering. In Proceedings of the International Workshop on Modeling in Software Engineering (MISE'07). IEEE Computer Society.
44. Moody, D. L. 2005. Theoretical and practical issues in evaluating the quality of conceptual models: Current state and future directions. Data Knowl. Engin. 55, 243--276.
45. Moreno, N., Fraternali, P., and Vallecillo, A. 2007. WebML modeling in uml. IET Softw. 1, 67--80.
46. OMG 2010. UML 2.3 superstructure specification. http://www.omg.org/spec/UML/2.3/.
47. Opdahl, A. L. and Henderson-Sellers, B. 2005. A unified modelling language without referential redundancy. Data Knowl. Engin. 55, 277--300.
48. Pastor, O. and Giachetti, G. 2010. Linking goal-oriented requirements and model-driven development. In Intentional Perpectives on Information Systems Engineering. Springer, 255--274.
49. Pastor, O., Hayes, F., and Bear, S. 1992. OASIS: An object-oriented specification language. In Proceedings of the International Conference on Advanced Information Systems Engineering (CAiSE'92). 348--363.
50. Pastor, O. and Molina, J. C. 2007. A software production environment based on conceptual modeling. In Model-Driven Architecture in Practice. Springer, New York.
51. Petticrew, M. and Roberts, H. 2005. Systematic Reviews in the Social Sciences: A Practical Guide. Blackwell Publishing.
52. Robson, C. 2002. Real World Research: A Resource for Social Scientists and Practitioner Researchers, 2^nd ed., Blackwell Publishing.
53. Runeson, P. and Host, M. 2009. Guidelines for conducting and reporting case study research in software engineering. Empirical Softw. Engin. J. 14, 131--164.
54. Schmidt, D. 2006. Model driven engineering. IEEE Comput. 39, 25--31.
55. Seaman, C. 1999. Qualitative methods in empirical studies of software engineering. IEEE Trans. Softw. Engin. 25, 557--572.
56. Selic, B. 2003. The pragmatics of model-driven development. IEEE Softw. 20, 19--25.
57. Selic, B. 2007. A systematic approach to domain-specific language design using uml. In Proceedings of the 10^th IEEE International Symposium on Object and Component-Oriented Real-Time Distributed Computing (ISORC'07). 2--9.
58. Travassos, G., Shull, F., Fredericks, M., and Basili, V. 1999. Detecting defects in object-oriented designs: Using reading techniques to increase software quality. In Proceedings of the 14^th ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications. 47--56.
59. Trudel, S. and Abran, A. 2008. Improving quality of functional requirements by measuring their functional size. In Proceedings of the International Conferences on Software Process and Product Measurement (IWSM/Metrikon/Mensura'08), Springer, 287--231.
60. Trudel, S. and Abran, A. 2010. Functional requirements improvements through size measurement: A case study with inexperienced measurers. In Proceedings of the 8^th ACIS International Conference on Software Engineering Research, Management and Applications (SERA'10). IEEE Computer Society, 181--189.
61. Yin, R. 2003. Case Study Research, Design and Methods 3^rd Ed. Sage, London.

Index Terms

• General and reference

  • Cross-computing tools and techniques

    • Validation

• Software and its engineering

  • Software creation and management

    • Software verification and validation

      • Empirical software validation

      • Process validation

  • Software organization and properties

    • Software functional properties

      • Correctness

• Theory of computation

  • Logic

    • Proof theory