Improving integrated development environment commands knowledge with recommender systems

ABSTRACT

Development tools have an impact on software engineers' productivity and quality of software construction. We believe that it is crucial to teach future software engineers how to exploit integrated development environment functionality, if we want to encourage the effective application of software development principles and practices. Our research shows that recommender systems can be deployed to improve integrated development environment knowledge of computer science students by automatically suggesting new and useful commands, such as buttons and shortcuts that execute different functions. While previous work focused on optimizing the algorithmic predictive capability of a recommender to identify the commands that the users will eventually use, we have addressed a set of research questions related to the overall acceptance of a complete recommender system in a real-life setting. The evaluation results show that a command recommender system can be well accepted by computer science students. In particular, when students are supported by such a system, they use a considerably larger set of commands available in their development environment. Moreover, the results show that the highest acceptance rate and the usefulness score were achieved by a non-personalized, popularity-based algorithm, while the most novel commands were suggested by a context-aware algorithm.

References

1. A. Ahmed. 2011. Software Project Management: A Process-Driven Approach. Taylor & Francis.
2. J. Anderson-Meger. 2016. Why Do I Need Research and Theory?: A Guide for Social Workers. Taylor & Francis.
3. D. Campbell and M. Miller. 2008. Designing Refactoring Tools for Developers. In Workshop on Refactoring Tools.
4. G. Fischer. 2001. User Modeling in Human-Computer Interaction. User Modeling and User-Adapted Interaction 11 (2001), 65--86.
5. M. Gasparic, T. Gurbanov, and F. Ricci. 2017. Context-Aware Integrated Development Environment Command Recommender Systems. In IEEE/ACM International Conference on Automated Software Engineering.
6. M. Gasparic and A. Janes. 2016. What recommendation systems for software engineering recommend: A systematic literature review. Journal of Systems and Software 113 (2016), 101--113.
7. M. Gasparic, A. Janes, F. Ricci, G. C. Murphy, and T. Gurbanov. 2017. A graphical user interface for presenting integrated development environment command recommendations: Design, evaluation, and implementation. Information and Software Technology 92 (2017), 236--255.
8. M. Gasparic, A. Janes, F. Ricci, and M. Zanellati. 2017. GUI Design for IDE Command Recommendations. In International Conference on Intelligent User Interfaces.
9. M. Gasparic, G. C. Murphy, and F. Ricci. 2017. A context model for IDE-based recommendation systems. Journal of Systems and Software 128 (2017), 200--219.
10. M. Gasparic and F. Ricci. 2017. Should Context-Aware IDE Command Recommendations Always Be Presented In-Context or Not?. In AWARE workshop.
11. T. Grossman, G. Fitzmaurice, and R. Attar. 2009. A Survey of Software Learnability: Metrics, Methodologies and Guidelines. In SIGCHI Conference on Human Factors in Computing Systems.
12. IEEE Computer Society, P. Bourque, and R. E. Fairley. 2014. Guide to the Software Engineering Body of Knowledge. IEEE Computer Society Press.
13. M. Kersten and G. C. Murphy. 2006. Using Task Context to Improve Programmer Productivity. In ACM/SIGSOFT International Symposium on Foundations of Software Engineering.
14. B. P. Knijnenburg, M. C. Willemsen, Z. Gantner, H. Soncu, and C. Newell. 2012. Explaining the User Experience of Recommender Systems. User Modeling and User-Adapted Interaction 22 (2012), 441--504.
15. W. Li, J. Matejka, T. Grossman, J. A. Konstan, and G. Fitzmaurice. 2011. Design and Evaluation of a Command Recommendation System for Software Applications. ACM Transactions on Computer-Human Interaction 18 (2011), 6:1--6:35.
16. J. Matejka, W. Li, T. Grossman, and G. Fitzmaurice. 2009. CommunityCommands: Command Recommendations for Software Applications. In ACM Symposium on User Interface Software and Technology.
17. S. M. McNee, J. Riedl, and J. A. Konstan. 2006. Being Accurate is Not Enough: How Accuracy Metrics Have Hurt Recommender Systems. In CHI Extended Abstracts on Human Factors in Computing Systems.
18. E. Murphy-Hill. 2012. Continuous Social Screencasting to Facilitate Software Tool Discovery. In International Conference on Software Engineering.
19. E. Murphy-Hill, R. Jiresal, and G. C. Murphy. 2012. Improving Software Developers' Fluency by Recommending Development Environment Commands. In ACM/SIGSOFT International Symposium on the Foundations of Software Engineering.
20. E. Murphy-Hill, D. Y. Lee, G. C. Murphy, and J. McGrenere. 2015. How Do Users Discover New Tools in Software Development and Beyond? Computer Supported Cooperative Work 24 (2015), 389--422.
21. P. Resnick and H. R. Varian. 1997. Recommender Systems. Commun. ACM 40 (1997), 56--58.
22. F. Ricci. 2014. Recommender Systems: Models and Techniques. In Encyclopedia of Social Network Analysis and Mining. Springer, 1511--1522.
23. F. Ricci, L. Rokach, and B. Shapira. 2015. Recommender Systems: Introduction and Challenges. In Recommender Systems Handbook. Springer, 1--34.
24. M. P. Robillard and R. J. Walker. 2014. An Introduction to Recommendation Systems in Software Engineering. In Recommendation Systems in Software Engineering. Springer, 1--11.
25. W. Scacchi. 1991. Understanding software productivity: towards a Knowledge-based approach. International Journal of Software Engineering and Knowledge Engineering 1 (1991), 293--321.
26. Janet Siegmund, Norbert Siegmund, and Sven Apel. 2015. Views on Internal and External Validity in Empirical Software Engineering. In International Conference on Software Engineering.
27. I. Sommerville. 2011. Software Engineering. Pearson.
28. S. Stuckemann. 2015. Vignelli - Automated Design Guidance for Developers. Imperial College London (2015).
29. P. Viriyakattiyaporn and G. C. Murphy. 2010. Improving Program Navigation with an Active Help System. In Conference of the Center for Advanced Studies on Collaborative Research.
30. B. Walraet. 2014. A Discipline of Software Engineering. Elsevier.
31. C. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, and B. Regnell. 2012. Experimentation in Software Engineering. Springer.
32. S. Zolaktaf and G. C. Murphy. 2015. What to Learn Next: Recommending Commands in a Feature-Rich Environment. In International Conference on Machine Learning and Applications.