Reactive information foraging: an empirical investigation of theory-based recommender systems for programmers

ABSTRACT

Information Foraging Theory (IFT) has established itself as an important theory to explain how people seek information, but most work has focused more on the theory itself than on how best to apply it. In this paper, we investigate how to apply a reactive variant of IFT (Reactive IFT) to design IFT-based tools, with a special focus on such tools for ill-structured problems. Toward this end, we designed and implemented a variety of recommender algorithms to empirically investigate how to help people with the ill-structured problem of finding where to look for information while debugging source code. We varied the algorithms based on scent type supported (words alone vs. words + code structure), and based on use of foraging momentum to estimate rapidity of foragers' goal changes. Our empirical results showed that (1) using both words and code structure significantly improved the ability of the algorithms to recommend where software developers should look for information; (2) participants used recommendations to discover new places in the code and also as shortcuts to navigate to known places; and (3) low-momentum recommendations were significantly more useful than high-momentum recommendations, suggesting rapid and numerous goal changes in this type of setting. Overall, our contributions include two new recommendation algorithms, empirical evidence about when and why participants found IFT-based recommendations useful, and implications for the design of tools based on Reactive IFT.

References

1. Baeza-Yates, R., Ribeiro-Neto, B. Modern Information Retrieval, Addison Wesley Longman, 1999.
2. Bragdon, A., Zeleznik, R., Reiss, S., Karumuri, S., Cheung, W., Kaplan, J., Coleman, C., Adeputra, F., and LaViola, J. Proc. CHI, ACM (2010), 2503--2512.
3. Chi, E., Pirolli, P, Chen, K., and Pitkow, J. Using information scent to model user information needs and actions on the web. Proc. CHI, ACM (2001), 490--497.
4. Chi, E., Rosien, A., Supattanasiri, G., Williams, A., Royer, C., Chow, C., Robles, E., Dalal, B., Chen, J., and Cousins, S. The Bloodhound project: Automating discovery of web usability issues using the InfoScent simulator, Proc. CHI, ACM (2003), 505--512.
5. Cubranic, D, and Murphy, G. Hipikat: Recommending pertinent software development artifacts. Proc. ICSE, ACM/IEEE (2003), 408--418.
6. DeLine, R., Khella, A., Czerwinski, M., and Robertson, G. Towards understanding programs through wearbased filtering. Proc. Softvis, ACM (2005), 183--192.
7. Fu, W. and Pirolli, P. SNIF-ACT: A cognitive model of user navigation on the World Wide Web, HumanComputer Interaction 22, 4 (2007), 355--412.
8. Grigoreanu, V., Burnett, M., Wiedenbeck, S., Cao, J., Rector, K., and Kwan, I. End-User Debugging Strategies: A Sensemaking Perspective. ACM Trans. Comput.Hum. Interact. (to appear).
9. Gross, P. and Kelleher, C. Non-programmers identifying functionality in unfamiliar code: Strategies and barriers. Proc. VL/HCC. IEEE (2009), 75--82.
10. Hill, W., Hollan, J., Wroblewski, D., McCandless, T. Edit wear and read wear. Proc. CHI, ACM (1992), 3--9.
11. Jakobsen, M. and Hornbaek, K. Evaluating a fisheye view of source code. Proc. CHI, ACM (2006), 377--386.
12. Kersten, M. and Murphy, G. Mylar: A degree-of-interest model for IDEs. Proc. ASOD, ACM (2005), 159--168.
13. Ko, A., Myers, B., Coblenz, M., and Aung, H. An exploratory study of how developers seek, relate, and collect relevant information during software maintenance tasks. IEEE Trans. Soft. Eng. 32, (2006), 971--987.
14. Lawrance, J., Bellamy, R., Burnett, M., and Rector, K. Using information scent to model the dynamic foraging behavior of programmers in maintenance tasks, Proc. CHI, ACM (2008), 1323--1332.
15. Lawrance, J., Bogart, C., Burnett, M., Bellamy, R., Rector, K., and Fleming, S. How programmers debug, revisited: An information foraging theory perspective, IEEE Trans. Soft. Eng. (to appear 2012).
16. Lawrance, J., Burnett, M., Bellamy, R., Bogart, C., and Swart, C. Reactive information foraging for evolving goals. Proc. CHI, ACM (2010), 25--34.
17. Nielsen, J. Information foraging: Why Google makes people leave your site faster. http://www.useit.com/ alertbox/20030630.html. June 30, 2003.
18. Olston, C. and Chi, E. ScentTrails: Integrating browsing and searching on the web. ACM Trans. Comput.-Hum. Interact. 10, 3 (2003), 177--197.
19. Parnin, C. and Gorg, C. Building usage contexts during program comprehension. Proc. ICPC, IEEE (2006), 13--22
20. Piorkowski, D., Fleming, S. D., Scaffidi, C., John, L., Bogart, C., John, B. E., Burnett, M., and Bellamy, R. Modeling programmer navigation: A head-to-head empirical evaluation of predictive models. Proc. VL/HCC, IEEE (2011), 109--116.
21. Pirolli, P. Computational models of information scentfollowing in a very large browsable text collection. Proc. CHI 1997, ACM Press (1997), 3--10.
22. Pirolli, P. Information Foraging Theory: Adaptive Interaction with Information. Oxford Univ. Press, 2007.
23. Pirolli, P. and Card, S.K. Information foraging. Psychological Review 106, 4 (1999), 643--675.
24. Resnick, P. and Varian, H. Recommender systems. Commun. ACM 40, 3 (1997), 56--58.
25. Sillito, J., Murphy, G., and De Volder, K. Questions programmers ask during software evolution tasks. Proc. FSE, ACM (2006), 23--34.
26. Sillito, J., De Voider, K., Fisher, B., and Murphy, G. Managing software change tasks: An exploratory study. Proc. ISESE, IEEE (2005), 1--10.
27. Sinha, V., Karger, D., and Miller, R. Relo: Helping users manage context during interactive exploratory visualization of large codebases. Proc. VL/HCC, IEEE (2006), 187--194.
28. Spool, J., Profetti, C., and Britain, D., Designing for the scent of information, User Interface Engineering, (2004).
29. Storey, M, Best, C, Michaud, J, Rayside, D, Litoiu, M, and Musen, M. SHriMP views: An interactive environment for information visualization and navigation. Ext. Abstracts CHI '02, (2002), 520--521.
30. Wiedenbeck, S. and Evans, N. Beacons in program comprehension. ACM SIGCHI Bulletin 18, 2 (1986), 56--57.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

• Software and its engineering

  • Software creation and management

    • Software verification and validation

      • Software defect analysis

        • Software testing and debugging