skip to main content
10.1145/1753326.1753478acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

What would other programmers do: suggesting solutions to error messages

Published: 10 April 2010 Publication History

Abstract

Interpreting compiler errors and exception messages is challenging for novice programmers. Presenting examples of how other programmers have corrected similar errors may help novices understand and correct such errors. This paper introduces HelpMeOut, a social recommender system that aids the debugging of error messages by suggesting solutions that peers have applied in the past. HelpMeOut comprises IDE instrumentation to collect examples of code changes that fix errors; a central database that stores fix reports from many users; and a suggestion interface that, given an error, queries the database for a list of relevant fixes and presents these to the programmer. We report on implementations of this architecture for two programming languages. An evaluation with novice programmers found that the technique can suggest useful fixes for 47% of errors after 39 person-hours of programming in an instrumented environment.

Supplementary Material

JPG File (p1019.jpg)
MOV File (p1019.mov)

References

[1]
Ahmadzadeh, M., Elliman, D., and Higgins, C. An analysis of patterns of debugging among novice computer science students. Proceedings of the 10th annual SIGCSE conference on Innovation and technology in computer science education, ACM (2005), 84--88.
[2]
Birnbaum, B.E. and Goldman, K.J. Achieving Flexibility in Direct-Manipulation Programming Environments by Relaxing the Edit-Time Grammar. Proceedings of the IEEE Symposium on Visual Languages and Human Centric Computing, IEEE Computer Society (2005), 259--266.
[3]
Boustani, N.E. and Hage, J. Improving type error messages for generic java. Proceedings of the 2009 ACM SIGPLAN workshop on Partial evaluation and program manipulation, ACM (2009), 131--140.
[4]
Brandt, J., Dontcheva, M., Weskamp, M., and Klemmer, S.R. Example-Centric Programming: Integrating Web Search into the Development Environment. Proceedings of CHI 2010, (2010).
[5]
Brandt, J., Guo, P.J., Lewenstein, J., Dontcheva, M., and Klemmer, S.R. Opportunistic Programming: Writing Code to Prototype, Ideate, and Discover. IEEE Software 26, 5 (2009), 18--24.
[6]
Brandt, J., Guo, P.J., Lewenstein, J., Dontcheva, M., and Klemmer, S.R. Two studies of opportunistic programming: interleaving web foraging, learning, and writing code. Proceedings of the 27th international conference on Human factors in computing systems, ACM (2009), 1589--1598.
[7]
Cheng, L., Souza, C.R.D., Hupfer, S., Patterson, J., and Ross, S. Building Collaboration into IDEs. Queue 1, 9 (2004), 40--50.
[8]
Dixon, P. pastebin -- collaborative debugging tool. http://pastebin.com/.
[9]
Domingue, J. and Mulholland, P. Fostering debugging communities on the Web. Communications of the ACM 40, 4 (1997), 65--71.
[10]
Fitzgerald, S., Lewandowski, G., McCauley, R., et al. Debugging: Finding, Fixing and Flailing, a Multi-Institutional Study of Novice Debuggers. Computer Science Education 18, 2 (2008), 93--116.
[11]
Gick, M.L. and Holyoak, K.J. Analogical Problem Solving. Cognitive Psychology 12, 3 (1980), 306--55.
[12]
Gick, M.L. and Holyoak, K.J. Schema induction and analogical transfer. Cognitive Psychology 15, 1 (1983), 1--38.
[13]
Grabler, F., Agrawala, M., Li, W., Dontcheva, M., and Igarashi, T. Generating photo manipulation tutorials by demonstration. ACM Transactions on Graphics 28, 3 (2009), 1--9.
[14]
Heckel, P. A technique for isolating differences between files. Communications of the ACM 21, 4 (1978), 264--268.
[15]
Hilbert, D.M. and Redmiles, D.F. Extracting usability information from user interface events. ACM Computing Surveys 32, 4 (2000), 384--421.
[16]
Hoffmann, R., Fogarty, J., and Weld, D.S. Assieme: finding and leveraging implicit references in a web search interface for programmers. Proceedings of the 20th annual ACM symposium on User interface software and technology, ACM (2007), 13--22.
[17]
Jeffery, C.L. Generating LR syntax error messages from examples. ACM Transactions on Programming Languages and Systems 25, 5 (2003), 631--640.
[18]
Jiang, L., Misherghi, G., Su, Z., and Glondu, S. DECKARD: Scalable and Accurate Tree-Based Detection of Code Clones. Proceedings of the 29th international conference on Software Engineering, IEEE (2007), 96--105.
[19]
Kim, S., Pan, K., and E. E. James Whitehead, J. Memories of bug fixes. Proceedings of the 14th ACM SIGSOFT international symposium on Foundations of software engineering, ACM (2006), 35--45.
[20]
Ko, A.J. and Myers, B.A. Debugging reinvented: asking and answering why and why not questions about program behavior. Proceedings of the 30th international conference on Software engineering, ACM (2008), 301--310.
[21]
Levenshtein, V.I. Binary codes capable of correcting deletions, insertions and reversals (in Russian). Soviet Physics Doklady 10, 8 (1966), 707--710.
[22]
Liblit, B., Naik, M., Zheng, A.X., Aiken, A., and Jordan, M.I. Scalable statistical bug isolation. Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation, ACM (2005), 15--26.
[23]
Linton, F. and Schaefer, H. Recommender Systems for Learning: Building User and Expert Models through Long-Term Observation of Application Use. User Modeling and User-Adapted Interaction 10, 2--3 (2000), 181--208.
[24]
Livshits, B. and Zimmermann, T. DynaMine: finding common error patterns by mining software revision histories. SIGSOFT Software Engineering Notes 30, 5 (2005), 296--305.
[25]
Matejka, J., Li, W., Grossman, T., and Fitzmaurice, G. CommunityCommands: command recommendations for software applications. Proceedings of the 22nd annual ACM symposium on User interface software and technology, ACM (2009), 193--202.
[26]
McCauley, R., Fitzgerald, S., Lewandowski, G., et al. Debugging: A Review of the Literature from an Educational Perspective. Computer Science Education 18, 2 (2008).
[27]
Nardi, B. A small matter of programming. MIT Press, 1993.
[28]
Nienaltowski, M., Pedroni, M., and Meyer, B. Compiler error messages: what can help novices? Proceedings of the 39th SIGCSE technical symposium on Computer science education, ACM (2008), 168--172.
[29]
Shiffman, D. Learning Processing: A Beginner's Guide to Programming Images, Animation, and Interaction. Morgan Kaufmann, 2008.
[30]
Smith, R.B., Wolczko, M., and Ungar, D. From Kansas to Oz: collaborative debugging when a shared world breaks. Communications of the ACM 40, 4 (1997), 72--78.
[31]
Stylos, J., Faulring, A., Yang, Z., and Myers, B.A. Improving API Documentation Using API Usage Information. Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing, VL/HCC'09, (2009).
[32]
Teitelbaum, T. and Reps, T. The Cornell program synthesizer: a syntax--directed programming environment. Communications of the ACM 24, 9 (1981), 563--573.
[33]
Terry, M., Kay, M., Vugt, B.V., Slack, B., and Park, T. Ingimp: introducing instrumentation to an end-user open source application. Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, ACM (2008), 607--616.
[34]
Yeh, R.B., Paepcke, A., and Klemmer, S.R. Iterative design and evaluation of an event architecture for pen-and-paper interfaces. Proceedings of the 21st annual ACM symposium on User interface software and technology, ACM (2008), 111--120.

Cited By

View all
  • (2025)Grammar-obeying program synthesis: A novel approach using large language models and many-objective genetic programmingComputer Standards & Interfaces10.1016/j.csi.2024.10393892(103938)Online publication date: Mar-2025
  • (2024)Evaluating the usability of differential privacy tools with data practitionersProceedings of the Twentieth USENIX Conference on Usable Privacy and Security10.5555/3696899.3696901(21-40)Online publication date: 12-Aug-2024
  • (2024)Enhancing Program Synthesis with Large Language Models Using Many-Objective Grammar-Guided Genetic ProgrammingAlgorithms10.3390/a1707028717:7(287)Online publication date: 1-Jul-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI '10: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
April 2010
2690 pages
ISBN:9781605589299
DOI:10.1145/1753326
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 April 2010

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. debugging
  2. recommender systems

Qualifiers

  • Research-article

Conference

CHI '10
Sponsor:

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025
ACM CHI Conference on Human Factors in Computing Systems
April 26 - May 1, 2025
Yokohama , Japan

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)82
  • Downloads (Last 6 weeks)9
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Grammar-obeying program synthesis: A novel approach using large language models and many-objective genetic programmingComputer Standards & Interfaces10.1016/j.csi.2024.10393892(103938)Online publication date: Mar-2025
  • (2024)Evaluating the usability of differential privacy tools with data practitionersProceedings of the Twentieth USENIX Conference on Usable Privacy and Security10.5555/3696899.3696901(21-40)Online publication date: 12-Aug-2024
  • (2024)Enhancing Program Synthesis with Large Language Models Using Many-Objective Grammar-Guided Genetic ProgrammingAlgorithms10.3390/a1707028717:7(287)Online publication date: 1-Jul-2024
  • (2024)Decoding Debugging Instruction: A Systematic Literature Review of Debugging InterventionsACM Transactions on Computing Education10.1145/369065224:4(1-44)Online publication date: 15-Nov-2024
  • (2024)Understanding Novice Users' Mental Models of Gesture Discoverability and Designing Effective OnboardingCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3678370(290-295)Online publication date: 5-Oct-2024
  • (2024)Say What? Real-time Linguistic Guidance Supports Novices in Writing Utterances for Conversational Agent TrainingProceedings of the 6th ACM Conference on Conversational User Interfaces10.1145/3640794.3665554(1-12)Online publication date: 8-Jul-2024
  • (2024)Why and When LLM-Based Assistants Can Go Wrong: Investigating the Effectiveness of Prompt-Based Interactions for Software Help-SeekingProceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640543.3645200(288-303)Online publication date: 18-Mar-2024
  • (2024)Annota: Peer-based AI Hints Towards Learning Qualitative Coding at ScaleProceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640543.3645168(455-470)Online publication date: 18-Mar-2024
  • (2024)Do I Just Tap My Headset?Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314517:4(1-28)Online publication date: 12-Jan-2024
  • (2024)A Large Scale RCT on Effective Error Messages in CS1Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630764(1395-1401)Online publication date: 7-Mar-2024
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media