skip to main content
10.1145/3106237.3106268acmconferencesArticle/Chapter ViewAbstractPublication PagesfseConference Proceedingsconference-collections
research-article

Measuring neural efficiency of program comprehension

Published: 21 August 2017 Publication History

Abstract

Most modern software programs cannot be understood in their entirety by a single programmer. Instead, programmers must rely on a set of cognitive processes that aid in seeking, filtering, and shaping relevant information for a given programming task. Several theories have been proposed to explain these processes, such as ``beacons,' for locating relevant code, and ``plans,'' for encoding cognitive models. However, these theories are decades old and lack validation with modern cognitive-neuroscience methods. In this paper, we report on a study using functional magnetic resonance imaging (fMRI) with 11 participants who performed program comprehension tasks. We manipulated experimental conditions related to beacons and layout to isolate specific cognitive processes related to bottom-up comprehension and comprehension based on semantic cues. We found evidence of semantic chunking during bottom-up comprehension and lower activation of brain areas during comprehension based on semantic cues, confirming that beacons ease comprehension.

References

[1]
And Turken and Nina Dronkers. 2011. The Neural Architecture of the Language Comprehension Network: Converging Evidence from Lesion and Connectivity Analyses. Frontiers in Systems Neuroscience 5, 1 (2011).
[2]
Measuring Neural Efficiency of Program Comprehension ESEC/FSE’17, September 4–8, 2017, Paderborn, Germany
[3]
Jörg Bahlmann, Ricarda Schubotz, and Angela Friederici. 2008. Hierarchical Artificial Grammar Processing Engages Broca’s Area. NeuroImage 42, 2 (2008), 525–534.
[4]
Marie T Banich. 1998. The missing link: the role of interhemispheric interaction in attentional processing. Brain and Cognition 36, 2 (March 1998), 128–157.
[5]
Jeffrey Binder, Rutvik Desai, William Graves, and Lisa Conant. 2009. Where Is the Semantic System? A Critical Review and Meta-Analysis of 120 Functional Neuroimaging Studies. Cerebral Cortex 19, 12 (2009), 2767–2796.
[6]
Jeffrey R Binder, Julia A Frost, Thomas A Hammeke, PSF Bellgowan, Stephen M Rao, and Robert W Cox. 1999. Conceptual Processing during the Conscious Resting State: A Functional MRI Study. J. Cognitive Neuroscience 11, 1 (1999), 80–95.
[7]
Korbinian Brodmann. 2006. Brodmann’s Localisation in the Cerebral Cortex. Springer.
[8]
Ruven Brooks. 1978. Using a Behavioral Theory of Program Comprehension in Software Engineering. In Proc. Int. Conf. Software Engineering (ICSE). IEEE, 196–201.
[9]
Ruven Brooks. 1983. Towards a Theory of the Comprehension of Computer Programs. Int. J. Man-Machine Studies 18, 6 (1983), 543–554.
[10]
B Chance, Z Zhuang, C UnAh, C Alter, and Lipton L. 1993. Cognition-Activated Low-Frequency Modulation of Light Absorption in Human Brain. Proc. Nat. Academy Sciences of the United States of America (PNAS) 90, 8 (1993), 3770–3774.
[11]
Sylvain Charron and Etienne Koechlin. 2010. Divided Representation of Concurrent Goals in the Human Frontal Lobes. Science 328, 5976 (16 April 2010), 360–363.
[12]
Igor Crk and Timothy Kluthe. 2014. Toward Using Alpha and Theta Brain Waves to Quantify Programmer Expertise. In Proc. Int. Conf. Engineering in Medicine and Biology Society (EMBC). IEEE, 5373–5376.
[13]
Igor Crk, Timothy Kluthe, and Andreas Stefik. 2015. Understanding Programming Expertise: An Empirical Study of Phasic Brain Wave Changes. ACM Trans. Comput.-Hum. Interact. 23, 1, Article 2 (Dec. 2015), 29 pages.
[14]
J Dejerine. 1891. Sur un cas de Cecite Verbale avec Agraphie, Suivi d’Autopsie. C. R. Societe de Biologie 43 (1891), 197–201.
[15]
Nina F Dronkers, David P Wilkins, Robert D Van Valin, Brenda B Redfern, and Jeri J Jaeger. 2004. Lesion Analysis of the Brain Areas Involved in Language Comprehension. Cognition 92, 1–2 (2004), 145–177.
[16]
João Duraes, Henrique Madeira, J Castelhano, C Duarte, and M Castelo Branco. 2016. WAP: Understanding the Brain at Software Debugging. In Proc. Int. Symp. Software Reliability Engineering (ISSRE). IEEE, 87–92.
[17]
Christian J Fiebach, Matthias Schlesewsky, Gabriele Lohmann, D Yves Von Cramon, and Angela D Friederici. 2005. Revisiting the Role of Broca’s Area in Sentence Processing: Syntactic Integration Versus Syntactic Working Memory. Human Brain Mapping 24, 2 (2005), 79–91.
[18]
Benjamin Floyd, Tyler Santander, and Westley Weimer. 2017. Decoding the Representation of Code in the Brain: An fMRI Study of Code Review and Expertise. In Proc. Int. Conf. Software Engineering (ICSE). IEEE.
[19]
Angela D Friederici. 2002. Towards a Neural Basis of Auditory Sentence Processing. Trends in Cognitive Sciences 6, 2 (2002), 78–84.
[20]
Angela D Friederici and Sonja A Kotz. 2003. The Brain Basis of Syntactic Processes: Functional Imaging and Lesion Studies. NeuroImage 20, 1 (2003), S8–S17.
[21]
Thomas Fritz, Andrew Begel, Sebastian C. Müller, Serap Yigit-Elliott, and Manuela Züger. 2014. Using Psycho-physiological Measures to Assess Task Difficulty in Software Development. In Proc. Int. Conf. Software Engineering (ICSE). ACM, 402–413.
[22]
Norman Geschwind. 1965. Disconnexion Syndromes in Animals and Man. Brain 88, 2 (1965), 237–294.
[23]
Yosef Grodzinsky and Andrea Santi. 2008. The Battle for Broca’s Region. Trends in Cognitive Sciences 12, 12 (2008), 474–480.
[24]
Scott A Huettel, Allen W Song, and Gregory McCarthy. 2004. Functional Magnetic Resonance Imaging. Vol. 1. Sinauer Associates Sunderland.
[25]
Yoshiharu Ikutani and Hidetake Uwano. 2014. Brain Activity Measurement during Program Comprehension with NIRS. In Proc. Int. Conf. Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD). IEEE, 1–6.
[26]
Seolhwa Lee, Danial Hooshyar, Hyesung Ji, Kichun Nam, and Heuiseok Lim. 2017. Mining Biometric Data to Predict Programmer Expertise and Task Difficulty. Cluster Computing (2017), 1–11.
[27]
SeolHwa Lee, Andrew Matteson, Danial Hooshyar, SongHyun Kim, JaeBum Jung, GiChun Nam, and HeuiSeok Lim. 2016. Comparing Programming Language Comprehension between Novice and Expert Programmers Using EEG Analysis. In Proc. Int. Conf. Bioinformatics and Bioengineering (BIBE). IEEE, 350–355.
[28]
Richard C Linger, Harlan D Mills, and Bernard I Witt. 1979. Structured Programming: Theory and Practice. (1979).
[29]
Richard J. Miara, Joyce A. Musselman, Juan A. Navarro, and Ben Shneiderman. 1983. Program Indentation and Comprehensibility. Commun. ACM 26, 11 (1983), 861–867.
[30]
Takao Nakagawa, Yasutaka Kamei, Hidetake Uwano, Akito Monden, Kenichi Matsumoto, and Daniel M. German. 2014. Quantifying Programmers’ Mental Workload During Program Comprehension Based on Cerebral Blood Flow Measurement: A Controlled Experiment. In Proc. Int. Conf. Software Engineering (ICSE). ACM, 448–451.
[31]
Aljoscha C. Neubauer and Andreas Fink. 2009. Intelligence and neural efficiency. Neuroscience and Biobehavioral Reviews 33, 7 (10 July 2009), 1004–1023.
[32]
Chris Parnin. 2011. Subvocalization — Toward Hearing the Inner Thoughts of Developers. In Proc. Int. Conf. Program Comprehension (ICPC). IEEE, 197–200.
[33]
Nancy Pennington. 1987. Stimulus Structures and Mental Representations in Expert Comprehension of Computer Programs. Cognitive Psychologys 19, 3 (1987), 295–341.
[34]
Karl Petersson, Vasiliki Folia, and Peter Hagoort. 2012. What Artificial Grammar Learning Reveals about the Neurobiology of Syntax. Brain and Language 298, 1089 (2012), 199–209.
[35]
Marcus Raichle, Ann MacLeod, Abraham Snyder, William Powers, Debra Gusnard, and Gordon Shulman. 2001. A Default Mode of Brain Function. Proc. Nat. Academy of Sciences 98, 2 (2001), 676–682.
[36]
Charles Rich. 1981. Inspection Methods in Programming. Technical Report TR-604. MIT.
[37]
Tobias Roehm, Rebecca Tiarks, Rainer Koschke, and Walid Maalej. 2012. How Do Professional Developers Comprehend Software?. In Proc. Int. Conf. Software Engineering (ICSE). IEEE, 255–265.
[38]
Yuri Sato, Sayako Masuda, Yoshiaki Someya, Takeo Tsujii, and Shigeru Watanabe. 2015. An fMRI Analysis of the Efficacy of Euler Diagrams in Logical Reasoning. In Proc. Symp. on Visual Languages and Human-Centric Computing (VL/HCC). IEEE, 143–151.
[39]
Mohamed L Seghier. 2013. The Angular Gyrus: Multiple Functions and Multiple Subdivisions. The Neuroscientist 19, 1 (2013), 43–61.
[40]
Ben Shneiderman and Richard Mayer. 1979. Syntactic/Semantic Interactions in Programmer Behavior: A Model and Experimental Results. Int. J. Parallel Programming 8, 3 (1979), 219–238.
[41]
Janet Siegmund, Christian Kästner, Sven Apel, Chris Parnin, Anja Bethmann, Thomas Leich, Gunter Saake, and André Brechmann. 2014. Understanding Understanding Source Code with Functional Magnetic Resonance Imaging. In Proc. Int. Conf. Software Engineering (ICSE). IEEE, 378–389.
[42]
Janet Siegmund, Christian Kästner, Jörg Liebig, Sven Apel, and Stefan Hanenberg. 2014. Measuring and Modeling Programming Experience. Empirical Softw. Eng. 19, 5 (Oct. 2014), 1299–1334.
[43]
Janet Siegmund, Norbert Siegmund, and Sven Apel. 2015. Views on Internal and External Validity in Empirical Software Engineering. In Proc. Int. Conf. Software Engineering (ICSE), Vol. 1. IEEE, 9–19.
[44]
P Skosnik, F Mirza, D Gitelman, T Parrish, M Mesulam, and P Reber. 2008. Neural Correlates of Artificial Grammar Learning. NeuroImage 17, 3 (2008), 1306–1314.
[45]
Elliot Soloway and Kate Ehrlich. 1984. Empirical Studies of Programming Knowledge. IEEE Trans. Softw. Eng. 10, 5 (1984), 595–609.
[46]
Craig EL Stark and Larry R Squire. 2001. When Zero is Not Zero: The Problem of Ambiguous Baseline Conditions in fMRI. Proc. Nat. Academy Sciences of the United States of America (PNAS) 98, 22 (2001), 12760–12766.
[47]
Jean Talairach and Pierre Tournoux. 1988. Co-Planar Stereotaxic Atlas of the Human Brain. Thieme.
[48]
Miranka Wirth, Kay Jann, Thomas Dierks, Andrea Federspiel, Roland Wiest, and Helge Horn. 2011. Semantic Memory Involvement in the Default Mode Network: A Functional Neuroimaging Study Using Independent Component Analysis. NeuroImage 54, 4 (2011), 3057–3066.

Cited By

View all
  • (2025)Toward a theory on programmer’s block inspired by writer’s blockEmpirical Software Engineering10.1007/s10664-024-10542-930:1Online publication date: 1-Feb-2025
  • (2024)EEG as a potential ground truth for the assessment of cognitive state in software development activities: A multimodal imaging studyPLOS ONE10.1371/journal.pone.029910819:3(e0299108)Online publication date: 7-Mar-2024
  • (2024)Investigating the impact of logical reasoning in honing code comprehension skills: An empirical analysisProceedings of the 2024 Sixteenth International Conference on Contemporary Computing10.1145/3675888.3676024(35-41)Online publication date: 8-Aug-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ESEC/FSE 2017: Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering
August 2017
1073 pages
ISBN:9781450351058
DOI:10.1145/3106237
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 August 2017

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. functional magnetic resonance imaging
  2. neural efficiency
  3. program comprehension

Qualifiers

  • Research-article

Conference

ESEC/FSE'17
Sponsor:

Acceptance Rates

Overall Acceptance Rate 112 of 543 submissions, 21%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)88
  • Downloads (Last 6 weeks)9
Reflects downloads up to 07 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Toward a theory on programmer’s block inspired by writer’s blockEmpirical Software Engineering10.1007/s10664-024-10542-930:1Online publication date: 1-Feb-2025
  • (2024)EEG as a potential ground truth for the assessment of cognitive state in software development activities: A multimodal imaging studyPLOS ONE10.1371/journal.pone.029910819:3(e0299108)Online publication date: 7-Mar-2024
  • (2024)Investigating the impact of logical reasoning in honing code comprehension skills: An empirical analysisProceedings of the 2024 Sixteenth International Conference on Contemporary Computing10.1145/3675888.3676024(35-41)Online publication date: 8-Aug-2024
  • (2024)Towards Inclusive Source Code Readability Based on the Preferences of Programmers with Visual ImpairmentsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642512(1-18)Online publication date: 11-May-2024
  • (2024)Towards a Cognitive Model of Dynamic Debugging: Does Identifier Construction Matter?IEEE Transactions on Software Engineering10.1109/TSE.2024.346522250:11(3007-3021)Online publication date: Nov-2024
  • (2024)VarGAN: Adversarial Learning of Variable Semantic RepresentationsIEEE Transactions on Software Engineering10.1109/TSE.2024.339173050:6(1505-1517)Online publication date: Jun-2024
  • (2024)Understanding Code Change with Micro-Changes2024 IEEE International Conference on Software Maintenance and Evolution (ICSME)10.1109/ICSME58944.2024.00041(363-374)Online publication date: 6-Oct-2024
  • (2024)An amalgamation of cognitive aspects in software engineering: A content analysisExpert Systems with Applications10.1016/j.eswa.2023.122968246(122968)Online publication date: Jul-2024
  • (2024)An eye tracking study assessing source code readability rules for program comprehensionEmpirical Software Engineering10.1007/s10664-024-10532-x29:6Online publication date: 5-Oct-2024
  • (2024)Neurophysiological Measurements in the Research Field of Interruption Science: Insights into Applied Methods for Different Interruption Types Based on an Umbrella ReviewInformation Systems and Neuroscience10.1007/978-3-031-58396-4_11(123-152)Online publication date: 26-Jul-2024
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media