The Story in the Notebook: Exploratory Data Science using a Literate Programming Tool

ABSTRACT

Literate programming tools are used by millions of programmers today, and are intended to facilitate presenting data analyses in the form of a narrative. We interviewed 21 data scientists to study coding behaviors in a literate programming environment and how data scientists kept track of variants they explored. For participants who tried to keep a detailed history of their experimentation, both informal and formal versioning attempts led to problems, such as reduced notebook readability. During iteration, participants actively curated their notebooks into narratives, although primarily through cell structure rather than markdown explanations. Next, we surveyed 45 data scientists and asked them to envision how they might use their past history in an future version control system. Based on these results, we give design guidance for future literate programming tools, such as providing history search based on how programmers recall their explorations, through contextual details including images and parameters.

References

1. Hugh Beyer and Karen Holtzblatt. 1997. Contextual design: defining customer-centered systems. Elsevier.
2. Eric A. Bier, Maureen C. Stone, Ken Pier, William Buxton, and Tony D. DeRose. 1993. Toolglass and magic lenses: the see-through interface. In Proceedings of the 20th annual conference on Computer graphics and interactive techniques, 73--80.
3. Joel Brandt, Philip J. Guo, Joel Lewenstein, and Scott R. Klemmer. 2008. Opportunistic programming: How rapid ideation and prototyping occur in practice. In Proceedings of the 4th international workshop on End-user software engineering, 1--5.
4. Juliet Corbin and Anselm Strauss. 1990. Grounded Theory Research: Procedures, Canons and Evaluative Criteria. Zeitschrift für Soziologie 19, 6: 515.
5. The Sage Developers. SageMath, the Sage Mathematics Software System (Version x.y.z).
6. Danyel Fisher, Badrish Chandramouli, Robert DeLine, Jonathan Goldstein, Andrei Aron, Mike Barnett, John C. Platt, James F. Terwilliger, and John Wernsing. 2014. Tempe: an interactive data science environment for exploration of temporal and streaming data. Tech. Rep. MSR-TR-2014--148.
7. Apache Software Foundation. 2017. Apache Zeppelin 0.7.0. Retrieved from https://zeppelin.apache.org/
8. Maik Riechert. 2016. Repairing Bad Pixels. Retrieved January 6, 2018 from https://github.com/letmaik/rawpynotebooks/blob/master/bad-pixel-repair/bad-pixelrepair.ipynb
9. Philip Jia Guo. 2012. Software tools to facilitate research programming. Ph.D. Dissertation. Stanford University.
10. Philip J. Guo and Margo I. Seltzer. 2012. Burrito: Wrapping your lab notebook in computational infrastructure. In Proceedings of the 4th USENIX Workshop on the Theory and Practice of Provenance.
11. Charles Hill, Rachel Bellamy, Thomas Erickson, and Margaret Burnett. 2016. Trials and tribulations of developers of intelligent systems: A field study. In Visual Languages and Human-Centric Computing (VL/HCC), 2016 IEEE Symposium on, 162--170.
12. Scott E. Hudson, Roy Rodenstein, and Ian Smith. 1997. Debugging Lenses: A New Class of Transparent Tools for User Interface Debugging. In Proceedings of the 10th Annual ACM Symposium on User Interface Software and Technology (UIST '97), 179--187.
13. Wolfram Research Inc. Mathematica, Version 11.2.
14. Mary Beth Kery, Amber Horvath, and Brad A. Myers. 2017. Variolite: Supporting Exploratory Programming by Data Scientists. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '17), 1265--1276.
15. Clemens Nylandsted Klokmose and Pär-Ola Zander. 2010. Rethinking Laboratory Notebooks. In Proceedings of COOP 2010. Springer, London, 119-- 139.
16. Thomas Kluyver, Benjamin Ragan-Kelley, Fernando Pérez, Brian E. Granger, Matthias Bussonnier, Jonathan Frederic, Kyle Kelley, Jessica B. Hamrick, Jason Grout, Sylvain Corlay, and Others. 2016. Jupyter Notebooks-a publishing format for reproducible computational workflows. In ELPUB, 87--90.
17. Donald Ervin Knuth. 1984. Literate programming. Computer Journal 27, 2: 97--111.
18. Timothy C. Lethbridge, Janice Singer, and Andrew Forward. 2003. How software engineers use documentation: The state of the practice. IEEE Software 20, 6: 35--39.
19. Robert C. Martin. 2009. Clean code: a handbook of agile software craftsmanship. Pearson Education.
20. Gerard Oleksik, Natasa Milic-Frayling, and Rachel Jones. 2014. Study of Electronic Lab Notebook Design and Practices That Emerged in a Collaborative Scientific Environment. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '14), 120--133.
21. David Lorge Parnas. 1994. Software aging. In Proceedings of the 16th international conference on Software engineering, 279--287.
22. Kayur Patel. 2010. Lowering the barrier to applying machine learning. In Adjunct proceedings of the 23nd annual ACM symposium on User interface software and technology, 355--358.
23. Kayur Patel, James Fogarty, James A. Landay, and Beverly Harrison. 2008. Investigating statistical machine learning as a tool for software development. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 667--676.
24. Fernando Pérez and Brian E. Granger. 2007. IPython: a System for Interactive Scientific Computing. Computing in Science and Engineering 9, 3: 21--29.
25. Fernando Perez and Brian E. Granger. 2015. Project Jupyter: Computational Narratives as the Engine of Collaborative Data Science. Project Jupyter Blog. Retrieved from http://blog.jupyter.org/2015/07/07/project-jupytercomputational-narratives-as-the-engine-ofcollaborative-data-science/
26. Helen Shen. 2014. Interactive notebooks: Sharing the code. Nature 515, 7525: 151.
27. Sruti Srinivasa Ragavan, Sandeep Kaur Kuttal, Charles Hill, Anita Sarma, David Piorkowski, and Margaret Burnett. 2016. Foraging Among an Overabundance of Similar Variants. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), 3509--3521.
28. Jean-Luc R. Stevens, Marco Elver, and James A. Bednar. 2013. An automated and reproducible workflow for running and analyzing neural simulations using Lancet and IPython Notebook. Frontiers in neuroinformatics 7.
29. Aurélien Tabard, Wendy E. Mackay, and Evelyn Eastmond. 2008. From Individual to Collaborative: The Evolution of Prism, a Hybrid Laboratory Notebook. In Proceedings of the 2008 ACM Conference on Computer Supported Cooperative Work (CSCW '08), 569--578.
30. Greg Wilson. 2006. Software carpentry: getting scientists to write better code by making them more productive. Computing in science & engineering 8, 6: 66--69.
31. Yihui Xie. 2014. knitr: a comprehensive tool for reproducible research in R. Implement Reprod Res 1: 20.
32. Youngseok Yoon, Brad A. Myers, and Sebon Koo. 2013. Visualization of fine-grained code change history. In Visual Languages and Human-Centric Computing (VL/HCC), 2013 IEEE Symposium on, 119--126.
33. 12/2015. Jupyter Notebook 2015 UX Survey Results. Jupyter Project Github Repository. Retrieved from https://github.com/jupyter/surveys/blob/master/survey s/2015--12-notebookux/analysis/report_dashboard.ipynb
34. 2013. Databricks. Retrieved from https://databricks.com/

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

• Software and its engineering

  • Software notations and tools

    • Development frameworks and environments