Repurposing citizen science games as software tools for professional scientists

ABSTRACT

Scientific software is often developed with professional scientists in mind, resulting in complex tools with a steep learning curve. Citizen science games, however, are designed for citizen scientists---members of the general public. These games maintain scientific accuracy while placing design goals such as usability and enjoyment at the forefront. In this paper, we identify an emerging use of game-based technology, in the repurposing of citizen science games to be software tools for professional scientists in their work. We discuss our experience in two such repurposings: Foldit, a protein folding and design game, and Eyewire, a web-based 3D neuron reconstruction game. Based on this experience, we provide evidence that the software artifacts produced for citizen science can be useful for professional scientists, and provide an overview of key design principles we found to be useful in the process of repurposing.

References

1. Paul D. Adams, Pavel V. Afonine, Gábor Bunkóczi, Vincent B. Chen, Ian W. Davis, Nathaniel Echols, Jeffrey J. Headd, Li-Wei Hung, Gary J. Kapral, Ralf W. Grosse-Kunstleve, Airlie J. McCoy, Nigel W. Moriarty, Robert Oeffner, Randy J. Read, David C. Richardson, Jane S. Richardson, Thomas C. Terwilliger, and Peter H. Zwart. 2010. PHENIX: a comprehensive Python-based system for macromolecular structure solution. Acta Crystallographica Section D: Biological Crystallography 66, 2 (Feb. 2010), 213--221.
2. Manda Banerji, Ofer Lahav, Chris J. Lintott, Filipe B. Abdalla, Kevin Schawinski, Steven P. Bamford, Dan Andreescu, Phil Murray, M. Jordan Raddick, Anze Slosar, Alex Szalay, Daniel Thomas, and Jan Vandenberg. 2010. Galaxy Zoo: reproducing galaxy morphologies via machine learning. Monthly Notices of the Royal Astronomical Society 406, 1 (2010), 342--353.
3. Jonathan Barone, Colin Bayer, Rowan Copley, Nova Barlow, Matthew Burns, Sundipta Rao, Georg Seelig, Zoran Popović, Seth Cooper, and Nanocrafter Players. 2015. Nanocrafter: design and evaluation of a DNA nanotechnology game. In Proceedings of the 10th International Conference on the Foundations of Digital Games.
4. Amaury Belin and Yannick Prié. 2012. DIAM: towards a model for describing appropriation processes through the evolution of digital artifacts. In Proceedings of the Designing Interactive Systems Conference (DIS '12). ACM, New York, NY, USA, 645--654.
5. Dylan Alexander Carlin, Ryan W. Caster, Xiaokang Wang, Stephanie A. Betzenderfer, Claire X. Chen, Veasna M. Duong, Carolina V. Ryklansky, Alp Alpekin, Nathan Beaumont, Harshul Kapoor, Nicole Kim, Hosna Mohabbot, Boyu Pang, Rachel Teel, Lillian Whithaus, Ilias Tagkopoulos, and Justin B. Siegel. 2016. Kinetic characterization of 100 glycoside hydrolase mutants enables the discovery of structural features correlated with kinetic constants. PLOS ONE 11, 1 (Jan. 2016),e0147596.
6. CCP Games. 2003. EVE Online. (2003).
7. Dennis Chao. 2001. Doom as an interface for process management. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '01). ACM, New York, NY, USA, 152--157.
8. Seth Cooper, Firas Khatib, Adrien Treuille, Janos Barbero, Jeehyung Lee, Michael Beenen, Andrew Leaver-Fay, David Baker, Zoran Popović, and Foldit Players. 2010. Predicting protein structures with a multiplayer online game. Nature 466, 7307 (Aug. 2010), 756--760.
9. Seth Cooper, Adrien Treuille, Janos Barbero, Andrew Leaver-Fay, Kathleen Tuite, Firas Khatib, Alex Cho Snyder, Michael Beenen, David Salesin, David Baker, Zoran Popović, and Foldit Players. 2010. The challenge of designing scientific discovery games. In Proceedings of the 5th International Conference on the Foundations of Digital Games (FDG '10). ACM, Monterey, California, 40--47.
10. Silvia Crivelli, Oliver Kreylos, Bernd Hamann, Nelson Max, and Wes Bethel. 2004. ProteinShop: a tool for interactive protein manipulation and steering. Journal of Computer-Aided Molecular Design 18, 4 (2004), 271--285.
11. W. L. DeLano. 2002. The PyMOL molecular graphics system. (2002).
12. Alan Dix. 2007. Designing for appropriation. In Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI...But Not As We Know It - Volume 2 (BCS-HCI '07). BCS Learning & Development Ltd., Swindon, UK, 27--30.
13. Christopher B. Eiben, Justin B. Siegel, Jacob B. Bale, Seth Cooper, Firas Khatib, Betty W. Shen, Foldit Players, Barry L. Stoddard, Zoran Popović, and David Baker. 2012. Increased Diels-Alderase activity through backbone remodeling guided by Foldit players. Nature Biotechnology 30, 2 (2012), 190--192.
14. Paul Emsley and Kevin Cowtan. 2004. Coot: model-building tools for molecular graphics. Acta Crystallographica Section D: Biological Crystallography 60, 12 (2004), 2126--2132.
15. Chiara Franzoni and Henry Sauermann. 2014. Crowd science: the organization of scientific research in open collaborative projects. Research Policy 43, 1 (Feb. 2014), 1--20.
16. Miroslaw Gilski, Maciej Kazmierczyk, Szymon Krzywda, Helena Zábranská, Seth Cooper, Zoran Popović, Firas Khatib, Frank DiMaio, James Thompson, David Baker, Iva Pichová, and Mariusz Jaskolski. 2011. High-resolution structure of a retroviral protease folded as a monomer. Acta Crystallographica Section D: Biological Crystallography D67, 11 (Nov. 2011), 907--914.
17. GlassLab. 2013. SimCityEDU. (2013).
18. Benjamin M. Good and Andrew I. Su. 2011. Games with a scientific purpose. Genome Biology 12, 12 (2011), 135.
19. Sydney R. Gordon, Elizabeth J. Stanley, Sarah Wolf, Angus Toland, Sean J. Wu, Daniel Hadidi, Jeremy H. Mills, David Baker, Ingrid Swanson Pultz, and Justin B. Siegel. 2012. Computational design of an α-gliadin peptidase. Journal of the American Chemical Society 134, 50 (Dec. 2012), 20513--20520.
20. Moritz Helmstaedter, Kevin L. Briggman, and Winfried Denk. 2011. High-accuracy neurite reconstruction for high-throughput neuroanatomy. Nature Neuroscience 14, 8 (Aug. 2011), 1081--1088.
21. Scott Horowitz, Brian Koepnick, Raoul Martin, Agnes Tymieniecki, Amanda A. Winburn, Seth Cooper, Jeff Flatten, David S. Rogawski, Nicole M. Koropatkin, Tsinatkeab T. Hailu, Neha Jain, Philipp Koldewey, Logan S. Ahlstrom, Matthew R. Chapman, Andrew P. Sikkema, Meredith A. Skiba, Finn P. Maloney, Felix R. M. Beinlich, Foldit Players, University of Michigan students, Zoran Popović, David Baker, Firas Khatib, and James C. A. Bardwell. 2016. Determining crystal structures through crowdsourcing and coursework. Nature Communications 7 (2016), 12549.
22. Narayanan Kasthuri, Kenneth Jeffrey Hayworth, Daniel Raimund Berger, Richard Lee Schalek, José Angel Conchello, Seymour Knowles-Barley, Dongil Lee, Amelio Vázquez-Reina, Verena Kaynig, Thouis Raymond Jones, Mike Roberts, Josh Lyskowski Morgan, Juan Carlos Tapia, H. Sebastian Seung, William Gray Roncal, Joshua Tzvi Vogelstein, Randal Burns, Daniel Lewis Sussman, Carey Eldin Priebe, Hanspeter Pfister, and Jeff William Lichtman. 2015. Saturated reconstruction of a volume of neocortex. Cell 162, 3 (July 2015), 648--661.
23. Alexander Kawrykow, Gary Roumanis, Alfred Kam, Daniel Kwak, Clarence Leung, Chu Wu, Eleyine Zarour, Luis Sarmenta, Mathieu Blanchette, Jérôme Waldispühl, and Phylo Players. 2012. Phylo: a citizen science approach for improving multiple sequence alignment. PLOS ONE 7, 3 (March 2012), e31362.
24. Firas Khatib, Frank DiMaio, Foldit Contenders Group, Foldit Void Crushers Group, Seth Cooper, Maciej Kazmierczyk, Miroslaw Gilski, Szymon Krzywda, Helena Zábranská, Iva Pichová, James Thompson, Zoran Popović, Mariusz Jaskolski, and David Baker. 2011. Crystal structure of monomeric retroviral protease solved by protein folding game players. Nature Structural and Molecular Biology 18, 10 (Sept. 2011), 1175--1177.
25. Jinseop S. Kim, Matthew J. Greene, Aleksandar Zlateski, Kisuk Lee, Mark Richardson, Srinivas C. Turaga, Michael Purcaro, Matthew Balkam, Amy Robinson, Bardia F. Behabadi, Michael Campos, Winfried Denk, H. Sebastian Seung, and EyeWirers. 2014. Space-time wiring specificity supports direction selectivity in the retina. Nature 509, 7500 (May 2014), 331--336.
26. Robert Kleffner, Jeff Flatten, Andrew Leaver-Fay, David Baker, Justin B. Siegel, Firas Khatib, and Seth Cooper. 2017. Foldit Standalone: a video game-derived protein structure manipulation interface using Rosetta. Bioinformatics (2017).
27. Belinda Lange, Sheryl Flynn, and Albert Rizzo. 2009. Initial usability assessment of off-the-shelf video game consoles for clinical game-based motor rehabilitation. Physical Therapy Reviews 14, 5 (Oct. 2009), 355--363.
28. Andrew Leaver-Fay, Michael Tyka, Steven M. Lewis, Oliver F. Lange, James Thompson, Ron Jacak, Kristian Kaufman, P. Douglas Renfrew, Colin A. Smith, Will Sheffler, Ian W. Davis, Seth Cooper, Adrien Treuille, Daniel J. Mandell, Florian Richter, Yih-En Andrew Ban, Sarel J. Fleishman, Jacob E. Corn, David E. Kim, Sergey Lyskov, Monica Berrondo, Stuart Mentzer, Zoran Popović, James J. Havranek, John Karanicolas, Rhiju Das, Jens Meiler, Tanja Kortemme, Jeffrey J. Gray, Brian Kuhlman, David Baker, and Philip Bradley. 2011. ROSETTA3: an object-oriented software suite for the simulation and design of macromolecules. Methods in Enzymology 487 (2011), 545--574.
29. Jeehyung Lee, Wipapat Kladwang, Minjae Lee, Daniel Cantu, Martin Azizyan, Hanjoo Kim, Alex Limpaecher, Sungroh Yoon, Adrien Treuille, Rhiju Das, and EteRNA Participants. 2014. RNA design rules from a massive open laboratory. Proceedings of the National Academy of Sciences 111, 6 (Feb. 2014), 2122--2127.
30. Edgar Luttmann, Daniel L. Ensign, Vishal Vaidyanathan, Mike Houston, Noam Rimon, Jeppe Øland, Guha Jayachandran, Mark Friedrichs, and Vijay S. Pande. 2009. Accelerating molecular dynamic simulation on the cell processor and Playstation 3. Journal of Computational Chemistry 30, 2 (Jan. 2009), 268--274.
31. Mojang. 2016. Minecraft: Education Edition. (2016).
32. Antônio Carlos A. Mól, Carlos Alexandre F. Jorge, and Pedro M. Couto. 2008. Using a game engine for VR simulations in evacuation planning. IEEE Computer Graphics and Applications 28, 3 (May 2008), 6--12.
33. Eric F. Pettersen, Thomas D. Goddard, Conrad C. Huang, Gregory S. Couch, Daniel M. Greenblatt, Elaine C. Meng, and Thomas E. Ferrin. 2004. UCSF Chimera-a visualization system for exploratory research and analysis. Journal of Computational Chemistry 25, 13 (Oct. 2004), 1605--1612.
34. M. Jordan Raddick, Georgia Bracey, Pamela L. Gay, Chris J. Lintott, Phil Murray, Kevin Schawinski, Alexander S. Szalay, and Jan Vandenberg. 2010. Galaxy Zoo: exploring the motivations of citizen science volunteers. Astronomy Education Review 9, 1 (Dec. 2010).
35. Ghanasyam Rallapalli, Fraxinus Players, Diane Go Saunders, Kentaro Yoshida, Anne Edwards, Carlos A. Lugo, Steve Collin, Bernardo Clavijo, Manuel Corpas, David Swarbreck, Matthew Clark, J. Allan Downie, Sophien Kamoun, Team Cooper, and Dan MacLean. 2015. Lessons from Fraxinus, a crowd-sourced citizen science game in genomics. eLife 4 (2015), e07460.
36. Ingmar H. Riedel-Kruse, Alice M. Chung, Burak Dura, Andrea L. Hamilton, and Byung C. Lee. 2011. Design, engineering and utility of biotic games. Lab on a Chip 11, 1 (2011), 14--22.
37. Carol A. Rohl, Charlie E. M. Strauss, Kira M. S. Misura, and David Baker. 2004. Protein structure prediction using Rosetta. Methods in Enzymology 383 (2004), 66--93.
38. Rachel Welles Shearer. 2009. OMNI: visualizing and editing large-scale volume segmentations of neuronal tissue. Master's. MIT.
39. Akihiko Shirai, Erik Geslin, and Simon Richir. 2007. WiiMedia: motion analysis methods and applications using a consumer video game controller. In Proceedings of the 2007 ACM SIGGRAPH Symposium on Video Games (Sandbox '07). ACM, New York, NY, USA, 133--140.
40. Mohd Fairuz Shiratuddin and Walid Thabet. 2002. Virtual office walkthrough using a 3D game engine. International Journal of Design Computing 4 (2002).
41. Justin B. Siegel, Amanda Lee Smith, Sean Poust, Adam J. Wargacki, Arren Bar-Even, Catherine Louw, Betty W. Shen, Christopher B. Eiben, Huu M. Tran, Elad Noor, Jasmine L. Gallaher, Jacob Bale, Yasuo Yoshikuni, Michael H. Gelb, Jay D. Keasling, Barry L. Stoddard, Mary E. Lidstrom, and David Baker. 2015. Computational protein design enables a novel one-carbon assimilation pathway. Proceedings of the National Academy of Sciences 112, 12 (March 2015), 3704--3709.
42. Brian L. Sullivan, Christopher L. Wood, Marshall J. Iliff, Rick E. Bonney, Daniel Fink, and Steve Kelling. 2009. eBird: a citizen-based bird observation network in the biological sciences. Biological Conservation 142, 10 (Oct. 2009), 2282--2292.
43. Mark C. Surles, Jane S. Richardson, David C. Richardson, and Fred P. Brooks. 1994. Sculpting proteins interactively: continual energy minimization embedded in a graphical modeling system. Protein Science 3 (1994), 198--210.
44. Valve Corporation. 2011. Learn With Portals. (2011).
45. Ashwin Vishwanathan, Kayvon Daie, Alexandro D. Ramirez, Jeff W. Lichtman, Emre R. F. Aksay, and H. Sebastian Seung. 2017. Electron microscopic reconstruction of functionally identified cells in a neural integrator. Current Biology 27, 14 (July 2017), 2137--2147.e3.
46. Luis von Ahn and Laura Dabbish. 2004. Labeling images with a computer game. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, Vienna, Austria, 319--326.
47. Jijun Wang, Michael Lewis, and Jeffrey Gennari. 2003. Emerging areas: urban operations and UCAVs: a game engine based simulation of the NIST urban search and rescue arenas. In Proceedings of the 35th Conference on Winter Simulation: Driving Innovation (WSC '03). Winter Simulation Conference, New Orleans, Louisiana, 1039--1045.
48. Andrew J. Westphal, Anna L. Butterworth, Christopher J. Snead, Nahide Craig, David Anderson, Steven M.Jones, Donald E. Brownlee, Richard Farnsworth, and Michael E. Zolensky. 2005. Stardust@home: a massively distributed public search for interstellar dust in the stardust interstellar dust collector.
49. Sean J. Wu, Christopher B. Eiben, John H. Carra, Ivan Huang, David Zong, Peixian Liu, Cindy T. Wu, Jeff Nivala, Josef Dunbar, Tomas Huber, Jeffrey Senft, Rowena Schokman, Matthew D. Smith, Jeremy H. Mills, Arthur M. Friedlander, David Baker, and Justin B. Siegel. 2011. Improvement of a potential anthrax therapeutic by computational protein design. Journal of Biological Chemistry 286, 37 (Sept. 2011), 32586--32592.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)