Fabian Fagerholm
ABSTRACT
Development of software-intensive products and services increasingly occurs by continuously deploying product or service increments, such as new features and enhancements, to customers. Product and service developers need to continuously find out what customers want by direct customer feedback and observation of usage behaviour, rather than indirectly through up-front business analyses. This paper examines the preconditions for setting up an experimentation system for continuous customer experiments. It describes the building blocks required for such a system. An initial model for continuous experimentation is analytically derived from prior work. The model is then matched against empirical case study findings from a startup company and adjusted. Building blocks for a continuous experimentation system and infrastructure are presented. A suitable experimentation system requires at least the ability to release minimum viable products or features with suitable instrumentation, design and manage experiment plans, link experiment results with a product roadmap, and manage a flexible business strategy. The main challenges are proper and rapid design of experiments, advanced instrumentation of software to collect, analyse, and store relevant data, and the integration of experiment results in both the product development cycle and the software development process.
- R. J. Adams, B. Evans, and J. Brandt. Creating Small Products at a Big Company: Adobe’s Pipeline Innovation Process. In CHI’13 Extended Abstracts on Human Factors in Computing Systems, pages 2331–2332. ACM, 2013. Google Scholar
Digital Library
- V. Basili, J. Heidrich, M. Lindvall, J. Münch, M. Regardie, D. Rombach, C. Seaman, and A. Trendowicz. GQM + Strategies: A comprehensive methodology for aligning business strategies with software measurement. In Proceedings of the DASMA Software Metric Congress (MetriKon 2007): Magdeburger Schriften zum Empirischen Software Engineering, pages 253–266, 2007.Google Scholar
- K. Beck and C. Andres. Extreme programming explained: embrace change. Addison-Wesley Professional, 2004. Google ScholarDigital Library
- J. Bosch. Building Products as Innovation Experiment Systems. In Software Business, pages 27–39. Springer, 2012.Google Scholar
- J. Bosch, H. Holmström Olsson, J. Björk, and J. Ljungblad. The Early Stage Software Startup Development Model: A Framework for Operationalizing Lean Principles in Software Startups. In Lean Enterprise Software and Systems, pages 1–15. Springer, 2013.Google Scholar
- T. Dybå and T. Dingsøyr. Empirical studies of agile software development: A systematic review. Information and software technology, pages 833–859, 2008. Google ScholarDigital Library
- F. Fagerholm, N. Oza, and J. Münch. A platform for teaching applied distributed software development: The ongoing journey of the Helsinki software factory. In 3rd International Workshop on Collaborative Teaching of Globally Distributed Software Development (CTGDSD), pages 1–5, 2013.Google ScholarCross Ref
- A. R. Hevner, S. T. March, J. Park, and S. Ram. Design Science in Information Systems Research. MIS Quarterly, 28(1):75–105, 2004. Google ScholarDigital Library
- H. Holmström Olsson, H. Alahyari, and J. Bosch. Climbing the “Stairway to Heaven” – A Mulitiple-Case Study Exploring Barriers in the Transition from Agile Development towards Continuous Deployment of Software. 39th EUROMICRO Conference on Software Engineering and Advanced Applications, pages 392–399, 2012. Google ScholarDigital Library
- R. Kohavi, A. Deng, B. Frasca, R. Longbotham, T. Walker, and Y. Xu. Trustworthy Online Controlled Experiments: Five Puzzling Outcomes Explained. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 786–794, New York, NY, USA, 2012. ACM. Google ScholarDigital Library
- C. Larman and V. R. Basili. Iterative and incremental developments. a brief history. IEEE Computer, pages 47–56, 2003. Google ScholarDigital Library
- B. May. Applying Lean Startup: An Experience Report – Lean & Lean UX by a UX Veteran: Lessons Learned in Creating & Launching a Complex Consumer App. In Agile Conference (AGILE) 2012, pages 141––147. IEEE, 2012. Google ScholarDigital Library
- J. Münch, F. Fagerholm, P. Johnson, J. Pirttilahti, J. Torkkel, and J. Järvinen. Creating Minimum Viable Products in Industry-Academia Collaborations. In Proceedings of the Lean Enterprise Software and Systems Conference (LESS 2013, Galway, Ireland, December 1-4), pages 137–151. Springer Berlin Heidelberg, 2013.Google ScholarCross Ref
- J. Münch, F. Fagerholm, P. Kettunen, M. Pagels, and J. Partanen. Experiences and Insights from Applying GQM +Strategies in a Systems Product Development Organisation. In Proceedings of the 39th EUROMICRO Conference on Software Engineering and Advanced Applications (SEAA 2013), 2013. Google ScholarDigital Library
- J. Nieters and A. Pande. Rapid Design Labs: A Tool to Turbocharge Design-led Innovation. Interactions, pages 72–77, 2012. Google ScholarDigital Library
- T. ¯ Ono. Toyota production system: beyond large-scale production. Productivity press, 1988.Google Scholar
- M. Poppendieck. Lean software development: an agile toolkit. Addison-Wesley Professional, 2003. Google ScholarDigital Library
- M. Poppendieck and M. A. Cusumano. Lean Software Development: A Tutorial. IEEE Software, pages 26–32, 2012. Google ScholarDigital Library
- E. Ries. The Lean Startup: How Today’s Entrepreneurs Use Continuous Innovation To Create Radically Successful Businesses. Crown Business, 2011.Google Scholar
- K. Schwaber and M. Beedle. Agile software development with Scrum. Prentice Hall Upper Saddle River, 2002. Google ScholarDigital Library
- D. Ståhl and J. Bosch. Modeling Continuous Integration Practice Di fferences in Industry Software Development. Journal of Systems and Software, pages 48–59, 2014. Google ScholarDigital Library
- J. Stapleton. DSDM: Business Focussed Development. Pearson Education, 2003.Google Scholar
- A. Steiber and S. Alänge. A Corporate System for Continuous Innovation: The case of Google Inc. European Journal of Innovation Management, pages 243–264, 2013.Google ScholarCross Ref
- J. E. van Aken. Management Research Based on the Paradigm of the Design Sciences: The Quest for Field-Tested and Grounded Technological Rules. Journal of Management Studies, 41(2):219–246, 2004.Google ScholarCross Ref
Index Terms
- Building blocks for continuous experimentation
Recommendations
On the Understanding of Experimentation Usage in Light of Lean Startup in Software Development Context
EASE '20: Proceedings of the 24th International Conference on Evaluation and Assessment in Software EngineeringExperimentation can be used to validate software solutions, reducing waste on efforts and costs that would be spent had the solution not been validated from the beginning. Aware of experimentation benefits, a methodology named as Lean Startup have ...
"Leagile" software development
In recent years there has been a noticeable shift in attention from those who use agile software development toward lean software development, often labelled as a shift "from agile to lean". However, the reality may not be as simple or linear as this ...
The Combination of Agile and Lean in Software Development: An Experience Report Analysis
AGILE '11: Proceedings of the 2011 Agile ConferenceThere has been a noticeable focus shift from agile methods such as extreme Programming (XP) and Scrum to lean software development in the last several years, which is indicated as â from agile to leanâ . However, the reality may not be as simple or ...
Comments