High Costs and Small Benefits: A Field Study of How Users Experience Operating System Upgrades

ABSTRACT

Users must manage frequent software and operating system upgrades across multiple computing devices. While current research focuses primarily on the security aspect, we investigate the user's perspective of upgrading software. Our first study (n=65) found that users delay major upgrades by an average of 80 days. We then ran a field study (n=14), beginning with in-depth observations during an operating system upgrade, followed by a four-week diary study. Very few participants prepared for upgrades (e.g., backing up files), and over half had negative reactions to the upgrade process and other changes (e.g., bugs, lost settings, unwanted features). During the upgrade process, waiting times were too long, feedback was confusing or misleading, and few had clear mental models of what was happening. Users almost never mentioned security as a concern or reason for upgrading. By contrast, interviews (n=3) with technical staff responsible for one organization's upgrades focused only on security and licensing, not user interface changes. We conclude with recommendations to improve the user's upgrade experience.

References

1. Ron Amadeo. 2016. Android N borrows Chrome OS code for "seamless" update installation | Ars Technica. http: //arstechnica.com/gadgets/2016/05/android-n-borrows-c hrome-os-code-for-seamless-update-installation/. (May 2016). (Accessed on August 2, 2016).
2. Apple. 2016. Get help with over-the-air iOS updates Apple Support. https://support.apple.com/en-us/HT201435. (June 2016). (Accessed on August 2, 2016).
3. Harry Brignull. 2013. Dark Patterns: inside the interfaces designed to trick you | The Verge. http://www.theverge.com/2013/8/29/4640308/dark-patte rns-inside-the-interfaces-designed-to-trick-you. (Aug. 2013). (Accessed on July 28, 2016).
4. Brad Chacos. 2016. Forced Windows 10 upgrades push users to dangerously disable Windows Update. http://www.pcworld.com/article/3075729/windows/fearin g-forced-windows-10-upgrades-users-are-disabling-cri tical-updates-at-their-own-risk.html. (May 2016). (Accessed on August 1, 2016).
5. David Coburn. 2014. How Apple's OS X Yosemite tracks you. https://www.washingtonpost.com/posttv/business/t echnology/how-apples-os-x-yosemite-tracks-you/2014/ 10/22/66df4386--59f1--11e4--9d6c-756a229d8b18_video.html. (Oct. 2014). (Accessed on August 8, 2016).
6. Shane Dingman. 2016. Considering updating to Windows 10? The choice may be out of your hands - The Globe and Mail. http://www.theglobeandmail.com/technology/t ech-news/considering-updating-to-windows-10-the-choic e-may-be-out-of-your-hands/article28533377/. (Feb. 2016). (Accessed on August 1, 2016).
7. Thomas Duebendorfer and Stefan Frei. 2009. Why silent updates boost security. TIK, ETH Zurich, Tech. Rep 302 (2009).
8. Kevin Dunn. 2004. Automatic update risks: can patching let a hacker in Network Security 2004, 7 (2004), 5--8.
9. Michael Fagan, Mohammad Maifi Hasan Khan, and Ross Buck. 2015. A study of users' experiences and beliefs about software update messages. Computers in Human Behavior 51 (2015), 504--519.
10. Michael Fagan, Mohammad Maifi Hasan Khan, and Nhan Nguyen. 2015. How does this message make you feel? A study of user perspectives on software update/warning message design. Human-centric Computing and Information Sciences 5, 1 (2015), 1.
11. Marvin Fleischmann, Miglena Amirpur, Tillmann Grupp, Alexander Benlian, and Thomas Hess. 2016. The role of software updates in information systems continuance-An experimental study from a user perspective. Decision Support Systems 83 (2016), 83--96.
12. Marita Franzke and John Rieman. 1993. Natural training wheels: Learning and transfer between two versions of a computer application. In Human Computer Interaction. Springer, 315--328.
13. Barbara L Fredrickson and Daniel Kahneman. 1993. Duration neglect in retrospective evaluations of affective episodes. Journal of personality and social psychology 65, 1 (1993), 45.
14. Stefan Frei, Thomas Duebendorfer, and Bernhard Plattner. 2008. Firefox (in) security update dynamics exposed. ACM SIGCOMM Computer Communication Review 39, 1 (2008), 16--22.
15. Samuel Gibbs. 2016. Windows 10: Microsoft launches intrusive full-screen upgrade reminder | Technology | The Guardian. https://www.theguardian.com/technology/2016/ jul/04/microsoft-windows-10-full-screen-upgrade-notif ication-pop-up-reminder. (July 2016). (Accessed on August 1, 2016).
16. Rob Griffiths. 2016. A useless analysis of OS X release dates | The Robservatory. http://robservatory.com/a-use less-analysis-of-os-x-release-dates/. (July 2016). (Accessed on August 1, 2016).
17. Iulia Ion, Rob Reeder, and Sunny Consolvo. 2015. "... no one can hack my mind": Comparing Expert and Non-Expert Security Practices. In Eleventh Symposium on Usable Privacy and Security (SOUPS 2015). 327--346.
18. Daniel Kahneman, Barbara L Fredrickson, Charles A Schreiber, and Donald A Redelmeier. 1993. When more pain is preferred to less: Adding a better end. Psychological science 4, 6 (1993), 401--405.
19. Moazzam Khan, Zehui Bi, and John A Copeland. 2012. Software updates as a security metric: Passive identification of update trends and effect on machine infection. In MILCOM 2012--2012 IEEE Military Communications Conference. IEEE, 1--6.
20. Huoy Min Khoo and Daniel Robey. 2007. Deciding to upgrade packaged software: a comparative case study of motives, contingencies and dependencies. European Journal of Information Systems 16, 5 (2007), 555--567.
21. Huoy Min Khoo, Daniel Robey, and Srinivasan Venkoba Rao. 2011. An exploratory study of the impacts of upgrading packaged software: a stakeholder perspective. Journal of Information technology 26, 3 (2011), 153--169.
22. Wendy E. Mackay. 1991. Triggers and Barriers to Customizing Software. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '91). ACM, NY, NY, USA, 153--160. DOI: http://dx.doi.org/10.1145/108844.108867
23. Wendy E Mackay. 2000. Responding to cognitive overload: Co-adaptation between users and technology. Intellectica 30, 1 (2000), 177--193.
24. Andrew Martonik. 2015. Hey Google, don't just change fundamental parts of my launcher without warning | Android Central. http://www.androidcentral.com/googl e-dont-just-change-my-launcher-without-warning. (September 2015). (Accessed on September, 10 2016).
25. Joanna McGrenere, Ronald M. Baecker, and Kellogg S. Booth. 2002. An Evaluation of a Multiple Interface Design Solution for Bloated Software. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '02). ACM, NY, NY, USA, 164--170. DOI:http://dx.doi.org/10.1145/503376.503406
26. Joanna McGrenere and Gale Moore. 2000. Are we all in the same" bloat"?. In Graphics interface, Vol. 2000. 187--196.
27. Microsoft. 2016. Upgrade to Windows 10: FAQ. https://support.microsoft.com/en-us/help/12435/window s-10-upgrade-faq. (July 2016). (Accessed on August 2, 2016).
28. Andreas Möller, Florian Michahelles, Stefan Diewald, Luis Roalter, and Matthias Kranz. 2012. Update behavior in app markets and security implications: A case study in google play. In Proc. of the 3rd Intl. Workshop on Research in the Large. Held in Conjunction with Mobile HCI. Citeseer, 3--6.
29. Brad A. Myers. 1985. The Importance of Percent-done Progress Indicators for Computer-human Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '85). ACM, NY, NY, USA, 11--17. DOI: http://dx.doi.org/10.1145/317456.317459
30. Antonio Nappa, Richard Johnson, Leyla Bilge, Juan Caballero, and Tudor Dumitras. 2015. The attack of the clones: a study of the impact of shared code on vulnerability patching. In 2015 IEEE Symposium on Security and Privacy. IEEE, 692--708.
31. Jakob Nielsen. 1995. 10 Heuristics for User Interface Design. https: //www.nngroup.com/articles/ten-usability-heuristics/. (Jan. 1995). (Accessed on August 12, 2016).
32. Donald A Norman. 1983. Some observations on mental models. Mental models 7, 112 (1983), 7--14.
33. Robert Plutchik. 2001. The Nature of Emotions Human emotions have deep evolutionary roots, a fact that may explain their complexity and provide tools for clinical practice. American Scientist 89, 4 (2001), 344--350.
34. Jonathan Posner, James A Russell, and Bradley S Peterson. 2005. The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology. Development and psychopathology 17, 03 (2005), 715--734.
35. John Siracusa. 2014. OS X 10.10 Yosemite: The Ars Technica Review | Ars Technica. http://arstechnica.co m/apple/2014/10/os-x-10--10/9/#spotlight. (Oct. 2014). (Accessed on August 1, 2016).
36. Ashkan Soltani and Craig Timberg. 2014. Apple's Mac computers can automatically collect your location information - The Washington Post. https://www.washingtonpost.com/news/the-switch/wp /2014/10/20/apples-mac-computers-can-automatically-c ollect-your-location-information/. (Oct. 2014). (Accessed on August 1, 2016).
37. Franck Tétard and Mikael Collan. 2009. Lazy user theory: A dynamic model to understand user selection of products and services. In System Sciences, 2009. HICSS'09. 42nd Hawaii International Conference on. IEEE, 1--9.
38. Yuan Tian, Bin Liu, Weisi Dai, Blase Ur, Patrick Tague, and Lorrie Faith Cranor. 2015. Supporting Privacy-Conscious App Update Decisions with User Reviews. In Proceedings of the 5th Annual ACM CCS Workshop on Security and Privacy in Smartphones and Mobile Devices. ACM, 51--61.
39. Kami Vaniea and Yasmeen Rashidi. 2016. Tales of Software Updates: The Process of Updating Software. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, NY, NY, USA, 3215--3226. DOI: http://dx.doi.org/10.1145/2858036.2858303
40. Kami E. Vaniea, Emilee Rader, and Rick Wash. 2014. Betrayed by Updates: How Negative Experiences Affect Future Security. In Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems (CHI '14). ACM, NY, NY, USA, 2671--2674. DOI: http://dx.doi.org/10.1145/2556288.2557275
41. Rick Wash, Emilee J Rader, Kami Vaniea, and Michelle Rizor. 2014. Out of the Loop: How Automated Software Updates Cause Unintended Security Consequences.. In SOUPS. 89--104.
42. Lance Whitney. 2016. Microsoft's Windows 10 upgrades are getting even more sneaky-pushy - CNET. http://www.cnet.com/news/microsoft-windows-10-upgrade s-get-more-sneaky-pushy/. (May 2016). (Accessed on August 1, 2016).
43. Timothy D Wilson and Daniel T Gilbert. 2005. Affective forecasting knowing what to want. Current Directions in Psychological Science 14, 3 (2005), 131--134.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

• Security and privacy

• Social and professional topics

  • Computing / technology policy

    • Computer crime

  • Professional topics

    • Management of computing and information systems

      • Software management

        • Software maintenance

• Software and its engineering

  • Software creation and management

    • Software post-development issues