skip to main content
article

Failures in complex systems: case studies, causes, and possible remedies

Published: 01 June 2007 Publication History

Abstract

Computers are the pervasive technology of our time. As computers become critically tied to human life, it also becomes more important that interactions with them are under control. They are no longer a novelty, but are integrated into the fabric of our world, performing both high and low-level tasks. That is, computers may be used to eliminate heavy, redundant work and more sophisticated machines have been deployed to perform remote surgery or detect subterranean landmines in repopulated civilian areas. The increasing importance of computers in our lives means that it is essential that the design of computer systems incorporates techniques that can ensure reliability, safety, and security. This paper will examine technological mishaps involving the use of computers. This review will include notorious software bugs that have affected finance, communication, transit, defense, health and medicine and other systems or industries. The sequences and etiology of these "accidents" will be discussed as well as how catastrophes may be avoided in the future through lessons and practices based on artificial intelligence research.

References

[1]
Perrow Charles, Normal Accidents: Living with High-Risk Technologies, Basic Books, NY, 1984.
[2]
Reason J. Human Error. Cambridge, Cambridge University Press; 1990.
[3]
Gornick CC, Hauser RG, Almquist AK, Maron BJ. Unpredictable implantable cardioverter-defibrillator pulse generator failure due to electrical overstress causing sudden death in a young high-risk patient with hypertrophic cardiomyopathy. Heart Rhythm. 2005 Jul;2(7):681--3.
[4]
Hauser RG, Maron BJ, Lessons from the failure and recall of an implantable cardioverter-defibrillator. Circulation. 2005 Sep 27;112(13):2040--2. Epub 2005 Sep 19.
[5]
Neumann, Peter G., Computer Related Risks, New York: Addison Wesley (1995).
[6]
Kohn L. T., Corrigan J. M. & Donaldsen M. S. (Eds.) (1999). To Err is Human: Building a Safer Health System. Washington D.C.: National Academies Press.
[7]
Euler, E. E., Jolly, S. D., and Curtis, H. H. "The Failures of the Mars Climate Orbiter and Mars Polar Lander: A Perspective from the People Involved." Proceedings of Guidance and Control 2001, American Astronautical Society, paper AAS 01--074, 2001.
[8]
NASA, Mars Climate Orbiter Mishap Investigation Board Phase I Report. November 10, 1999.
[9]
Mars Polar Lander. NSSDC Master Catalog: Spacecraft. Retrieved on Feb 7, 2007.
[10]
United States of America, General Accounting Office, Patriot missile defense: Software problem led to system failure at Dhahran, Saudi Arabia. Technical Report of the U.S. General Accounting Office, GAO/IMTEC-92-26, GAO, 1992.
[11]
United States of America, Department of Defense, "Formal Investigation of the Circumstances Surrounding the Downing of Iran Air Flight 655 on 3 July, 1998", http://www.dod.mil/pubs/foi/reading_room/172.pdf,
[12]
Larry Seese, AT&T's director of technology development.
[13]
Brennan, T. A., Leape, L. L. & Laird, N. (1996). Incidence of Adverse Events and Negligence in Hospitalized Patients. New England Journal of Medicine, 32(5), 5--28.
[14]
Bates DW, Reducing the Frequency of Errors in Medicine using Information Technology, Journal of the American Medical Informatics Association: JAMIA, 8:299--308, July --- Aug 2001.
[15]
FDA Preliminary Public Health Notification*: Guidant VENTAK PRIZM® 2 DR and CONTAK RENEWAL® Implantable Cardioverter Defibrillators http://www.fda.gov/cdrh/safety/071405-guidant.html
[16]
Meier, B., (2005, May 25). Maker of Heart Device Kept Flaw From Doctors. Nw York Times on the Web {Online}. Retrieved January 19, 2007, from http://query.nytimes.com/gst/fullpage.html?res=9803E4D71239F937A15756C0A9639C8B63&sec=health&spon=&pagewanted=print
[17]
Leveson, Nancy G., SAFEWARE System safety and Computers: a Guide to Preventing Accidents and Loses Caused by Technology, Addison Wesley Publishing Company Inc. (1995).
[18]
Leveson, Nancy G. and Clark S. Turner. 1992. "An Investigation of the Therac-25 Accidents". Technical Report #92--108. Department of Information and Computer Science, University of California, Irvine.
[19]
Roberts, K, Risk, Regulation, Litigation and Organizational Issues in Safety High-Hazard Industries, position paper for Workshop on Organizational Analysis in High Hazard Production Systems: An Academy/Industry Dialogue, MIT Endicott House, NSF Grant No. 9510883-SBR, April 15--18, 1997.
[20]
Sagan, S. D., The Limits of Safety, Princeton: Princeton University Press, 1993.
[21]
Kopec, D. & Michie, D. Mismatch between machine representations and human concepts: dangers and remedies. Report to the EEC under subprogram FAST (Forecasting and Assessment in Science and Technology), Brussels, Belgium, 1992.

Cited By

View all
  • (2021) HUMAN FACTORS AND ERGONOMICS IN DESIGN OF A 3 : AUTOMATION, AUTONOMY, AND ARTIFICIAL INTELLIGENCE HANDBOOK OF HUMAN FACTORS AND ERGONOMICS10.1002/9781119636113.ch52(1385-1416)Online publication date: 13-Aug-2021
  • (2016)Requirements engineering for safety-critical systemsInformation and Software Technology10.1016/j.infsof.2016.04.00275:C(71-89)Online publication date: 1-Jul-2016
  • (2015)The Effect of Moral Intensity on it Employees' Bad News ReportingJournal of Computer Information Systems10.1080/08874417.2015.1164576655:3(1-10)Online publication date: 10-Dec-2015
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM SIGCSE Bulletin
ACM SIGCSE Bulletin  Volume 39, Issue 2
June 2007
158 pages
ISSN:0097-8418
DOI:10.1145/1272848
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 June 2007
Published in SIGCSE Volume 39, Issue 2

Check for updates

Author Tags

  1. complex systems
  2. human errors
  3. medical errors
  4. safety-critical systems
  5. software bugs
  6. software engineering
  7. technical disasters

Qualifiers

  • Article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)18
  • Downloads (Last 6 weeks)1
Reflects downloads up to 19 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2021) HUMAN FACTORS AND ERGONOMICS IN DESIGN OF A 3 : AUTOMATION, AUTONOMY, AND ARTIFICIAL INTELLIGENCE HANDBOOK OF HUMAN FACTORS AND ERGONOMICS10.1002/9781119636113.ch52(1385-1416)Online publication date: 13-Aug-2021
  • (2016)Requirements engineering for safety-critical systemsInformation and Software Technology10.1016/j.infsof.2016.04.00275:C(71-89)Online publication date: 1-Jul-2016
  • (2015)The Effect of Moral Intensity on it Employees' Bad News ReportingJournal of Computer Information Systems10.1080/08874417.2015.1164576655:3(1-10)Online publication date: 10-Dec-2015
  • (2014)Automatic Numerical Analysis Based on Infinite-Precision Arithmetic2014 Eighth International Conference on Software Security and Reliability10.1109/SERE.2014.35(216-224)Online publication date: Jun-2014
  • (2013)High-Energy PhysicsHandbook of Signal Processing Systems10.1007/978-1-4614-6859-2_5(135-169)Online publication date: 10-May-2013
  • (2012)Performance Evaluation of Proactive Peer-to-Peer File-Sharing Application for Releasing Network Congestion of Backbone Network in an IntranetJournal of Software Engineering and Applications10.4236/jsea.2012.5807005:08(613-620)Online publication date: 2012
  • (2012)IntroductionImproving Software Testing10.1007/978-3-642-27464-0_1(1-9)Online publication date: 3-Feb-2012
  • (2010)High-Energy PhysicsHandbook of Signal Processing Systems10.1007/978-1-4419-6345-1_8(179-211)Online publication date: 16-Jul-2010
  • (2009)Disseminating Content through a Peer-to-Peer Infrastructure in a Disaster Zone2009 International Conference on Management and Service Science10.1109/ICMSS.2009.5303104(1-6)Online publication date: Sep-2009
  • (2009)Generating the Semantic Containers for the Query Interfaces of Deep Web2009 International Conference on Management and Service Science10.1109/ICMSS.2009.5302641(1-4)Online publication date: Sep-2009

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media