Documentation quality and time costs: A randomized controlled trial of structured entry versus dictation

Abstract

The Department of Veterans Affairs (VA) performs over 800,000 disability exams and distributes over $37 billion in disability benefits per year. VA developed and deployed a computer-based disability exam documentation system in order to improve exam report quality and timeliness. We conducted a randomized controlled trial comparing joint disability examinations supported by computerized templates to the examinations documented via dictation, to determine if the system met the intended goals or had unintended consequences. Consenting veterans were randomized to undergo exams documented using computerized templates or via dictation. We compared exam report quality, documentation time costs, encounter length, total time to fulfill an exam request with a finalized exam report, and veteran satisfaction. Computer-based templates resulted in disability exam reports that had higher quality scores (p. 0.042) and were returned to the requesting office faster than exam reports created via dictation (p. 0.02).

Documentation time and veteran satisfaction were similar for both the documentation techniques. Encounter length was significantly longer for the template group. Computer-based templates impacted the VA disability evaluation system by improving report quality scores and production time and lengthening encounter times. Oversight bodies have called for mandated use of computer-based templates nationwide. We believe mandates regarding use of health information technology should be guided by data regarding its positive and negative impacts.

References

1. Ahlers, P., Sullivan, R. J., Hammond, W. E., Walter, E. L., and Tolley, H. D. 1976. The cost of auditing outpatient records. South Med. J. 69, 1328--1330.
2. Apkon, M. and Singhaviranon, P. 2001. Impact of an electronic information system on physician workflow and data collection in the intensive care unit. Intensive Care Med. 27, 122--130.
3. Aronsky, D. and Haug, P. J. 2000. Assessing the quality of clinical data in a computer-based record for calculating the pneumonia severity index. J. Amer. Med. Inform. Assoc. 7, 55--65.
4. Ash, J. S., Berg, M., and Coiera, E. 2003. Some unintended consequences of information technology in health care: The nature of patient care informations system related errors. J. Amer. Med. Inform. Assoc. Preprint.
5. Barnett, G. O., Winickoff, R., Dorsey, J. L., Morgan, M. M., and Lurie, R. S. 1978. Quality assurance through automated monitoring and concurrent feedback using a computer-based medical information system. Med. Care 16, 962--970.
6. Barnett, G. O., Winickoff, R. N., Morgan, M. M., and Zielstorff, R. D. 1983. A computer-based monitoring system for follow-up of elevated blood pressure. Med Care 21, 400--409.
7. Bell, D. S. and Greenes, R. A. 1994. Evaluation of UltraSTAR: performance of a collaborative structured data entry system. In Proceedings of the Annual Symposium on Computer Applications Medical Care. 216--222.
8. Blake, R. and Mangiameli, P. 2009. The effects and interactions of data quality and problem complexity on data mining. In Proceedings of the 13th International Conference on Information Quality (ICIQ), 160--165.
9. Brown, S. H., Hardenbrook, S., Herrick, L., St. Onge, J., Bailey, K., and Elkin, P. L. 2001. Usability evaluation of the progress note construction set. In Proceedings of the AMIA Symposium. 76--80.
10. Brown, S. H., Lincoln, M. J., Groen, P. J., and Kolodner, R. M. 2003. VistA--U.S. Department of Veterans Affairs national-scale HIS. Int. J. Med. Inf. 69, 135--156.
11. Brown, S. H., Speroff, T., Fielstein, E. M., Bauer, B. A., Wahner-Roedler, D. L., Greevy, R., and Elkin, P. L. 2006. eQuality: Electronic quality assessment from narrative clinical reports. Mayo Clin. Proc. 81, 1472--1481.
12. Cimino, J. J., Johnson, S. B., Aguirre, A., Roderer, N., and Clayton, P. D. 1992. The MEDLINE button. In Proceedings of the Annual Symposium on Computer Applications Medical Care. 81--85.
13. Cimino, J. J., Patel, V. L., and Kushniruk, A. W. 2001. Studying the human-computer-terminology interface. J Amer. Med. Inform. Assoc. 8, 163--173.
14. Dawdy, M. R., Munter, D. W., and Gilmore, R. A. 1997. Correlation of patient entry rates and physician documentation errors in dictated and handwritten emergency treatment records. Amer. J. Emerg. Med. 15, 115--117.
15. Dollery, C. T., Beilin, L. J., Bulpitt, C. J., Coles, E. C., Johnson, B. F., Munro-Faure, A. D., and Turner, S. C. 1977. Initial care of hypertensive patients. Influence of different types of clinical records. British Heart J. 39, 181--185.
16. Duggan, A. K., Starfield, B., and Deangelis, C. 1990. Structured encounter form: The impact on provider performance and recording of well-child care. Pediatrics 85, 104--113.
17. Essin, D. J., Dishakjian, R., Deciutiis, V. L., Essin, C. D., and Steen, S. N. 1998. Development and assessment of a computer-based preanesthetic patient evaluation system for obstetrical anesthesia. J. Clin. Monit. Comput. 14, 95--100.
18. Evans, R. S., Pestotnik, S. L., Classen, D. C., Clemmer, T. P., Weaver, L. K., Orme, J. F., Jr., Lloyd, J. F., and Burke, J. P. 1998. A computer-assisted management program for antibiotics and other antiinfective agents. New. Eng. J. Med. 338, 232--238.
19. Fatzinger, P., Kammer, A., and Garrett, M. 1992. Development and use of preprinted forms and adhesive labels in medical record charting. J. Amer. Diet Assoc. 92, 982--985.
20. Fielstein, E. M., Brown, S. H., McBrine, C. S., Clark, T. K., Hardenbrook, S. P., and Speroff, T. 2006. The effect of standardized computer-guided templates on quality of VA disability exams. In Proceedings of the American Medical Information Association Fall Symposium. 249--253.
21. Geissbuhler, A. and Miller, R. A. 1996. A new approach to the implementation of direct care-provider order entry. In Proceedings of the AMIA Annual Fall Symposium. 689--693.
22. Gouveia-Oliveira, A., Raposo, V. D., Salgado, N. C., Almeida, I., Nobre-Leitao, C., and De Melo, F. G. 1991. Longitudinal comparative study on the influence of computers on reporting of clinical data. Endoscopy 23, 334--337.
23. Gray, J. E., Safran, C., Davis, R. B., Pompilio-Weitzner, G., Stewart, J. E., Zaccagnini, L., and Pursley, D. 2000. Baby CareLink: Using the internet and telemedicine to improve care for high-risk infants. Pediatrics 106, 1318--1324.
24. Hammond, K. W. and Munnecke, T. H. 1984. A computerized psychiatric treatment planning system. Hosp. Comm. Psych. 35, 160--163.
25. Hershberg, P. I., Goldfinger, S. E., Lemon, F. R., and Fessel, W. J. 1972. Medical record as index of quality of care. New. Eng. J. Med,. 286, 725--726.
26. Hiddema-Van De Wal, A., Smith, R. J., Van Der Werf, G. T., and Meyboom-De Jong, B. 2001. Towards improvement of the accuracy and completeness of medication registration with the use of an electronic medical record (EMR). Fam. Pract. 18, 288--291.
27. Humphreys, T., Shofer, F. S., Jacobson, S., Coutifaris, C., and Stemhagen, A. 1992. Preformatted charts improve documentation in the emergency department. Ann. Emerg. Med. 21, 534--540.
28. Jeans, W. D., Danton, R. M., and Kilburn, A. R. 1980. An assessment of a computerized reporting system (SIREP). British J. Radiol. 53, 421--427.
29. Johnson, K. B. and Cowan, J. 2002. Clictate: A computer-based documentation tool for guideline-based care. J. Med. Syst. 26, 47--60.
30. Johnson, K. B., Ravich, W. J., and Cowan, J. A., Jr. 2004. Brainstorming about next-generation computer-based documentation: an AMIA clinical working group survey. Int. J. Med. Inf. 73, 665--674.
31. Johnson, K. B., Serwint, J. R., Fagan, L. M., Thompson, R. E., and Wilson, M. H. 2005. Computer-based documentation: effect on parent and physician satisfaction during a pediatric health maintenance encounter. Arch. Pediatr. Adolesc. Med. 159, 250--254.
32. Kahn, C. E., Jr. and Huynh, P. N. 1996. Knowledge representation for platform-independent structured reporting. In Proceedings of the AMIA Annual Fall Symposium. 478--482.
33. Kiefer, V. F., Schwartz, R. J., and Jacobs, L. M. 1993. The effect of quality assurance on flight nurse documentation. Air Med. J. 1, 11--14.
34. Kuhn, K., Gaus, W., Wechsler, J. G., Janowitz, P., Tudyka, J., Kratzer, W., Swobodnik, W., and Ditschuneit, H. 1992. Structured reporting of medical findings: Evaluation of a system in gastroenterology. Methods Inf. Med. 31, 268--274.
35. Kuhn, K., Swobodnik, W., Johannes, R. S., Zemmler, T., Stange, E. F., Ditschuneit, H., and Classen, M. 1991. The quality of gastroenterological reports based on free text dictation: an evaluation in endoscopy and ultrasonography. Endoscopy 23, 262--264.
36. Ledley, R. S. 1966. Computer aids to medical diagnosis. J. Amer. Med. Assoc. 196, 933--943.
37. Lum, F., Schein, O., Schachat, A. P., Abbott, R. L., Hoskins, H. D., Jr., and Steinberg, E. P. 2000. Initial two years of experience with the AAO National Eyecare Outcomes Network (NEON) cataract surgery database. Ophthalmology 107, 691--697.
38. Madlon-Kay, D. J. 1998. Use of a structured encounter form to improve well-child care documentation. Arch. Fam. Med. 7, 480--483.
39. Marill, K. A., Gauharou, E. S., Nelson, B. K., Peterson, M. A., Curtis, R. L., and Gonzalez, M. R. 1999. Prospective, randomized trial of template-assisted versus undirected written recording of physician records in the emergency department. Ann. Emerg. Med. 33, 500--509.
40. Marshall, M., Campbell, C., Hacker, J., and Rowland, M. 2002. Quality Indicators for General Practice: A Practical Guide for Health Professionalsa and Managers. Oxford University Press, London.
41. McDonald, C. J. 1976. Protocol-based computer reminders, the quality of care and the non-perfectability of man. New. Eng. J. Med. 295, 1351--1355.
42. McDonald, C. J. 1997. The barriers to electronic medical record systems and how to overcome them. J. Amer. Med. Inf. Assoc 4, 213--221.
43. McDonald, C. J., Overhage, J. M., Dexter, P., Takesue, B. Y., and Dwyer, D. M. 1997. A framework for capturing clinical data sets from computerized sources. Ann. Intern. Med. 127, 675--682.
44. McDonald, C. J., Overhage, J. M., Tierney, W. M., Dexter, P. R., Martin, D. K., Suico, J. G., Zafar, A., Schadow, G., Blevins, L., Glazener, T., Meeks-Johnson, J., Lemmon, L., Warvel, J., Porterfield, B., Warvel, J., Cassidy, P., Lindbergh, D., Belsito, A., Tucker, M., Williams, B., and Wodniak, C. 1999. The Regenstrief Medical Record System: A quarter century experience. Int. J. Med. Inf. 54, 225--253.
45. McDonald, C. J., Tierney, W. M., Overhage, J. M., Martin, D. K., and Wilson, G. A. 1992. The Regenstrief Medical Record System: 20 years of experience in hospitals, clinics, and neighborhood health centers. MD Comput. 9, 206--217.
46. McDonald, C. J., Wilson, G. A., and McCabe, G. P., Jr. 1980. Physician response to computer reminders. J. Amer. Med. Assoc. 244, 1579--1581.
47. McGeary, M., Ford, M. A., McCutchen, M. R., and Barnes, D. K. 2007. A 21st century system for evaluating veterans for the disability benefits. IOM Board on Military and Veterans Health: Committee on Medical Evaluation of Veterans for Disability Compensation, Washington, D.C.
48. McKnight, L. K., Elkin, P. L., Ogren, P. V., and Chute, C. G. 1999. Barriers to the clinical implementation of compositionality. In Proceedings of the AMIA Symposium. 320--324.
49. Mehmood, K., Cherfi, S., and Comyn-Wattiau, I. 2009. Data quality through conceptual model quality -- reconciling researchers and practitioners through a customizable quality model. In Proceedings of the 14th International Conference on Information Quality (ICIQ'09). 61--74.
50. Modai, I. and Rabinowitz, J. 1993. Why and how to establish a computerized system for psychiatric case records. Hosp. Commun. Psych. 44, 1091--1095.
51. Nightingale, F. 1863. Notes on Hospitals. Longman, London.
52. O'Connor, A. E., Finnel, L., and Reid, J. 2001. Do preformatted charts improve doctors' documentation in a rural hospital emergency department? A prospective trial. NZ Med. J. 114, 443--444.
53. Orman, L., Storey, V. C., and Wang, R. Y. 1996. Systems approaches to improving data quality. In Proceedings of the Conference on Information Quality. 117--126.
54. Ornstein, S. M., Jenkins, R. G., Lee, F. W., Sack, J. L., Lakier, E. I., Roskin, S. D., Wulfman, J. S., and Wriston, G. A. 1997. The computer-based patient record as a CQI tool in a family medicine center. Joint Comm. J. Qual. Improv. 23, 347--361.
55. Orr, K. 1998. Data quality and systems theory. Comm. ACM 41, 66--71.
56. Pierik, F. H., Van Ginneken, A. M., Dohle, G. R., Vreeburg, J. T., and Weber, R. F. 2000. The advantages of standardized evaluation of male infertility. Int. J. Androl. 23, 340--346.
57. Poon, A. D., Fagan, L. M., and Shortliffe, E. H. 1996. The PEN-Ivory project: Exploring user-interface design for the selection of items from large controlled vocabularies of medicine. J. Amer. Med. Inform. Assoc. 3, 168--183.
58. Powsner, S. M., Riely, C. A., Barwick, K. W., Morrow, J. S., and Miller, P. L. 1989. Automated bibliographic retrieval based on current topics in hepatology: hepatopix. Comput. Biomed. Res. 22, 552--564.
59. Rabinowitz, J., Modai, I., Valevski, A., Zemishlany, Z., and Mark, M. 1993. Benefits of a structured format for paper and computerized psychiatric case records. Hosp. Commun. Psych. 44, 1095--1097.
60. VA Office of the Inspector General. Review of state variances in VA disability compensation payments.
61. Rosenbloom, S. T., Crow, A. N., Blackford, J. U., and Johnson, K. B. 2006. Cognitive factors influencing perceptions of clinical documentation tools. J. Biomed. Inf.
62. Rosenbloom, S. T., Kiepek, W., Belletti, J., Adams, P., Shuxteau, K., Johnson, K. B., Elkin, P. L., and Shultz, E. K. 2004. Generating complex clinical documents using structured entry and reporting. MEDINFO'04, 683--687.
63. Rosenbloom, S. T., Miller, R. A., Johnson, K. B., Elkin, P. L., and Brown, S. H. 2006. Interface terminologies: facilitating direct entry of clinical data into electronic health record systems. J. Amer. Med. Inform. Assoc. 13, 277--288.
64. Safran, D. G., Kosinski, M., Tarlov, A. R., Rogers, W. H., Taira, D. H., Lieberman, N., and Ware, J. E. 1998. The Primary Care Assessment Survey: Tests of data quality and measurement performance. Med. Care 36, 728--739.
65. Sahlstedt, S., Adolfsson, H., Ehnfors, M., and Kallstrom, B. 1997. Nursing process documentation--Effects on workload and quality when using a computer program and a key word model for nursing documentation. Stud. Health Technol. Inf. 46, 330--336.
66. Schriger, D. L., Baraff, L. J., Rogers, W. H., and Cretin, S. 1997. Implementation of clinical guidelines using a computer charting system. Effect on the initial care of health care workers exposed to body fluids. J. Amer. Med. Assoc. 278, 1585--1590.
67. Schwartz, R. J., Boisoneau, D.. and Jacobs, L. M. 1995. The quantity of cause-of-injury information documented on the medical record: an appeal for injury prevention. Acad. Emerg. Med. 2, 98--103.
68. Scott, J. T. 2007. Honoring the call to duty: Veterans' disability benefits in the 21st century. Disability Commission, Washington, D.C.
69. Seltzer, S. E., Kelly, P., Adams, D. F., Chiango, B. F., Viera, M. A., Fener, E., Hooton, S., Bannon-Rohrbach, S., Healy, C. D., Doubilet, P. M., and Holman, B. L. 1997. Expediting the turnaround of radiology reports in a teaching hospital setting. AJR Amer. J. Roentgenol 168, 889--893.
70. Solomon, D. H., Schaffer, J. L., Katz, J. N., Horsky, J., Burdick, E., Nadler, E., and Bates, D. W. 2000. Can history and physical examination be used as markers of quality? An analysis of the initial visit note in musculoskeletal care. Med. Care 38, 383--391.
71. Stead, W. W., Hammond, W. E., and Estes, E. H. 1977. Evaluation of an audio mode of the automated medical history. Methods Inf. Med. 16, 20--23.
72. Stead, W. W., Heyman, A., Thompson, H. K., and Hammond, W. E. 1972. Computer-assisted interview of patients with functional headache. Arch. Intern. Med. 129, 950--955.
73. Stetson, P. D., Morrison, F. P., Bakken, S., and Johnson, S. B. 2008. Preliminary development of the physician documentation quality instrument. J. Amer. Med. Inform. Assoc. 15, 534--541.
74. Teo, S., Hanson, R., Van Asperen, P., Giles, H., Fasher, B., Davis, A. M., and Kristidis, P. 1995. Improving asthma documentation in a paediatric emergency department. J. Paediatr. Child Health 31, 130--133.
75. Tierney, W. M., Miller, M. E., Overhage, J. M., and McDonald, C. J. 1993. Physician inpatient order writing on microcomputer workstations. Effects on resource utilization. J. Amer. Med. Assoc. 269, 379--383.
76. Veterans' Disability Benefits. 2005. VA could enhance its progress in complying with court decision on disability criteria. United States Government Accountability Office, Washington, D.C.
77. Walters, B. C. and McNeill, I. 1990. Improving the record of patient assessment in the trauma room. J. Trauma 30, 398--409.
78. Weeks, W. B., Mills, P. D., Waldron, J., Brown, S. H., Speroff, T., and Coulson, L. R. 2003. A model for improving the quality and timeliness of compensation and pension examinations in VA facilities. J. Healthcare Manage. 48, 252--261; discussion, 262.
79. Weinger, M. B., Herndon, O. W., and Gaba, D. M. 1997. The effect of electronic record keeping and transesophageal echocardiography on task distribution, workload, and vigilance during cardiac anesthesia. Anesthesiology 87, 144--155; 129A--130A.
80. Winickoff, R. N., Coltin, K. L., Fleishman, S. J., and Barnett, G. O. 1986. Semiautomated reminder system for improving syphilis management. J. Gen. Intern. Med. 1, 78--84.
81. Wirtschafter, D. D., Scalise, M., Henke, C., and Gams, R. A. 1981. Do information systems improve the quality of clinical research? Results of a randomized trial in a cooperative multi-institutional cancer group. Comput. Biomed. Res. 14, 78--90.
82. Wrenn, K., Rodewald, L., Lumb, E., and Slovis, C. 1993. The use of structured, complaint-specific patient encounter forms in the emergency department. Ann. Emerg. Med. 22, 805--812.
83. Zenni, E. A. and Robinson, T. N. 1996. Effects of structured encounter forms on pediatric house staff knowledge, parent satisfaction, and quality of care. A randomized, controlled trial. Arch. Pediatr. Adolesc. Med. 150, 975--980.

Index Terms

• Applied computing

  • Life and medical sciences

    • Health care information systems

• Human-centered computing

  • Accessibility

  • Human computer interaction (HCI)

    • Empirical studies in HCI

    • HCI design and evaluation methods

• Mathematics of computing

  • Information theory

• Social and professional topics

  • Professional topics

    • Management of computing and information systems

      • Information system economics