Emergency department admission decision-making: An opportunity for quality improvement in medical education and practice
Article Outline
Abbreviations: ED, Emergency department, PRISA, Pediatric Risk of Admission
Healthcare providers are currently involved in a transformation around quality as profound as the changes that occurred 100 years ago after the Flexner report. The ability to define quality as the gap between care that is being delivered and care that could be delivered, based on best evidence and the development of quality measures, has led to documentation of significant unexplained variation in care even in the best of healthcare organizations. Identifying causes of variation is an important step in finding leverage points for improvement. However, developing quality measures, assessing gaps in quality, and identifying causes of unnecessary variation do not automatically lead to improvement. Knowledge of how to systematically redesign and improve care—quality improvement science—is also necessary. In 2000, the Accreditation Council for Graduate Medical Education (ACGME) and the 24 ABMS certifying boards simultaneously endorsed six core physician competencies necessary to deliver quality care.1 Two of these competencies require residents completing training and practicing physicians maintaining board certification to demonstrate competency in quality improvement and system-based practice as important steps to ensure that physicians can take action to systematically close gaps in quality in their practices. Integrating assessment and improvement of quality of care into clinical practice during residency training and afterward is a major challenge.
See related article, p 644
The article by Chamberlain et al2 in this issue of The Journal addresses one of the most important issues in children’s healthcare, the decision to admit a child to the hospital. As the authors indicate, more than 30 million children are treated in emergency departments (EDs) in the United States each year, and an estimated 1.5 to 2.0 million are admitted. There are significant consequences related to both inappropriate admissions and failures to admit. Interestingly, the work of Wennberg et al3 in 1977 on small area variation in pediatric tonsillectomy rates has been credited as one of the studies that began the current revolution in quality and safety. Chamberlain et al have developed a tool (PRISA II) for risk-adjusting hospital admissions from pediatric EDs. They have applied this tool to admission and return visit rates to develop a measure of quality for ED admission decision-making called The Combined Admission and Return Index. They have used this quality measure to document significant variation in decision-making across pediatric EDs and to identify factors that are associated with variation.2 The presence of a resident training program was strongly associated with a higher index score, whereas neither volume nor the presence of a pediatric emergency medicine specialist was significant. Although the authors state that “the study was not designed to determine the components of the ED decision-making process that contribute to differences in decision-making, such as resident supervision, structure, process, and organizational culture,” the findings suggest that something in the educational process influences admission decision-making. One obvious explanation, suggested by the authors, is that residents are less effective in determining which pediatric patients require admission and which can be safely released. Assuming residents played a direct role in the admission process or influenced the supervising physician’s decision to admit, this is not a surprising conclusion. The decision to admit a child is complex and involves not only clinical knowledge about a host of problems that dictate mandatory admission under the PRISA II scoring system but also an understanding of the social, cultural, and contextual issues surrounding each patient. Paul Batalden, a pediatrician at Dartmouth and a leading spokesman for quality in medical education, has described quality in healthcare as the combination of the best science in the context of what individual patients want and need. Healthcare professionals are just beginning to understand the contextual issues that make up patient-centered care such as health literacy, cultural beliefs, and social and environmental determinants of health. It can be particularly difficult for physicians in training to determine and appreciate contextual issues in clinical microsystems such as EDs, where there is a focus on immediate problem solving, significant time restraints, and the lack of long-term relationships that provide patient background for the physician in other situations. One might expect residents to be more cautious and err on the side of overadmitting, which this study suggests. General measures of knowledge acquisition such as the In Training Exams (ITEs) administered by the American Board of Pediatrics (Althouse L, American Board of Pediatrics, personal communication) show a gradient of medical knowledge from the beginning of residency training to completion of residency and initial board certification. One would suspect that a similar gradient exists for the other competencies as well.
Numerous studies show significant variation across the field of healthcare,4 and the same is true of quality of care related to the presence of a training program. It is not a given that the presence of a training program is always associated with gaps in quality.5 Although the authors refer to several studies in other areas of practice that demonstrate such an association, they also reference studies of training programs in which there are not significant gaps in quality. How are some training programs able to integrate resident training without adversely affecting quality of care, and can this experience be applied to ED admission decision-making?
The report by Chamberlain et al is important to directors of EDs and program directors who are committed to improving quality of care and education. Awareness of the problem and a commitment to change are essential first steps, but what next?
A program director or ED director could ask, “What is the Combined Index for my program, and is it stable over time? If there is a potential gap in quality (index significantly above or below zero), what can I do about it?” The methodology for systematic improvement is well established.6, 7 A proven effective approach, where possible, is to participate in a multicenter improvement collaborative.8, 9 Good ideas of what works could come from those EDs that are performing well on the Combined Index. This would require transparency in reporting, with centers being willing to publicly release quality performance data for the purposes of quality improvement. This is beginning to occur in the treatment of patients with cystic fibrosis.10 It may be sufficient and less threatening if only the best performers were asked to disclose their results. In this study, the range of the Combined Admission and Return Index for EDs with residents was 31.0 to 47, indicating that some programs were better than others at decision-making, even though as a group they were significantly worse than departments without residents.2 In this study, risk-adjusted, multicenter comparative studies can be used to identify factors associated with differences in quality between study groups. They can also be used to identify high-performing practices that can be studied to identify and spread factors associated with success.
Training programs from other specialties that do not show gaps in quality11 should be studied as well. Anesthesiology is the only specialty to achieve close to six sigma levels of patient safety error rates (3.4 defects per 1 million events).12 How has this been achieved in anesthesia training programs? One possibility that could be tested in other settings is the intense one-to-one faculty/trainee clinical supervision that occurs over the first several months of anesthesiology residency training until the trainee is thought to be competent to practice with intermittent supervision.
The broader and more important message of the study by Chamberlain et al is that variation in ED admission decision-making is not restricted to physicians in training. For EDs without residents, the Combined Index varied from −15.9 to +12.2, suggesting that some departments may have not admitted enough patients and some may have admitted more than expected. Every practice can improve. The aim is to close the gap in quality and not just to demonstrate better care than peers. The PRISA II risk adjustment tool and the Combined Admission and Return Index are important developments that can be used to assess quality and to document improvement over time in formal improvement efforts.
There is a tendency in comparative studies to attribute blame for poor performance, in this case to resident decision-making. It is usually difficult to adjust for all confounding factors, and the greater value of comparative studies is overlooked, namely the opportunity to learn from best practices and the opportunity for even the most successful practices to improve.
Variation in admission decision-making exists and may be related to professional development for both physicians in training as well physicians in practice. The authors state, “Future studies identifying best practices and examining these factors would be helpful in elucidating the mechanisms of admission decision-making.” There is a step beyond future research that involves translating research into practice. The practices in this study and others who have identified a gap in quality of ED admission decision-making are challenged with a professional imperative to improve this process.
References
- . Board certification and physician quality. Med Educ. 2004;38:10–11
- . The association of emergency department care factors with admission and discharge decisions for pediatric patients. J Pediatr. 2006;149:644–649
- . Changes in tonsillectomy rates associated with feedback and review. Pediatrics. 1977;59:821–826
- . Practice variation: implication for our health care system. Managed Care. 2004;13(Suppl 9):3–7
- . Quality of care in teaching hospitals: a literature review. Acad Med. 2005;80:458–466
- . The improvement guide: a practical approach to enhancing organizational performance. San Francisco, Calif: Jossey-Bass; 1996;
- . Improvement in healthcare: how can we change the outcome?. J Pediatr. 2005;147:279–281
- Geographic variation in the treatment of acute myocardial infarction: the cooperative Cardiovascular Project. JAMA. 1999;281:627–633
- . Collaborative quality improvement for neonatal intensive care: NIC/Q Project Investigators of the Vermont Oxford Network. Pediatrics. 2001;107:14–22
- . Improving subspecialty healthcare: lessons from cystic fibrosis. J Pediatr. 2005;147:295–301
- . Surgical resident supervision in the operating room and outcomes of care in Veterans Affairs hospitals. Am J Surg. 2005;190:725–731
- . Anaesthesiology as a model for patient safety in health care. BMJ. 2000;320:785–788
PII: S0022-3476(06)00796-7
doi:10.1016/j.jpeds.2006.08.031
© 2006 Mosby, Inc. All rights reserved.
Refers to article:
- Association of emergency department care factors with admission and discharge decisions for pediatric patients
