Quality and Safety Case Studies

New Quality Improvement Framework Improves How Hospitals Perform QI Work

In early 2021, a work group comprising representatives from the ACS Quality Programs was formed to understand the limitations and barriers affecting the performance of quality improvement (QI) projects in hospitals and develop a framework to help hospitals improve how they perform their QI work. 

The work group consisted of representatives from the following ACS Quality Programs: 

• Children’s Surgery Verification QI Program
• Commission on Cancer 
• Metabolic and Bariatric Surgery Accreditation and QI Program
• National Accreditation Program for Breast Centers 
• Trauma Verification, Review, and Consultation Program 

The QI Framework will facilitate execution of more comprehensive and effective projects as well as offer support for the planning, execution, evaluation, and reporting of these efforts. It can be used across all surgical specialties.

The QI Framework includes eight components and 40 specific criteria that will guide a project team through the execution and documentation of quality improvement projects, including problem detailing, aim specification, strategic planning, process evaluation, outcome evaluation, cost evaluation, and knowledge acquisition.

To help hospitals become familiar with the components and criteria of the QI Framework, it has been retroactively applied to the 2022 Best Practices Case Studies featured in the following case study. While this example does not include all 40 criteria of the QI Framework, it demonstrates how the eight components compose a successful QI project. For more information on the ACS Quality Framework, contact ACSQualityFramework@facs.org.

Approach to Decreasing CT Utilization for Diagnosing Appendicitis

by Kavita Bhakta, BSN, RN, Theresa Viduya, MSN RN, Kathryn Danko, BSN RN, and Erich Grethel, MD, FACS, FAAP

Reducing radiation exposure in the pediatric population has long been a quality initiative nationwide. Dell Children’s Medical Center (DCMC) joined the Pediatric Surgery Quality Collaborative (PSQC) in 2020 and one of the first projects initiated in the collaborative focused on decreasing computed tomography (CT) use to diagnose appendicitis. A review of the literature by the collaborative found a study specific to CT use for diagnosing appendicitis in the pediatric population that showed a correlation between radiation exposure and increased cancer risk in later adult life.* After interviewing participating hospitals that were either high or low outliers, the collaborative created an implementation guideline to assist members in decreasing their CT rates. 

DCMC’s National Surgical Quality Improvement Program Pediatric (NSQIP-P) 2020 Semi-Annual Report (SAR) showed an increased CT rate to diagnose appendicitis compared with the previous report. After further historical review, the institution found the CT rate of 30.3% in 2020 was the highest since data collection for the appendectomy variable in NSQIP-P started in 2015.

DCMC is a free-standing pediatric hospital located in Austin, TX. This 240-bed institution has 50 subspecialties and is a designated magnet hospital with a Level I Pediatric Trauma Center, Level I Children’s Surgery Center, and a Level IV Neonatal Intensive Care Unit. In fiscal year 2021 alone, DCMC had an average daily census of 121 patients, more than 39,800 Emergency Department  (ED) visits, and 7,121 surgeries. 

DCMC is affiliated with The University of Texas at Austin Dell Medical School and is part of the Ascension Healthcare Company, a faith-based nonprofit healthcare system that includes more than 150,000 associates, 40,000 aligned partners, and operates more than 2,600 sites of care in 19 states and the District of Columbia. 

Beginning in May 2021, a project team consisting of two RN coordinators, a nurse manager, project manager, and surgeon champion met to develop a plan to decrease DCMC’s CT rate.

The implementation guideline provided by the PSQC encouraged the use of a pediatric appendicitis scoring tool, appendicitis guideline, ultrasound (US) protocol and training, US report in electronic health records (EHR), and US strategies for patients with BMI ≥30. The team noticed that besides magnetic resonance imaging (MRI) use, DCMC already had most implementations advised by the collaborative in place but compliance with some of these factors had decreased over time. The project team organized a larger interdepartmental CT reduction team and included the addition of 14 representatives:

• Administration: Director of trauma services (1), interim surgical services director (1) 
• Surgery: General surgeons (2), surgery advanced practice providers (2)
• ED: Emergency medicine physicians (2), RN (1)
• Radiology Department: Radiologist (1), radiology manager (1)
• Imaging Leads: US technician (1), CT technician (1), MRI technician(1) 

Goal Specification 

SMART Goals:

Specific: Using the implementation guideline provided by the PSQC, DCMC aimed to decrease its CT use rate to ≤15% while maintaining a negative appendicitis rate of ≤1.75%.

Measurable: NSQIP-P and institutional data (Centricity and EHR)

Achievable: PSQC set a goal to have the CT use rate decrease to ≤15% by the end of 2021. The team discussed this goal and determined it was not attainable as the project started mid-year (July 2021); therefore, DCMC decided to increase the timeline to 1 year (June 2022). 

Relevant: Historically, DCMC’s CT rates for diagnosing appendicitis have been lower, as low as 12.8% in 2015. DCMC needed to address the changes that have occurred since this time and also determine any new implementations that could be added to the Acute Appendicitis Guideline to decrease the CT rate. 

Timeline: July 2021–June 2022.

Strategic Planning 

Monthly meetings were scheduled starting in May in which each department was introduced to the project and end goal. Historical NSQIP-P data were reviewed and showed a decrease in compliance with Pediatric Appendicitis Score (PAS) documentation by the ED and surgery. The group found that it would be beneficial also to collect data on all patients who were evaluated for appendicitis and received imaging at DCMC. This larger pool of data allowed the team to monitor the success rates of appendix visualization via US and CT in patients who did not have appendicitis, as both data points are not collected in NSQIP-P. The additional data were collected via Centricity and hand-pulling from EHR. 

Since MRI use would be a new amendment to the current institutional Acute Appendicitis Guideline, the team also conducted a literature review to investigate the effectiveness and cost difference between MRI and CT for diagnosing appendicitis. As the actual cost of these procedures is institution based, DCMC connected with its Billing Department for the internal costs. The team determined that the fast-sequence MRI would be just as effective as CT for diagnosing appendicitis and the long-term benefits of decreased radiation exposure outweighed the cost difference. 

A CT-use dashboard was created for easy data visualization and ad lib monitoring by the team. Data points on the dashboards included monthly CT rate, PAS completion, US visualization rates, ED duration, admissions for observation, and lab counts (see Figure 1).

Many of the implementations involved in DCMC’s project were already in place; for example, the institution had an established Acute Appendicitis Guideline, the physicians were using an appendicitis scoring tool, and the ultrasound technicians were trained on visualizing the appendix. Ultimately, it came down to focusing on bringing these processes back to light and increasing compliance. 

Process Evaluation

The representatives took the information from the monthly meetings back to their respective departments and returned with follow-up interventions and goals. The ED stated it would reach 100% compliance with PAS documentation by conducting inservices with attendings and residents, posting reminder flyers at workstations with instructions on simple EHR documentation, and counseling individuals who remained noncompliant at each data review. The radiology department made it mandatory to scan patients for a minimum of 15 minutes to visualize the appendix and, when available, ask another technician to scan the patient if the appendix was not visualized. The surgery department encouraged colleagues to give families the option to admit patients with equivocal exams for observation and next-day repeat US in lieu of a CT, held an inservice with its group to complete PAS documentation, and initiated the institutional process to incorporate MRI use for diagnosis of appendicitis. 

The institutional Acute Appendicitis Guideline is in the process of being amended to incorporate MRI use for diagnosing appendicitis and is under review for approval by the evidence-based outcomes center committee. At present, the MRI implementation is in the logistics phase, which includes developing an MRI protocol and navigating how to incorporate the stat MRI orders from the emergency department into the daily MRI schedule. 

Outcome Evaluation

The 2020 NSQIP-P data showed DCMC to have a CT-rate of 30.3% with a negative appendicitis rate of 0.7%. According to the 2021 NSQIP-P data, the CT rate decreased to 23.2% with a negative appendicitis rate of 0.9%. PAS completion in both emergency and surgical departments and US visualization had an increasing trend over the year (see Figure 2 and Figure 3). DCMC also has shown an increase in hospital admissions for observation since the project began. 

Setbacks

Visualization of the appendix is highly dependent on the experience of the technician.^† Staff turnover increased during the coronavirus pandemic, leading to a loss of experienced technicians. The data do show improvement, but the numbers may have been better if staff turnover did not occur. 

There was a setback related to visualization of the appendix in patients with a high body mass index; the radiology department is troubleshooting to determine if anything can be done differently in these patients other than changing patient position and emptying bladder prior to US scan. 

As expected, operationalizing MRI imaging instead of CT brought up some reservations in each involved department. DCMC addressed some of these reservations, such as MRI technician availability and interpretation of results, by seeking advice from other institutions involved in the PSQC who already were utilizing MRI for diagnosing appendicitis. The institutions shared their available resources, some of which included literature supporting MRI use, MRI protocol information, and coding. With the knowledge gained from these resources, the team is working through its concerns with hospital administration while emphasizing the importance of reducing radiation exposure to young children. 

Cost Evaluation

No additional costs or funding beyond normal hospital operations were needed to implement or maintain the project at the time this case study was written. However, the MRI implementation pilot may reveal future additional costs. 

Cost considerations were not the focus of this project. The goal being to reduce the number of CT scans significantly, a cost savings may or may not be balanced out with resources committed to other aspects of the abdominal pain work-up. 

Knowledge Acquisition

There were several lessons learned from this quality improvement project, including:  

• Leadership is very important for change; having an involved leadership group increases buy-in from other stakeholders and provides the support and encouragement for the team to move forward toward project goals. 
• Access to data is vital and presentation matters. Information Systems Department involvement was necessary to pull the required non-NSQIP-P data. Additionally, the creation of a dashboard to visualize the data allowed stakeholders to review data any time, so meetings were more efficient and the project team received more feedback. 
• Not having a dedicated data analyst meant the project team took on this extra role. 
• PSQC has been an excellent networking resource, which has been very helpful as this institution proceeds with MRI use for diagnosing appendicitis. 

End of Project Decision-Making

Upon project completion, the dashboard will be available to the entire hospital in Tableau allowing for sustainability of data dissemination and continuous monitoring. 

The Acute Appendicitis Guideline is under review to incorporate MRI use. Once implemented and if CT rate continues to decline, DCMC does not foresee additional changes until the 3-year guideline review mark. DCMC plans to share the guideline and project results with other Ascension hospitals in the surrounding area. 

Acknowledgments 

Special thanks to Terry Fisher and Kevin Lally, MD, FACS, from the PSQC for help with this project. The team also would like to thank Afif Kulaylat, MD, and Michael Moore, MD, from Penn State, Loren Berman, MD, FACS, from Nemours, and KuoJen Tsao, MD, from Children’s Memorial Hermann for MRI resources and guidance. Lastly, thank you to the CT reduction team, for the time and hard work dedicated to this effort.

*Lee KH, Lee S, Park JH, et al. Risk of hematologic malignant neoplasms from abdomniopelvic computed tomographic radiation in patients who underwent appendectomy. JAMA Surg. 2021;156(4):343-351.

^†Kim J, Kim K, Kim J, et al. The learning curve in diagnosing acute appendicitis with emergency sonography among novice emergency medicine residents. Wiley. 2017;46(5):305-310.

Kavita Bhakta is RN coordinator/NSQIP-P surgical clinical reviewer at Dell Children's Medical Center in Austin, TX.