End Diagnostic Overshadowing

People with disabilities (PWD) experience increased risk of diagnostic error-sometimes due to attributing symptoms to disability rather than a potentially new or co-morbid conditions. As well, some diagnoses are prone to error. Based on literature we identified the following twenty-six diagnoses prone to error with ICD-10 codes: Aortic aneurysm and dissection I71.0 - I71.9; Arterial thromboembolism I74.0 - I74.9; Venous thromboembolism I82.0-I82.99 and I82.A-I82.C; Congestive heart failure I50.1-150.9; Stroke All I60, I61, I62, I63, I64; Myocardial infarction I21.0-I21.9 and I21.A-I21.B; Spinal abscess G06.0, G06.1 and G06.2; Meningitis and encephalitis G04 -G04.91; Endocarditis I33.0-I33.9 and I38; Sepsis A41.0-A41.9; Pneumonia J12.0-J95.851; Lung cancer C34.0-C34.92; Melanoma C43.0- C43.9; Colorectal cancer C18.0-C18.9; Breast cancer C50 to C50.929, and C79.81; Prostate cancer C61; Pediatric Arterial ischemic stroke I63.0-163.9xx; Appendicitis K35-K35.8xx; Asthma J45.2-J45.998; Retinal blastoma C69.20, C69.21, C60.22; Brain tumor C71.0-C71.9; Polyateritis M30.0-M30.8; Congenital heart disease Q20 - Q28 (Q24.9 particularly important); Duchense muscular dystrophy G71.0-G71.9; Inflammatory bowel disease K51.0-K51.9; Scleroderma M34.0-M34.9.The goal of this research is to identify and create understanding of what underlies and contributes to increased risk of diagnostic error with these diagnoses. The investigators plan to develop ways to reduce diagnostic error, specifically ways to identify people with disabilities at risk of diagnostic error (DE). The investigators will also develop education programs and decision supports targeted to healthcare professionals. If it is effective, ways to reduce diagnostic error will have been developed among people with disabilities.

Aim 1: Identify and create understanding of mechanisms underlying diagnostic overshadowing. We will conduct baseline and post-analysis of CPT codes related to diagnostic processes to examine differences between patients aged ≥3 years with and without the specific disabilities listed above, along with demographic and clinical characteristics associated with health outcomes (e.g., race, ethnicity, gender, insurance type, specific diagnoses). Based on differences identified in baseline CPT analyses, we will conduct follow-up chart reviews and targeted interviews and develop Joint Commission-style individual mock tracers, following the care of the listed specific populations of PWD, and systems-of-care tracers focused on evaluating the extent to which care systems incorporate accessibility, effective communication, reasonable accommodations, trauma-informed care, and other processes that support timely and accurate diagnosis of conditions prone to diagnostic error. Mock tracer teams will provide formative evaluation of care to involved staff. Using inductive thematic analysis45 of notes from chart reviews, interviews, and mock tracers, we will identify mechanisms underlying diagnostic overshadowing. We will evaluate CPT codes (quantitative), chart reviews (mixed methods), and interview and tracer results (qualitative) at Year 5 compared with Year 1 to determine changes.

Our hypothesis is that there will be statistical difference in diagnostic processes between people with the specified disabilities and people without the specified disabilities.

Aim 2: Co-produce a framework of mechanisms underlying diagnostic overshadowing to develop educational programs and EHR decision supports. We will collaborate with stakeholders to refine, confirm, and prioritize mechanisms underlying diagnostic overshadowing identified in Aim 1 and use these findings for the co-production of educational programs and EHR decision supports. We will evaluate these mitigation efforts through: (1) pre- and post-knowledge assessments related to use of the educational programs; and (2) descriptive pre- and post-data on the use of specific EHR decision supports.47 Our hypothesis is that we will have information that can be used to develop algorithms for identifying PWD from the specific populations at risk of DO/DE as evidenced by diagnostic process data from the Safer DX Checklist and usage of CPT E/M code.

Our Hypotheses are that there will be statistical change in time to diagnostic evaluation for PWD from the specified 5 population groups and with the specified diagnoses prone to error. We will Evaluate for change after implementation of algorithms to identify patients with the specified disabilities at risk for DO/DE.

Our research is innovative and fills a critical need for improving health outcomes among PWD. We will integrate CPT code analysis with Joint Commission recommendations in a novel way to establish a system for identifying mechanisms underlying diagnostic overshadowing and developing and evaluating targeted educational programs and EHR decision supports to reduce diagnostic errors.

Evaluation of diagnostic processes

Safer DX Checklist

The Safer DX checklist was developed to guide chart reviews including patient history, examination, diagnostic test interpretation and follow-up, ordering of tests, referrals, and diagnostic assessment. The checklist is used to assess five main aspects of the diagnostic process (1) the patient-provider encounter; (2) use and interpretation of diagnostic tests; (3) follow-up and tracking of diagnostic information; (4) referrals and follow-up; and (5) patient-related factors. It is used in multiple studies addressing diagnostic error.We modified the Safer DX Checklist for use by nurses.

CPT Evaluation and Management codes

CPT codes were developed by the American Medical Association and undergo periodic revisions and ongoing maintenance. CPT codes are the universal way that providers document their services, providing standardized reporting needed for billing and reimbursement of healthcare providers, including physicians, nurse practitioners, physician assistants, other professionals.66 The system provides numeric codes for issues such as: 1. The site of service (e.g., Emergency Department, inpatient, outpatient, preventive services); 2. The service provided; 3. The complexity of clinical information-gathering and decision-making, and 4) Time spent. The accuracy of CPT codes can vary, as indicated in a study of CPT codes related to hip fractures in the National Surgical Quality Improvement Program.68 However, CPT codes provide a standardized database used to report aggregated outcomes and to highlight potential problem areas/issues needing further investigation. CPT E/M codes usage is a stage in the diagnostic process where errors can occur.

CPT E/M codes are used to bill for services by providers related to the diagnostic process in evaluating and managing the health of a patient. Each setting has a specific group of CPT E/M codes ranges from lowest time and complexity of decision making to highest time and complexity of decision making. A recent study used CPT E/M codes for video telehealth visits compared to in-person visits with established patients of a large urban public healthcare system above the 50th percentile in video telehealth utilization. Evaluation indicated lower complexity of E/M with telehealth visits.

Furthermore, use of CPT E/M codes at telehealth visits varied by specialty, but the authors noted that it was not known if the differences were due to the two types of visits or to the comfort level of providers and patients. In a small study using emergency department (ED) data at Rush University Medical Center (July 1, 2019 to December 1, 2020), differences in CPT E/M codes were found between patients with and without IDD visiting the ED for the same reasons and same level of severity. Moderate (99284) and high complexity (99285) evaluation and management codes, with no differences in time-intensive cases, were used with 25.7% of patients with IDD compared to 39.6% of patients without IDD, with statistical significance. Additionally, median professional charges for patients with IDD were lower. An analysis of differences in use of CPT E/M codes shows promise in using them to identify and understand mechanisms underlying diagnostic overshadowing and diagnostic error. In our chart reviews we will collect data on any use of CPT E/M codes including dates and specific codes used.

Mock tracers Mock tracers were developed by the Joint Commission for use in preparing for accreditation visits and are often used in healthcare systems as part of ongoing quality assurance and professional development efforts. Mock tracers provide information on patient experiences, quality of care, healthcare processes and products, and areas needing improvement. Tracers involve one-on-one and small group interviews with prompts for the questions that will be asked in addition to a review of patient charts and forms. For this project, questions will center around diagnostic processes (as evaluated using the Safer DX Checklist and evaluation of the use of CPT codes. Deeper inquiry is expected based on answers.

Algorithms

We recognize that lack of data on people with disabilities can lead to inadequate algorithms . People with disabilities expressed concerns of being denied life-saving health services during COVID related to crisis triage algorithms that didn't reflect their needs. Algorithms are already in use to address diagnostic error such as identifying patients at risk of delayed test results; delays in follow up of chest imaging results tests for hypothyroidism, and delayed/missed diagnoses related to abdominal pain. The study that addressed missed/delayed diagnoses related to abdominal pain was conducted in an ED. An algorithm was developed to identify patients at high risk of diagnostic error related to abdominal pain and then used for chart reviews to identify patients at high-risk for diagnostic error related to abdominal pain. In a randomized clinical trial, algorithms were used to prospectively identify patients at high risk of delayed/missed diagnoses of lung, colorectal or prostate cancer. Time to diagnostic evaluation was significantly reduced in the intervention group vs. control group for colorectal and prostate cancers, but not lung cancers. None of the research on algorithms specifically addressed diagnostic overshadowing as part of delayed/missed diagnoses and none addressed intersectionality. A 2009 study specifically on educing diagnostic overshadowing found case studies for educational purposes to be useful.

EHR prompts and alerts

The stage of the diagnostic process (e.g., obtaining clinical history, conducting exams, ordering specific tests, assessments, developing diagnoses, post-diagnostic referrals) requires different clinical decision supports; Furthermore, conditions that are not common require specific supports. Through standard order sets, alerts and reminders, and other means of diagnosticsupport (e.g., website), clinicians can access guidelines more easily. However, poorly designed EHR support can contribute to diagnostic error.82-84 Therefore, developing EHR decision supports requires attention to issues such as how the supports are accepted by clinicians, how they fit with workflow, time requirements, formatting, and how supports promote system-thinking. The EPIC EHR system, as an example, provides a means to improve decision support. Further, patient participation is important, and EHR systems can provide decision supports that can be used by both clinicians and patients.86

Co-production of healthcare programs

Involving impacted persons in co-production of services impacting them is considered an ethical issue in healthcare, transcending the traditional dichotomy between knowledge and program developers and users. Co-production involves building collaboration of people from impacted groups in the production and use of knowledge and programs from the start of the process. Participation of PWD impacted by the results of research and program planning is often limited to providing input after key decisions have already been made rather than throughout the process. However, beginning work to involve people with IDD in the co-production of programs for behavioral health indicated improvements in social networks and confidence for participation. In addition, co-production has been used in developing a framework in healthcare quality improvement. In co-production, the team will work with academic, health systems, health professional, PWDs, family members, and community stakeholders.

Participatory Planning and Decision-Making (PPDM) process versus structured focus groups

To guide development of educational materials addressing diagnostic error, the original plan for was to conduct a new Participatory Planning and Decision-Making (PPDM) process. As we have advanced in our work , two issues led us to revise this approach in favor of structured focus groups with individuals with disabilities and other stakeholders to bring real-world experiences and themes to inform our algorithms and clinical decision-support tools and related targeted education programs The two issues were; 1) An integrative review of diagnostic-error interventions (2017-2024), indicated that the core themes identified from the earlier Administration for Community Living -funded PPDM process continue to be valid and foundational. These themes include the need for dedicated education in the care of individuals with disabilities; the importance of interprofessional education; critique of the medical model of disability and attention to social, civil-rights, and other inclusive models; attention to intersectionality; meaningful involvement of individuals with disabilities in program design, implementation, and evaluation; and the use of experiential pedagogical approaches. Repeating a full PPDM process would be duplicative and would not yield substantially new themes. 2) End Diagnostic Overshadowing program educational materials will need to be targeted to specific audiences-patients, community members, health-care staff, and advanced practice providers. At this stage, the most useful input is not the generation of new themes, but the development and evaluation of educational materials using current themes. Structured focus groups using an organized set of questions are better suited to this type of targeted, content-specific feedback.

Study endpoints:

Primary

At Year 5 compared to Year 1, diagnostic process usage with the five identified groups of people with disabilities (quantitative) will be evaluated for changes following implementation of algorithms to identify people with disabilities at risk of DO/DE along with EHR decision supports and prompts/alerts on specific issues. We expect statistical changes.

We will evaluate our education programs through 1) pre- and post- knowledge checks of usage and 2) descriptive data on use of specific EHR decision supports.101 We expect statistical change in knowledge.

We plan time to conduct pre-post time to diagnostic for 2-3 issues still yet to be determined to address delayed/missed diagnoses. We expect time to diagnostic evaluation to decrease.

Secondary For quantitative measures (Safer DX Checklist data, CPT E/M code usage analysis, knowledge checks, descriptive data on use of EHR decision supports and prompts/alert), we will use ANCOVA to probe for interaction effects using pre-test measures and independent variables from Year one: age increments, gender, race/ethnicity, urban/rural, co-morbidities, disability type, insurance type, severity of illness [relevant to site]), and the 10 issues suggested by stakeholders (singly or in composites) as covariates and evaluate for variation in post-test results. Interaction effects are expected.

Final mock tracers will be conducted at the end of Year 4 and in Year 5. Tracer notes will be compared to notes before the intervention using qualitative analysis. Changes will provide context for any changes in quantitative measures.

Framework The Collective Impact Model for social change is the organizing framework for this project. Multiple impacted groups are brought to bear on the problem of diagnostic overshadowing. Previous efforts to develop algorithms to identify patients at high risk of diagnostic error have not specifically addressed diagnostic overshadowing affecting patients with disabilities and have not previously addressed intersectionality. The Collective Impact Model has not previously been used to address diagnostic overshadowing or the overall problem of diagnostic errors. The following five tenets must be met to facilitate organization and planning with multiple impacted groups for a Collective Impact project: 1) achieving a common agenda; 2) ensuring continuous Communication; 3) identifying shared measurement strategies; 4) employing mutually reinforcing activities to deliver programs and services that will achieve the intended outcome of Collective Impact efforts; and 5) employing a dedicated staff as backbone support. Partnering requires attention to bringing in the experiences and voices of all impacted groups.

Building Organizational Structures using the Collective Impact Model In the first six months, members of research team will meet at least once a month to solidify the team, hire new staff, and create structures based on the Collective Impact Model. A Cross-Sector Partnership Steering Committee, the Cross-Disability Advocate Advisory Committee, and three Consortium Action Networks (Communication, Measurement, Education) will be organized. The Steering Committee and Cross-Disability Advocate Committee will take overall accountability for developing a shared agenda (Collective Impact Tenet 1). Practices that improve understanding of diagnostic overshadowing and the identification of underlying mechanisms will be developed through continuous communication. The Communication Action Network will take accountability (Collective Impact Tenet 2). The Measurement Action Network will take accountability for ongoing evaluation and final evaluation in Year 5(Collective Impact Tenet 3). The Education Action Network will take accountability for facilitating development of targeted education programs and EHR decision supports to mitigate diagnostic overshadowing (Collective Impact Tenet 4). The developed infrastructure will facilitate mutually reinforcing activities that encourage the sharing of perspectives and best practices of the project's interdisciplinary partners. Processes leading to the achievement of project goals and outcomes will be facilitated by dedicated backbone staff who will assist with the management, planning, and logistics required by the project. RUSH University is the lead institution, and each consortium partner has specific responsibilities. (Collective Impact Tenet 5).

Design Aim 1: Identify and create understanding of mechanisms underlying diagnostic overshadowing.

Introduction: Partnership will be built between three not for profit medical center systems that place prominence on improving the health of the populations they serve. RUSH University System for Health and affiliated RUSH University Medical Center, RUSH Oak Park Hospital and RUSH Copley Medical Center; Rochester Regional Health and affiliated Rochester General Hospital; and Erie County Medical Center (ECMC). These will be sites for pre-post analysis of diagnostic error among PWD via use of the Safer DX Checklist and CPT E/M code data, implementation of mock tracers, and then implementation of targeted education programs and EHR decision supports. Data use agreements between the three institutions are being obtained.

Data Collection will be in three steps. For the first, data were retrieved for period January 1, 2023 - June 30, 2024 for patients aged 3-89 who at any time had one or more of the 26 diagnoses prone to error from RUSH University Medical Center, RUSH Oak Park Hospital, RUSH Copley Hospital, and associated outpatient practices. These dates were chosen as changes were made to CPT E/M codes in 2023 in a way expected to reduce burden. Data are from cases of patients from and not from the specified disability groups. Data are being used to compare diagnostic processes for patients with and without the specified disabilities. To compare usage of CPT E/M codes data were retrieved on patients who received a billed CPT E/M code from the Emergency Departments (codes 99281-99285), from inpatient services (99221-99223, 99231-99233, 99238-99239), from outpatient visits with new patients (99202-99205), with established patients (99211-99215), and for preventive care (99384-99387). E/M is a stage in the diagnostic process where errors can occur. Quantitative methods for evaluation of CPT E/M codes data will be used. Data will be programmed with variable range checks and skip rules and will be exported in an automated manner into SPSS. Based on experience, patients with the specific disabilities will be identified through a comprehensive list of secondary diagnosis codes for the specific disabilities for patients aged 3-89 years old. Data will be age-disaggregated in groupings of five years, except the group aged 3-5 years old. All variables will be checked for errant values. Descriptive statistics will be computed for all items (CPT E/M codes), and distributions examined for non-normality and outliers. Descriptive statistics for all measures will be reported. For each type of visit, the investigators will first compare the overall proportion of each CPT E/M codes by disability status using pairwise Fisher's exact tests with a descriptive analysis of case frequency, age increments, gender, race/ethnicity, urban/rural, co-morbidities, disability type, insurance type, and severity of illness [relevant to site]). Evaluation will be conducted on whether data can be collapsed into values of higher level and lower level complexity of evaluation and management codes. If so, binary logistic regression analysis will be conducted using the outcome of higher level and lower level of complexity of CPT evaluation and management codes and addressing the influence of race, ethnicity, gender, age ranges, disability type, insurance status, severity of illness and 10 chart review questions (previously described). Otherwise, the investigators will use ordinal regression analysis on the outcomes of the CPT E/M codes (using all levels of complexity). Site-specific analyses will be conducted (ie. ED, inpatient, outpatient, preventive care) and a combined model that accounts for site using cluster-robust standard errors.

Binary (or ordinal) regression analysis will be conducted for each setting (Emergency Department, inpatient, outpatient, preventive care) with the dependent variable being E/M codes (separately or split into lower and higher complexity codes). Independent variables will be demographics of age increments, gender, race/ethnicity, urban/rural, co-morbidities, disability type, insurance type, and severity of illness [relevant to site]), and the 10 chart review issues listed above. Variable loadings will be assessed for use in developing algorithms for PWD from the specified groups at risk of diagnostic overshadowing.

We also plan to use Coincidence Analysis, to evaluate necessary and sufficient factors predicting diagnostic error, noting that two members of the team have taken training in usage of this type of analysis. Coincidence analysis is a configurational comparative method that historically was used primarily in the social sciences and is increasingly used in implementation science. Unlike traditional statistical methods that focus on the net effects of individual variables, coincidence analysis emphasizes the interplay of multiple factors and seeks to identify factors that might affect an outcome through patterns of co-occurrence. Coincidence analysis uses Boolean algebra in the identification of one or more combinations of minimally sufficient and necessary factors contributing to an outcome that may lack pairwise correlation used in regression analysis.

For the second step we will identify charts of patients in the specific disability groups and with specific diagnoses prone to error for retrospective manual chart reviews to improve identification of underlying mechanisms of diagnostic overshadowing. There is existing literature on diagnoses prone to error. At each partnering hospital (5) associated with the three medical centers, we will review at least five charts of patients from each of the five specific We will also develop a profile of patients at populations of people with disabilities from which we are collecting data (25 at each of the five institutions and associated outpatient practices). Considering additional targeted chart reviews we expect another 100. We will evaluate for issues which may provide insight into diagnostic overshadowing. We will use the Safer DX Checklist in the chart reviews. In discussions with staff, suggestions to explore include the same issues listed above in previous work. We will take notes on the ten issues. Notes will be evaluated for themes using inductive thematic analysis. The team will meet weekly to discuss themes. If, during the chart reviews, we determine other issues, we will submit an amendment to evaluate additional issues. We are developing protocols and training for the chart reviews with drafts that consider experiences of two previous groups in place. Protocols will be shared with the IRB

Third, with these baseline data that indicated higher rates of sepsis among patients with disabilities, we consulted with our Measurement Action Network and with our Advocate Advisory Committee and developed a system of care mock tracer focused on sepsis care with attention to people with disabilities that were implemented in the Emergency Departments at RUMC, Rush Oak Park, and Rush Copley. We also modified the Safer DX for use by nurses and were able to use the modified version in chart reviews. We will also develop separate mock tracers following the care of each specific population of PWD prone to diagnostic overshadowing. We will conduct 25 retrospective tracers of patients (5 each from the specific populations listed above with 10 tracers to be among children and at least 10 to be among patients from marginalized racial/ethnic groups) at each of the five partnering institutions with associated outpatient practices (at RUSH University Medical Center, RUSH Oak Park Hospital, and RUSH Copley Hospital) In our meetings with Rochester Regional Health and Erie County Medical Center we decided to determine numbers once we have experience in the Rush System.

We will inform the IRB once decisions are made. For tracers, we will also choose patients with issues such as high acuity, complexity of care (e.g., multiple tests, surgeries), transfers between units, and history of trauma. We recognize that issues affecting care differ by population. Therefore, we will develop a system of care tracer focused on understanding system facilitators and barriers to reducing diagnostic overshadowing including trauma-informed care as part of the system of care. As we will develop the mock tracers at Rush, we will revise/develop and conduct the first baseline tracers at Rochester Regional in Year two. We expect 75 tracers at baseline in the Rush system with numbers at Rochester Regional Health to be determined based on experience. We will inform the IRB once determined. We will use developmental formative evaluation methods for the mock tracers using guides, with the formative evaluation communicated to the respective units and practices.11 During years 3 and 4, we will conduct mock tracers in EDs (nationally 70% of inpatients at hospitals are processed through EDs), selective inpatient units (including pediatrics), and selective outpatient practices (including pediatrics). We will conduct final mock tracers and analysis at the end of Year 4 and in Year 5. We will conduct qualitative analysis of review notes compared to notes before intervention. We expect changes that will provide qualitative data on context of changes in quantitative measures.

The development and first baseline implementation of mock tracers and any edits will be completed at Rush by the beginning of Year three. At other systems, mock tracers will be conducted in year three. This will be a total of 125 tracers at baseline. With guides, developmental formative evaluation methods will be used for the mock tracers, with the formative evaluation communicated to the respective units and practices. During years 3 and 4, mock tracers will be conducted in EDs, selective inpatient units (including pediatrics), and selective outpatient practices (including pediatrics) across the five institutions - 15 tracers at each for a total of 75.

Aim 2: Co-produce a frame of themes underlying diagnostic overshadowing to develop algorithms to identify patients with specific disabilities at risk of DE along with EHR decision supports and prompts/alerts on specific issues. Educational materials on the algorithms and the EHR decision supports and prompts/alerts along with case studies to educate providers on DE and EHR materials will be developed.

Data analysis plan

At Year 5 compared to Year 1, CPT E/M code usage with the five identified groups of people with disabilities (quantitative) will be conducted to evaluate whether coding usage changed for PWD after implementation of algorithms to identify people with disabilities at risk of diagnostic overshadowing/diagnostic error along with EHR decision supports and prompts/alerts on specific issues. Binary or ordinal pre-post ANCOVA regression analysis will be conducted for each setting (ED, inpatient, outpatient, preventive care) either separately or as clusters with the pretreatment outcome and post-treatment outcomes as binary or ordinal percentages. Independent variables will be patient demographics of age increments, gender, race/ethnicity, urban/rural, co-morbidities, disability type, type of insurance, and applicable severity index, and the 10 chart review issues listed above either separately or as composites. Developed education programs will be evaluated through 1) pre- and post- knowledge checks of usage and 2) descriptive data on use of specific EHR decision supports as independent t tests and as regression analysis with independent variables being setting, gender, race/ethnicity, age range, type of provider, and setting of the provider (ED, inpatient, outpatient, preventive care). Pre-post time to diagnostic evaluation using ANCOVA to evaluate for differences will be conducted for 2-3 issues still yet to be determined with demographic characteristics of patients with disabilities and provider characteristics as independent variables. Interaction effects will be evaluated. For quantitative measures (CPT E/M code usage analysis, knowledge checks, descriptive data on use of EHR decision supports and prompts/alert), ANCOVA will be used to probe for interaction effects using pre-test measures and independent variables from Year one.

Final mock tracers and analysis will be conducted at the end of Year 4 and in Year 5. Qualitative analysis of review notes compared to notes before intervention will be conducted.

Inclusion criteria

• Patients aged 3-89 who received billed charges

Exclusion criteria

• Patients under age 3 or over age 89.
• Patients with secondary diagnosis of dementia as the population is already known to be at increased risk of diagnostic error