Skip to main content
OpenTrials
Completed

NCT Number: NCT05009251

Using Explainable AI Risk Predictions to Nudge Influenza Vaccine Uptake

The study team previously demonstrated that patients are more likely to receive flu vaccine after learning that they are at high risk for flu complications. Building on this past work, the present study will explore whether providing reasons that patients are considered high risk for flu complications (a) further increases the likelihood they will receive flu vaccine and (b) decreases the likelihood that they receive diagnoses of flu and/or flu-like symptoms in the ensuing flu season. It will also examine whether informing patients that their high-risk status was determined by analyzing their medical records or by an artificial intelligence (AI) / machine-learning (ML) algorithm analyzing their medical records will affect the likelihood of receiving the flu vaccine or diagnoses of flu and/or flu-like symptoms.

Completed

Looking for future studies?

Notify Me

Key information

Conditions

Influenza Behavior Health Behavior Health Promotion Infections Influenza, Human Orthomyxoviridae Infections RNA Virus Infections Respiratory Tract Diseases Respiratory Tract Infections Risk Reduction Risk Reduction Behavior Vaccination Virus Diseases

Age range

18 year and older

Sex eligibility

All sexes

Study type

Interventional

Phase

Not applicable

Primary location

Geisinger Clinic

Danville, Pennsylvania, 17822, United States

About this study

Geisinger has partnered with Medial EarlySign and developed an ML algorithm to identify patients at risk for serious (moderate to severe) flu-associated complications on the basis of their existing electronic health record (EHR) data. Geisinger will apply this algorithm to current patients during the 2021-22 flu season.

This study will evaluate the effect of contacting patients identified as high risk with special messages to encourage vaccination. These communications will inform patients they are at high risk with either (a) no additional explanation, (b) an explanation that this determination comes from an analysis of their medical records, along with a short list of the top factors from their medical record that explain their risk, and (c) the additional explanation that an AI or ML algorithm made this determination, along with a short list of the top factors from their medical record that explain their risk.

Included in the study will be current Geisinger patients 18+ years of age with no contraindications for flu vaccine and who have been assessed by the Medial algorithm and assigned a risk score. The primary study outcomes will be the rates of flu vaccination and flu diagnosis during the 2020-21 season by targeted patients.

Who can participate

Healthy volunteers accepted: No

Only the study team can determine whether someone qualifies for participation.

Inclusion criteria

  • Aged 18 or older
  • Current Geisinger patient at the time of study
  • Falls in the top 10% of patients at highest risk, as identified by the flu-complication risk scores of machine learning algorithm (which operates on coded EHR data)

Exclusion criteria

  • Has contraindications for flu vaccination
  • Has opted out of receiving communications from Geisinger via all of the modalities being tested

Treatment and study plan

Reminder

Behavioral

Mailed letter, short message service (SMS) text, and/or patient portal message

Risk reduction

Behavioral

Mailed letter, SMS, and/or patient portal message

Medical records-based recommendation

Behavioral

Mailed letter, SMS, and/or patient portal message

Other names: Credibility

Algorithm-based recommendation

Behavioral

Mailed letter, SMS, and/or patient portal message

Other names: Credibility

Primary outcomes

  1. Flu Vaccination at 2 Weeks After Final Outreach Date

    Time frame: Within 2 weeks of the final outreach date, 57 days (8.14 weeks) after the study start

    Received flu vaccination

Secondary outcomes

  1. Flu Vaccination at 9 Weeks After Final Outreach Date

    Time frame: Within 9 weeks of the final outreach date, 106 days (15.14 weeks) after the study start

    Received a flu vaccination

  2. Flu Diagnosis

    Time frame: 8 months (between September 9, 2021 and April 30, 2022)

    Received a "high confidence flu" diagnosis (with positive polymerase chain reaction [PCR]/antigen/molecular test) and/or "likely flu" diagnosis (as assessed via International Classification of Disease [ICD] codes or Tamiflu administration or positive PCR/antigen/molecular test)

    Note that "likely flu" is a superset of the "high confidence flu" diagnoses.

  3. Flu Complications

    Time frame: 11 months (between September 9, 2021 and July 31, 2022)

    Diagnosed with flu-related complications

  4. Healthcare Utilization

    Time frame: 11 months (between September 9, 2021 and July 31, 2022)

    Visited ER or was hospitalized

    NOTE: Our prespecified outcome was description was "Visited ER, was hospitalized, or had flu-related insurance claims." We did not receive claims data, so this outcomes includes only ER visits and hospitalizations.

Other outcomes

  1. Flu Vaccination at 2 Weeks After Final Outreach Date by Gender

    Time frame: Within 2 weeks of the final outreach date, 57 days (8.14 weeks) after the study start

    Received a flu vaccination

  2. Flu Vaccination at 2 Weeks After Final Outreach Date by Race

    Time frame: Within 2 weeks of the final outreach date, 57 days (8.14 weeks) after the study start

    Received Flu Vaccination

  3. Flu Vaccination at 2 Weeks After Final Outreach Date by Ethnicity

    Time frame: Within 2 weeks of the final outreach date, 57 days (8.14 weeks) after the study start

    Received flu vaccination

Sponsors and collaborators

Lead sponsor

National Bureau of Economic Research, Inc.

Other

Collaborators

  • Geisinger Clinic
  • Massachusetts Institute of Technology
  • National Institute on Aging (NIA)

Registry information

Important dates

Study start
2021
Primary completion
2021
Study completion
2022
First posted
Aug 17, 2021
Registry last updated
Jan 3, 2025

OpenTrials presents study information sourced from ClinicalTrials.gov. The official registry record should be consulted for the latest information.

View the official ClinicalTrials.gov record (opens in a new tab)

This listing is for discovery and informational purposes only. It is not medical advice, does not guarantee that a study is recruiting, and does not determine eligibility. Contact the study team and a qualified healthcare professional when considering participation.

Related clinical trials

Published trials that share one or more normalized conditions with this study.