Skip to main content
OpenTrials
Active, Not Recruiting

NCT Number: NCT06996626

Real-time NOMA Evaluation

Alerts related to outlier clinician behavior are generated in real-time by an intelligent system continuously scraping EHR (electronic health record) data. These alerts are passed to the bedside and their potential impact on bedside clinical behavior is evaluated.

Active, Not Recruiting

This study is active but is not currently recruiting participants.

Notify Me

Key information

Conditions

Critical Illness Disease Attributes Pathologic Processes Pathological Conditions, Signs and Symptoms

Age range

18 year–100 year

Sex eligibility

All sexes

Study type

Interventional

Phase

Not applicable

Primary location

UPMC Montefiore, Pittsburgh, Pennsylvania, United States

Loading trial locations.

About this study

A clinician-informed AI model will generate outlier alerts from real-time review of the EHR (electronic health record) of UPMC Presbyterian/Montefiore ICU patients. These alerts will first be reviewed by an ICU clinician, along with the patients' EHR, for clinical relevance. For those alerts deemed potentially relevant, the ICU clinician will contact the treating ICU clinician (eg, an ICU pharmacist, physician, advanced practice provider) and discuss the alert. The treating ICU clinician will take whatever action, including no action, they deem best.

Who can participate

Healthy volunteers accepted: No

Only the study team can determine whether someone qualifies for participation.

Inclusion criteria

  • All patients in the Presbyterian and Montefiore ICUs

Exclusion criteria

  • None

Treatment and study plan

Revealed Alerts

Device

Bedside reveal of alerts generated by the alerting system

Unrevealed Alerts

Device

Alerts will be generated but not revealed.

Primary outcomes

  1. Rate of Clinical Actions Following Revealed vs. Non-Revealed Alerts

    Time frame: From time of ICU admission until ICU discharge

    The primary outcome compares the proportion of alerts that lead to documented clinical actions when revealed to treating ICU clinicians versus when not revealed. Alerts are generated by a decision support system and reviewed daily by ICU study clinicians. On average, 30 alerts are reviewed per ICU per day, with approximately 5 alerts revealed to the treating clinicians. The analysis uses a stepped wedge design with ICU beds as the unit of analysis, where each ICU acts as its own control. The outcome will assess whether revealing alerts increases the rate of appropriate clinical actions taken, as compared to when alerts are withheld.

Secondary outcomes

  1. Rate of Any Clinically Responsive Action Following Alerts

    Time frame: Up to 90 days after ICU admission

    For each alert, a range of clinical actions may be deemed "responsive," including but not limited to the specific recommended action. This outcome assesses the differential rate of any clinically appropriate response (whether or not it matches the recommended action) between alerts that are revealed versus not revealed to treating ICU clinicians.

  2. True Positive Alert Rate (TPAR) by Study Group and Alert Type

    Time frame: Up to 90 days after ICU admission

    This outcome evaluates the overall and alert-specific True Positive Alert Rate (TPAR), defined as the proportion of alerts that are associated with a clinically appropriate action. TPARs will be calculated separately for the intervention group (alerts revealed), the control group (alerts not revealed), and the combined population. Comparisons will assess whether revealing alerts is associated with a higher TPAR across alert categories.

  3. Rate of Non-Responsive Actions Following Alerts

    Time frame: Up to 90 days after ICU admission

    This outcome assesses the difference in the rate of clinical actions that are not considered responsive to the alert (i.e., actions taken that do not address the alert's content or recommended intervention) between the intervention group (alerts revealed) and the control group (alerts not revealed). This helps evaluate potential unintended or off-target responses to alerting.

  4. Overall Alert Rate

    Time frame: Through study completion, an average of 2 years

    Total number of alerts generated per ICU per day.

  5. Alert Rate per Alerting Model

    Time frame: Daily, up to 90 days

    Number of alerts generated per model type per ICU per day.

  6. Delay Between Alert Generation and First Responsive Action

    Time frame: Measured continuously through study completion, an average of 2 years

    Median time from alert generation to the first documented responsive clinical action.

  7. ICU Length of Stay

    Time frame: Up to 90 days after ICU admission

    Total number of days each participant spends in the ICU during the index hospitalization.

  8. Hospital Length of Stay

    Time frame: up to 90 days after hospital admission

    Total number of days each participant spends in the hospital during the index hospitalization.

  9. In-Hospital Mortality

    Time frame: Up to 90 days after hospital admission

    Proportion of participants who die during the index hospital stay.

  10. Time Trend of True Positive Alert Rate (TPAR)

    Time frame: Through study completion, an average of 2 years

    Evaluate changes in the True Positive Alert Rate (TPAR) over time during the study period to assess model performance stability and potential temporal variation in responsiveness.

Sponsors and collaborators

Lead sponsor

David T Huang

Other

Collaborators

  • National Institute for Biomedical Imaging and Bioengineering (NIBIB)

Registry information

Official study title

Real-time Evaluation of an Outlier-based Alerting System

Important dates

Study start
2025
Primary completion
2026
Study completion
2026
First posted
May 30, 2025
Registry last updated
Jul 2, 2026

OpenTrials presents study information sourced from ClinicalTrials.gov. The official registry record should be consulted for the latest information.

View the official ClinicalTrials.gov record (opens in a new tab)

This listing is for discovery and informational purposes only. It is not medical advice, does not guarantee that a study is recruiting, and does not determine eligibility. Contact the study team and a qualified healthcare professional when considering participation.

Related clinical trials

Published trials that share one or more normalized conditions with this study.