Clinical Validation of Vital-PICASO for Predicting Hypoxia Within 24 Hours in General Ward Inpatients

This study is a retrospective, single-center, superiority-confirmatory clinical performance study conducted at Seoul National University Hospital.

Data Source and Study Population

Electronic medical records will be reviewed for male and female patients aged 19 years or older who were admitted to a general ward for at least 12 hours between January 1, 2023, and May 31, 2025. Clinical data previously used in the development of Vital-PICASO will be excluded.

The collected variables will include systolic blood pressure, diastolic blood pressure, heart rate, respiratory rate, body temperature, oxygen saturation, and the corresponding measurement times. Additional variables will include the occurrence, time, and location of hypoxia; oxygen administration status, flow rate, delivery method, and start time; month and year of birth; sex; height and weight when available; do-not-resuscitate status; chronic obstructive pulmonary disease status; pregnancy status; date of admission to the general ward; clinical department; and the vital-sign data collection period.

Reference-Standard Groups

Hypoxia will be evaluated using two separate reference-standard criteria:

• Oxygen saturation criterion: SpO2 below 94%.
• Oxygen-treatment criterion: administration of oxygen at 4 L/min or more.

For hypoxia-positive datasets, vital-sign data collected for up to 72 hours before the first eligible hypoxia event will be extracted. If hypoxia occurs within 24 hours after admission or transfer to the general ward, all available vital-sign data recorded before the event will be collected. If a patient experiences multiple eligible hypoxia events, only the first eligible event will be included for each reference-standard criterion. A patient who meets both hypoxia criteria may contribute a dataset to both positive-group analyses.

Hypoxia-negative datasets will be selected from patients who did not experience hypoxia during the selected general ward admission. Negative datasets will be randomly selected from the same general wards as the positive datasets and from a time interval comparable with the hypoxia-event reference time or from up to 72 hours after admission to the general ward. Only one eligible data interval will be selected per patient.

Data Selection and Reference-Standard Establishment

To reduce selection bias, candidate positive and negative datasets will be numbered and provided to an independent statistician. The statistician will use R or SAS software to randomly select the required number of datasets and generate a randomly ordered identification-code list.

The initial positive or negative classification will be determined according to the predefined oxygen saturation or oxygen-treatment criterion. To improve the reliability of the study dataset, a clinician with at least 1 year of clinical experience will review the data collection reference time and relevant patient information while blinded to the initial positive or negative classification. The clinician's final determination will be used as the reference standard.

Application of Vital-PICASO

The medical device operator will receive vital-sign datasets identified only by screening numbers and will remain blinded to the reference-standard classification. Each dataset will be analyzed using Vital-PICASO to estimate the risk of hypoxia within 24 hours. The peak risk score and corresponding time within the analysis interval will be recorded.

Performance Evaluation

The software-generated risk scores will be compared with the reference-standard classifications. The two primary performance measures will be:

• Area under the receiver operating characteristic curve for predicting hypoxia defined as SpO2 below 94%.
• Area under the receiver operating characteristic curve for predicting hypoxia defined as oxygen administration at 4 L/min or more.

Each AUROC will be compared with the prespecified comparator performance value of 0.7891. The success criterion for each primary outcome will be met if the lower bound of the 95% confidence interval for the difference between the Vital-PICASO AUROC and the comparator AUROC is greater than zero. The overall study will be considered successful if both primary outcomes meet their respective success criteria.

Secondary performance measures will include area under the precision-recall curve, precision, recall, F1 score, threshold-specific performance, performance by prediction-time interval, cumulative-time performance, sensitivity, specificity, positive predictive value, and negative predictive value.

Planned Analysis Datasets

For the SpO2-below-94% criterion, 2,250 datasets are planned, including 1,125 positive and 1,125 negative datasets. For the oxygen-at-4-L/min-or-more criterion, 546 datasets are planned, including 273 positive and 273 negative datasets. Positive datasets may overlap between the two analyses, and the negative datasets may be used in both analyses.

Because the study uses previously collected electronic medical record data, participants will not undergo any additional examination, treatment, or change in clinical care.

Inclusion criteria

General Criteria

• Male or female patient aged 19 years or older at the time the clinical data were recorded
• Electronic medical record data available between January 1, 2023, and May 31, 2025
• Admission to a general ward for at least 12 hours
• Clinical data not previously used in the development of Vital-PICASO

SpO2-Based Hypoxia-Positive Group

• Hypoxia occurred in a general ward
• Hypoxia was defined as oxygen saturation below 94%
• Vital-sign data were available for up to 72 hours before the hypoxia event
• If hypoxia occurred within 24 hours after admission or transfer to the general ward, all available vital-sign data recorded before the event were included
• If multiple eligible hypoxia events occurred during the data collection period, only the first eligible event was included for this criterion

SpO2-Based Hypoxia-Negative Group

• No hypoxia occurred during the selected general ward admission
• Data were randomly selected from the same general wards as the corresponding hypoxia-positive group
• Vital-sign data were available from a time interval comparable with the hypoxia-event reference time or for up to 72 hours after admission to the general ward
• Only one eligible data interval was included per patient

Oxygen-Treatment-Based Hypoxia-Positive Group

• Hypoxia occurred in a general ward
• Hypoxia was defined as oxygen administration at a flow rate of 4 L/min or more
• Vital-sign data were available for up to 72 hours before the initiation of oxygen administration
• If oxygen administration began within 24 hours after admission or transfer to the general ward, all available vital-sign data recorded before the event were included
• If multiple eligible events occurred during the data collection period, only the first eligible event was included for this criterion

Oxygen-Treatment-Based Hypoxia-Negative Group

• No hypoxia occurred during the selected general ward admission
• Data were randomly selected from the same general wards as the corresponding hypoxia-positive group
• Vital-sign data were available from a time interval comparable with the hypoxia-event reference time or for up to 72 hours after admission to the general ward
• Only one eligible data interval was included per patient

Exclusion criteria

• Hypoxia occurred outside a general ward
• The patient had chronic obstructive pulmonary disease
• A do-not-resuscitate order was in effect at the time the clinical data were recorded
• The patient was pregnant or within 3 months after delivery at the time the clinical data were recorded
• The data were considered inappropriate for the study by the principal investigator