Skip to main content
OpenTrials
Completed

NCT Number: NCT05816473

Artificial Intelligent Clinical Decision Support System Simulation Center Study for Technology Acceptance

The purpose of this research study is to measure the effect on of a large language model interface on the usability, attitudes, and provider trust when using a machine learning algorithm-based clinical decision support system in the setting of bleeding from the upper gastrointestinal tract (upper GIB). Specifically, the investigators are looking to assess the optimal implementation of such machine learning algorithms in simulation scenarios to best engender trust and improve usability. Participants will be randomized to either machine learning algorithm alone or algorithm with a large language model interface and exposed to simulation cases of upper GIB.

Completed

Looking for future studies?

Notify Me

Key information

Conditions

Gastrointestinal Hemorrhage Digestive System Diseases Gastrointestinal Diseases Hemorrhage Pathologic Processes Pathological Conditions, Signs and Symptoms

Sex eligibility

All sexes

Study type

Interventional

Phase

Not applicable

Primary location

Yale New Haven Hospital

New Haven, Connecticut, 06510, United States

About this study

The experiment will deploy a previously validated machine learning algorithm trained on existing clinical datasets within simulation scenarios in which a patient with acute gastrointestinal bleeding (at low, moderate, and high risk for poor outcome) is evaluated.

Prior to the simulation, a baseline educational module about artificial intelligence, machine learning, and clinical decision support will be provided to all participants. The investigators will establish psychological safety by detailing what is available in the room, the opportunity to call a consultant, and availability of laboratory and radiographic studies. Each clinical scenario will run for approximately 10 minutes based on real patient cases where vital signs change over time and laboratory values are made available at specific points in the assessment. The study will evaluate the effect of a large language model-based interaction with the machine learning algorithm with interpretability dashboard compared to the machine learning algorithm with interpretability dashboard alone. Each participant will receive three scenarios in randomized order of risk.

For the large language model interaction arm, participants will be provided the computer workstation a LLM chatbot interface of the algorithm and interpretability dashboard For the machine learning dashboard arm, participants will be provided the computer workstation with the algorithm and interpretability dashboard.

Who can participate

Healthy volunteers accepted: Yes

Only the study team can determine whether someone qualifies for participation.

Inclusion criteria

  • Internal Medicine residency trainees at study institution
  • Emergency Medicine residency trainees at study institution

Exclusion criteria

  • N/A

Treatment and study plan

LLM

Other

Use of a Large Language Model (LLM) chatbot interface to Interact with the Machine Learning Algorithm and interpretability dashboard.

Primary outcomes

  1. Median Change in Attitudes Towards Machine Learning Algorithms in Clinical Care Using UTAUT

    Time frame: Approximately 60 minutes

    The study will use a common set of dependent variables to assess baseline and post-intervention attitudes towards machine learning algorithms in clinical care using an adapted Unified Theory of Acceptance and Use of Technology (UTAUT) survey assessing perceived usefulness of the system, perceived ease of use, attitudes towards using it, behavioral intentions, and trust, measured with a 5-point Likert scale. Percent change in UTAUT survey response between Large Language Model-based Interaction and Machine Learning Dashboard at recruitment prior to administration of scenarios and immediately after completion of scenarios. The difference in time between the two will be approximately 60 minutes. Higher change indicates greater acceptance/intention to use the GutGPT+Dashboard.

Secondary outcomes

  1. Clinician Decision Making of Triage of GI Bleeding

    Time frame: Approximately 60 minutes

    Mean percentage of decision accuracy per participant. Accuracy is defined as the percentage of times participants accurately choose the correct clinical decision for each simulation scenario of acute upper GI bleeding for each treatment condition. Immediately after completion of scenarios (60 minutes from initiation of study for each participant). No further follow up afterwards.

Sponsors and collaborators

Lead sponsor

Yale University

Other

Collaborators

  • National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK)

Registry information

Official study title

Artificial Intelligent Clinical Decision Support System Simulation Center Study: Trust and Usefulness of Machine Learning Risk Stratification Tool for Acute Gastrointestinal Bleeding Using the Technology Acceptance Model

Important dates

Study start
2023
Primary completion
2024
Study completion
2024
First posted
Apr 18, 2023
Registry last updated
May 22, 2026

OpenTrials presents study information sourced from ClinicalTrials.gov. The official registry record should be consulted for the latest information.

View the official ClinicalTrials.gov record (opens in a new tab)

This listing is for discovery and informational purposes only. It is not medical advice, does not guarantee that a study is recruiting, and does not determine eligibility. Contact the study team and a qualified healthcare professional when considering participation.

Related clinical trials

Published trials that share one or more normalized conditions with this study.