Skip to main content
OpenTrials
Completed

NCT Number: NCT07166692

Evaluation of a Retrieval Augmented Large Language Model as a Diagnostic Copilot in Rheumatology

This trial evaluates whether providing physicians with access to Prof. Valmed, a clinical decision support medical product, improves identification of rheumatic diseases and formulation of differential diagnoses compared with conventional decision support.

Completed

Looking for future studies?

Notify Me

Key information

Conditions

Diagnosis Disease Pathologic Processes Pathological Conditions, Signs and Symptoms

Sex eligibility

All sexes

Study type

Interventional

Phase

Not applicable

Primary location

Institute for Digital Medicine, University Hospital of Giessen and Marburg, Philipps University Marburg

Marburg, 35043, Germany

About this study

Advanced AI, particularly large language models, shows promise for enhancing clinical reasoning, yet most systems such as ChatGPT are not certified as medical products. Prof. Valmed is a clinical decision support medical product designed to assist physicians in diagnostic decision making. Given frequent referral problems and diagnostic delays in rheumatology, evaluating such support is highly relevant for clinical workflows.

This randomized controlled trial will test whether access to Prof. Valmed improves physicians' diagnostic performance in cases of suspected rheumatic disease compared with conventional decision support. Participants will be randomized to either use Prof. Valmed or rely on conventional tools while working through standardized clinical cases. For each case, participants will submit up to three differential diagnoses and a confidence rating. Independent reviewers, blinded to group allocation, will adjudicate accuracy. Findings will clarify the benefits and limitations of integrating Prof. Valmed into routine practice.

Who can participate

Healthy volunteers accepted: Yes

Only the study team can determine whether someone qualifies for participation.

Inclusion criteria

  • Participants must be licensed physicians.
  • Training in rheumatology, internal medicine, emergency medicine, family medicine, dermatology or orthopedics.

Exclusion criteria

  • Not currently practicing clinically.

Treatment and study plan

Prof. Valmed

Other

Prof Valmed. decision support system.

Primary outcomes

  1. Diagnostic accuracy of top diagnosis

    Time frame: directly (within 10 minutes) after Intervention

    Participants in each group will make at least one disease suggestion (top diagnosis) and up to a total of a maximum of 3 suggestions. Percentage of exact matches of the top suggestion with the actual diagnosis will be analyzed

Secondary outcomes

  1. Diagnostic accuracy of top 3 suggestions

    Time frame: directly (within 10 minutes) after Intervention

    Participants in each group will make at least one disease suggestion (top diagnosis) and up to a total of a maximum of 3 suggestions. Percentage of exact matches with the actual diagnosis included in the top 3 suggestions will be analyzed

  2. Diagnostic confidence

    Time frame: directly (within 10 minutes) after Intervention

    For each case participants will be asked for their diagnostic confidence (VAS 0-10). The mean score will be compared between groups.

  3. Time spent for diagnosis

    Time frame: directly (within 10 minutes) after Intervention

    We will compare how much time (in seconds) participants spend per case between the two study arms.

  4. Perceived Information Timeliness

    Time frame: directly (within 10 minutes) after Intervention

    Perceived ability to receive the information needed without delay (Likert scale from 1 to 5)

  5. Perceived diagnostic support quality

    Time frame: directly (within 10 minutes) after Intervention

    Perceived quality of the diagnostic support (Likert scale from 1 to 5)

  6. Diagnostic reasoning

    Time frame: during evaluation

    For each case, participants will receive 1 point for each plausible diagnosis and 2 points for a completely correct response. The total scores will be compared between the randomized groups.

Sponsors and collaborators

Lead sponsor

Philipps University Marburg

Other

Collaborators

  • Charite University, Berlin, Germany
  • Diakonhjemmet Hospital
  • Klinikum Fulda
  • Oslo University Hospital
  • Rheumazentrum Ruhrgebiet
  • University Hospital Erlangen
  • University of Lausanne Hospitals
  • Universitätsklinikum Hamburg-Eppendorf

Registry information

Acronym: ALLIANCE

Important dates

Study start
2025
Primary completion
2026
Study completion
2026
First posted
Sep 10, 2025
Registry last updated
Apr 8, 2026

OpenTrials presents study information sourced from ClinicalTrials.gov. The official registry record should be consulted for the latest information.

View the official ClinicalTrials.gov record (opens in a new tab)

This listing is for discovery and informational purposes only. It is not medical advice, does not guarantee that a study is recruiting, and does not determine eligibility. Contact the study team and a qualified healthcare professional when considering participation.

Related clinical trials

Published trials that share one or more normalized conditions with this study.