Skip to main content
OpenTrials
Completed

NCT Number: NCT06334796

Artificial Intelligence-powered Virtual Assistant for Emergency Triage in Neurology

This study examines the use of an AI-powered virtual assistant for quickly identifying and handling neurological emergencies, particularly in places with limited medical resources. The research aimed to check if this AI tool is safe and accurate enough to move on to more advanced testing stages. In a first-of-its-kind trial, the virtual assistant was tested with patients having urgent neurological issues. Neurologists first reviewed the AI's recommendations using clinical records and then assessed its performance directly with patients. The findings were as follows: neurologists agreed with the AI's decisions nearly all the time, and the AI outperformed earlier versions of Chat GPT in every tested aspect. Patients and doctors found the AI to be highly effective, rating it as excellent or very good in most cases. This suggests the AI could significantly enhance how quickly and accurately neurological emergencies are dealt with, although further trials are needed before it can be widely used.

Completed

Looking for future studies?

Notify Me

Key information

Conditions

Stroke Autoimmune Diseases Autoimmune Diseases of the Nervous System Benign Paroxysmal Positional Vertigo Brain Diseases Cardiovascular Diseases Central Nervous System Diseases Cerebrovascular Disorders Chronic Disease Confusion Cranial Nerve Diseases Delirium Demyelinating Diseases Disease Attributes Ear Diseases Facial Nerve Diseases Facial Neuralgia Facial Palsy Facial Paralysis Guillain-Barre Syndrome Headache Disorders Headache Disorders, Primary Hemorrhage Immune System Diseases Intracranial Hemorrhages Labyrinth Diseases Meningitis Mental Disorders Migraine Migraine Disorders Mouth Diseases Nervous System Diseases Neurobehavioral Manifestations Neurocognitive Disorders Neuroinflammatory Diseases Neurologic Manifestations Neuromuscular Diseases Otorhinolaryngologic Diseases Paralysis Pathologic Processes Pathological Conditions, Signs and Symptoms Peripheral Nervous System Diseases Polyneuropathies Polyradiculoneuropathy Post-Infectious Disorders Seizures Signs and Symptoms Status Epilepticus Stomatognathic Diseases Subarachnoid Hemorrhage Trigeminal Nerve Diseases Trigeminal Neuralgia Vascular Diseases Vertigo Vertigo Benign Positional Vestibular Diseases

Age range

18 year and older

Sex eligibility

All sexes

Study type

Interventional

Phase

Early Phase 1

Primary location

Fleni

Buenos Aires, 1428, Argentina

About this study

Background and Objectives: Neurological emergencies pose significant challenges in medical care, especially in resource-limited countries. Artificial Intelligence (AI), particularly health chatbots, offers a promising solution. However, rigorous validation is required to ensure safety and accuracy. The objective of our work is to evaluate the diagnostic accuracy and resolution effectiveness of an AI-powered virtual assistant designed for the triage of emergency neurological pathologies, to ensure the minimum standard of safety that allows for the progression to successive validation tests.

Methods: This Phase 1 trial evaluates the performance of an AI-powered virtual assistant for emergency neurological triage. Ten patients over 18 years old with urgent neurological pathologies were selected. In the first stage, nine neurologists assessed the safety of the virtual assistant using their clinical records. In the second part, the assistant's accuracy when used by patients was evaluated. Finally, its performance was compared with Chat GPT 3.5 and 4.

Who can participate

Healthy volunteers accepted: No

Only the study team can determine whether someone qualifies for participation.

Inclusion criteria

  • Patients over 18 years old consulting in the ER due to a neurological emergency

Exclusion criteria

  • Pregnancy

Treatment and study plan

Virtual Assistant

Diagnostic Test

Stage 1 focused on safety, using only medical information from clinical records for the virtual assistant. In Stage 2, which evaluated accuracy, participants interacted with the virtual assistant post-medical stabilization. Additionally, participants also provided initial symptom details for Chat-GPT input. Nine neurologists specializing in emergency participated in the study. In Stage 1, they assessed the virtual assistant's performance using clinical history information. In Stage 2, they analyzed the results from participant interactions with the assistant and performed a comparative evaluation of Chat-GPT. The virtual assistant functioned as a chatbot on WhatsApp and Telegram, using Spanish and incorporating advanced algorithms, decision trees, and large language models for interaction. For comparison, we utilized Chat-GPT versions 3.5 and 4, employing two prompt types in natural Spanish: one incorporating clinical record data and the other based on participant narratives.

Primary outcomes

  1. Diagnostic performance

    Time frame: The first interaction between participants and the virtual assistant occurred within less than a year after the event. Outcome measures were evaluated immediately after the interaction between patients and the virtual assistant.

    Refers to the accuracy and effectiveness of medical tests or diagnostic tools in correctly identifying a disease or condition in patients.

    Syndromic diagnosis agreement: evaluating neurologists considered a syndromic diagnosis accurate when AI tools could identify a condition based on a set of commonly coexisting signs and symptoms, rather than identifying a specific disease. This method is applied when the precise disease causing the symptoms is not immediately identifiable, allowing healthcare providers to effectively monitor and treat the patient's presenting symptoms.

    Differential diagnosis agreement: a differential diagnosis was considered accurate when the differentials provided by each AI tool matched those presented by the participants.

    The gold standard for diagnosis was considered to be the one given in the emergency department, unchanged over a one-month period.

Secondary outcomes

  1. Appropriate medical conduct or recommendation

    Time frame: The first interaction between participants and the virtual assistant occurred within less than a year after the event. Outcome measures were evaluated immediately after the interaction between patients and the virtual assistant.

    Case resolution was evaluated based on appropriate medical conduct or recommendation, categorizing 1) urgency as immediate, 2) short-term (within 48 hours), 3) or non-urgent.

    The recommendations provided by each AI tool were assessed based on information gathered from clinical histories and input from participants.

    The gold standard of appropriate medical conduct or recommendation was considered to be that given in the emergency department, with no changes over a period of one month.

  2. Assessment of Usability and Satisfaction

    Time frame: The first interaction between participants and the virtual assistant occurred within less than a year after the event. Outcome measures were evaluated immediately after the interaction between patients and the virtual assistant.

    Usability was measured by the time and number of questions needed for final diagnosis and resolution, both by neurologists and participants. For Chat GPT, we evaluated the time taken to draft the consultation reason.

    A satisfaction scale from 1 to 5 was implemented, with 1 indicating a negative experience ("poor", potentially risky for the patient) and 5 highly positive ("excellent", potentially surpassing non-specialized human triage). A simple yes/no survey was also applied to participants, asking about the comprehensibility of the assistant's questions, the adequacy of referral according to urgency, and whether they considered the assistant could replace non-specialized triage or reduce emergency arrival time.

Sponsors and collaborators

Lead sponsor

Fundación para la Lucha contra las Enfermedades Neurológicas de la Infancia

Other

Collaborators

  • Entelai

Registry information

Official study title

Phase 1 Trial of the Implementation of an Artificial Intelligence-powered Virtual Assistant for Emergency Triage in Neurology

Acronym: AIDEN

Important dates

Study start
2023
Primary completion
2024
Study completion
2024
First posted
Mar 28, 2024
Registry last updated
Mar 28, 2024

OpenTrials presents study information sourced from ClinicalTrials.gov. The official registry record should be consulted for the latest information.

View the official ClinicalTrials.gov record (opens in a new tab)

This listing is for discovery and informational purposes only. It is not medical advice, does not guarantee that a study is recruiting, and does not determine eligibility. Contact the study team and a qualified healthcare professional when considering participation.

Related clinical trials

Published trials that share one or more normalized conditions with this study.