Evaluation and Optimization of Telephone Triage Using Artificial Intelligence (AI) Models for the Detection of Demands for Time-dependent Pathology at the Emergency and Urgent Care Coordination Center (CCUE).

The Emergency and Urgent Care Coordination Center in Andalusia (CCUE) receives thousands of calls daily, during which each case must be classified by severity level based on the information provided over the phone. The conditions for which citizens seek help span a wide range-from minor ailments to cardiac arrest. This project addresses the challenge of telephone triage in out-of-hospital emergencies for several frequent reasons for care requests that may indicate emergent and potentially life-threatening medical conditions: unconsciousness/cardiac arrest, respiratory distress, non-traumatic chest pain, and stroke.

The goal is to improve the accuracy and efficiency of telephone triage using advanced Artificial Intelligence (AI) techniques-both symbolic and generative-including machine learning (ML) and natural language processing (NLP). This will enable CCUE CES-061 Andalucía operators to make faster, more informed decisions to provide timely and appropriate care.

The investigators will collect anonymized historical data from calls related to these four care request categories, extracted from the relational database systems of the Networked Coordination Centers (CCR) and the Mobile Digital Health Record (HCDM) of CES-061 Andalucía. The analysis will include both structured data (predefined fields with specific formats, including triage questions asked during the call) and unstructured data (free text and other formats) generated during demand management, encompassing coordination and care delivery aspects.

The investigators will implement a hybrid approach that integrates classical AI techniques (supervised and deep learning for classification) with generative AI (large language models to analyze and extract valuable insights from unstructured data). Various classification algorithms-such as decision trees, random forests, SVM, XGBoost, ensemble methods, and neural networks-will be tested to build the most accurate predictive model.

Through feature importance analysis, we will identify the most predictive questions and variables, proposing modifications to the current triage questions to enhance prediction accuracy.

The model will be evaluated using multiple metrics (including accuracy, sensitivity, specificity, positive and negative likelihood ratios, false positive rate [alarm failure], false negative rate [omission failure], area under the ROC curve, and F1-score). These metrics will help assess the model's ability to correctly predict final diagnoses-maximizing detection of truly urgent cases (high sensitivity, high positive likelihood ratio, and low false negative rate) while avoiding excessive false positives (high specificity, low false positive rate, and low negative likelihood ratio).

The aim of this project is to assess the effectiveness of the current CCUE telephone triage model and develop a new AI-based model to improve it. We aim to provide a solid foundation for future implementation of the improved model within CCUEs, helping personnel to quickly identify and prioritize the most severe cases, ultimately reducing response times and improving health outcomes.

Inclusion criteria

Telephone calls recorded with codes A36 + A58 (unconsciousness/cardiorespiratory arrest), A16 (respiratory distress), A23 (non-traumatic chest pain) and A54 (stroke).

Exclusion criteria

• Demands with relevant information about the patient or the event incomplete or absent.