Performance Comparison of Large Language Models in TAP Block Ultrasound Interpretation

This study aims to evaluate how two advanced artificial intelligence (AI) models, Gemini 2.5 Pro and ChatGPT-5.1, interpret ultrasound videos of Transversus Abdominis Plane (TAP) block procedures. TAP blocks are performed as part of routine clinical care by experienced anesthesiologists. The ultrasound videos recorded during these procedures serve as the data source for this study. No additional procedures or patient involvement are required beyond standard care.

Ultrasound Video Processing All ultrasound recordings will be fully de-identified by removing patient names, dates, and any other identifying information.

Gemini 2.5 Pro will receive original video files. ChatGPT-5.1 will receive high-resolution GIF segments generated from the same recordings.

Both models will be given identical structured prompts consisting of eight clinically relevant questions about anatomic structures, needle placement, local anesthetic spread, dermatomal effects, and potential safety concerns.

Expert Participation

Two anesthesiology experts will participate independently:

Expert A will review each ultrasound video and answer the same set of eight clinical questions. These answers will serve as the primary human clinical reference.

Expert B will independently evaluate all responses, those from Expert A, Gemini, and ChatGPT-5.1, after they have been anonymized and randomly ordered. Expert B will not know whether a response originated from an AI model or a human expert. Expert B will assess anatomical accuracy, clarity, clinical appropriateness, and overall content quality for each answer.

If Expert A and Expert B disagree on the interpretation or quality assessment of any response, a third expert (Expert C), who is also experienced in ultrasound-guided regional anesthesia, will independently review the relevant responses. Expert C's evaluation will be used to resolve discrepancies and establish the final consensus.

Data Collected

For each TAP block video, the following information will be recorded:

Ultrasound and procedural details. Patient demographic descriptors (age, sex, BMI, ASA classification), used only for general characterization of the sample.

AI-related performance features such as response completeness, relevance, confidence level, and response time.

Inclusion criteria

• Adults aged 18-85 years
• ASA I-III physical status
• Undergoing elective surgery with a lateral TAP block performed as part of routine anesthesia care
• Complete ultrasound-guided block procedure recorded on video
• Able to provide written informed consent

Exclusion criteria

• Unsuccessful or incomplete TAP block procedure
• Poor-quality ultrasound video (needle tip or anesthetic spread not visible)
• Missing demographic or clinical data
• Withdrawal of consent at any time