Agreement Between Large Language Model-Generated Treatment Recommendations With Guideline-Based and Tumor Board Decisions in Gastrointestinal Cancer

Gastrointestinal cancers require complex treatment planning that often involves surgery, systemic therapy, and multidisciplinary coordination. Clinical decision-making is typically guided by evidence-based recommendations and discussed in multidisciplinary tumor boards. However, the increasing complexity of treatment strategies and guideline frameworks can make consistent and reproducible decision-making challenging in routine clinical practice.

Recent advances in artificial intelligence have enabled the development of large language models (LLMs) that can process structured clinical information and generate text-based recommendations. These systems may offer a scalable approach to support clinical workflows, but their ability to produce reliable and clinically appropriate treatment suggestions in oncology remains uncertain.

This study evaluates the performance of an LLM-based system in the context of gastrointestinal oncology using retrospectively collected clinical case data. Structured case summaries derived from routine clinical documentation are used as standardized input. The model generates treatment recommendations under controlled conditions, allowing systematic comparison with established clinical reference standards.

The analysis focuses on the level of agreement between model-generated recommendations and established decision-making frameworks. In addition, the study explores how model performance varies across different clinical scenarios, including varying levels of disease complexity. Particular attention is given to situations in which recommendations differ, in order to better understand potential limitations of the model and identify patterns that may be clinically relevant.

Furthermore, the study examines the consistency of model outputs when the same clinical information is processed multiple times. This provides insight into the stability and reproducibility of the system, which are important considerations for potential real-world use.

The findings of this study are intended to inform the potential role of LLM-based tools as supportive systems in clinical decision-making. The study does not evaluate clinical outcomes or patient benefit, but instead focuses on agreement with established standards and expert-driven decisions as an initial step in assessing feasibility and safety.

Inclusion criteria

• Histologically confirmed pancreatic, gastric, or colorectal adenocarcinoma
• Treatment discussed in a multidisciplinary tumor board

Exclusion criteria

• Non-adenocarcinoma histology