COLORS-Validate Study of Decision-Support Software for Surgical Sequencing in Colorectal Liver Metastases

The selection of surgical sequencing-whether to resect the primary tumor first or the liver metastases first-in patients with colorectal cancer liver metastases (CRLM) undergoing synchronous resection represents a critical and highly complex decision in current clinical practice. With the advancement of systemic therapies and surgical techniques, an increasing number of patients with initially unresectable CRLM have gained the opportunity for surgical treatment. However, due to substantial heterogeneity among patients in terms of tumor burden, biological behavior of the disease, and overall physical condition, no unified standard currently exists for determining the optimal surgical sequence. Existing studies comparing different surgical strategies have yielded inconsistent prognostic results, and most are based on retrospective analyses lacking consistent conclusions and clear stratification criteria. Consequently, in real-world clinical practice, such decisions rely heavily on physician experience and institutional preference, resulting in considerable inter-physician variability and uncertainty. This scenario, characterized by the absence of a clearly optimal strategy, renders the decision-making process inherently a form of decision-making under uncertainty.

Based on this context, we previously developed, using multicenter retrospective data, two multi-outcome predictive models corresponding to different surgical strategies. These models integrate postoperative major complications, Comprehensive Complication Index (CCI), progression-free survival (PFS), and overall survival (OS), and have been externally validated. However, in real-world clinical decision-making, the central challenge faced by surgeons is not merely the prediction of a single outcome, but rather the need to balance multiple competing outcomes. For example, one surgical strategy may be associated with a lower risk of postoperative complications but limited long-term survival benefit, whereas another may involve higher perioperative risk in exchange for improved oncological outcomes. This inherent tension among multidimensional outcomes transforms clinical decision-making into a multi-objective decision-making problem, rather than a simple comparison of a single endpoint.

In current practice, such trade-offs across outcomes are typically based on physicians' experiential judgment, whereby the relative importance of different outcomes is implicitly weighted. However, this weighting process lacks explicit representation and standardized criteria, and is prone to influence by individual experience, risk preference, and institutional norms, thereby leading to substantial variability and inconsistency in decision-making. To address this key limitation, the present study further constructs a structured multi-outcome weighting framework using the Delphi method combined with the analytic hierarchy process (AHP). This approach establishes the relative importance of different clinical outcomes through expert consensus and translates it into a quantifiable weighting system, thereby enabling the integration of multidimensional predictive results into a comparable composite score. Based on this framework, we developed a decision-support software system capable of simultaneously providing individualized multi-outcome predictions and composite reference scores, transforming the previously experience-dependent implicit trade-off process into a transparent, interpretable, and standardized decision-support process.

Nevertheless, the establishment of predictive models and weighting frameworks does not necessarily translate into practical clinical value. For decision-support systems, focusing solely on predictive performance is insufficient to reflect their real-world clinical utility. More importantly, it is necessary to evaluate whether such systems can influence physician decision-making behavior under uncertainty, reduce unnecessary variability, and provide a structured reference framework for decision-making. In addition, because clinical decision-making involves counterfactual comparisons (i.e., potential outcomes under alternative strategies for the same patient cannot be simultaneously observed), traditional outcome-based evaluation methods have inherent limitations. Therefore, it is necessary to adopt a model-informed standardized reference framework to evaluate the impact of AI assistance at the level of decision behavior.

Based on the above considerations, this study adopts a multicenter, prospective crossover simulation design to systematically evaluate the clinical utility of the decision-support system in a standardized setting. The evaluation focuses on the following two dimensions:

• From a scientific decision perspective: whether the system alters physician decision behavior, improves decision consistency, and reduces variability;
• From a human factors perspective: whether the system affects cognitive load, decision confidence, and user experience.

This study extends AI evaluation beyond predictive performance toward its influence on decision-making behavior, with a particular focus on the human-AI collaborative decision-making process in uncertain clinical scenarios.

Inclusion criteria

• Licensed physicians specializing in hepatobiliary surgery or colorectal surgery
• At least 1 year of clinical experience after graduation
• Willing to participate in the simulation-based decision-making study
• Practicing at one of the participating tertiary academic hospitals

Exclusion criteria

• Not actively involved in clinical surgical decision-making
• Unable to complete all required simulation sessions
• Prior involvement in the development of the decision-support software