AI-Orchestrated Workflow Versus Consultant Ophthalmologist for Refractive Surgery and Keratoconus Diagnosis (AEYE Trial)

Technical Protocol Summary

This is a diagnostic performance study evaluating AEYE (Automated Evaluation for Your Eye), an orchestrated multi-agent artificial intelligence (AI) system designed to assist ophthalmologists in the screening and management of keratoconus and refractive surgery planning. AEYE combines large language models (LLMs) with deterministic code logic to replicate and support the clinical decision-making process typically performed by anterior segment specialists.

System Architecture and Workflow

AEYE is structured as a modular pipeline of three specialized agents, each focused on a distinct diagnostic task:

History and Risk Agent: Extracts structured data from unstructured clinical records, including demographics, ocular/systemic history, medication use, and risk factors relevant to keratoconus or refractive surgery eligibility.

Imaging Analysis Agent: Processes multimodal anterior segment imaging such as Scheimpflug-based tomography (e.g., Pentacam), anterior segment optical coherence tomography (AS-OCT), and epithelial thickness mapping. It standardizes key metrics like maximum keratometry (Kmax), thinnest corneal pachymetry, anterior/posterior elevation, Belin-Ambrósio Deviation Index (BAD-D), and Pachymetric Progression Index (PPI). Each eye and each imaging file is processed independently to avoid misattribution.

Surgical Decision Agent: Integrates the outputs of the previous agents to generate a final diagnosis, assign keratoconus staging (e.g., ABCD, Amsler-Krumeich classification), and recommend next steps, such as LASIK (Laser-Assisted In Situ Keratomileusis), PRK (Photorefractive Keratectomy), SMILE (Small Incision Lenticule Extraction), phakic intraocular lenses (ICL), or corneal collagen cross-linking (CXL). The output is a structured, auditable report.

All agents are controlled by a deterministic workflow using Python scripts for data merging, schema validation, and output formatting. Structured memory is maintained using JSON objects that store the full diagnostic context per patient. The system is designed to ensure reproducibility, reduce human variability, and support explainable clinical decision-making.

Study Workflow and Scope

Fifty real-world patient cases are included, comprising both retrospective records (from January 2020 onward) and newly enrolled prospective cases. Each case includes comprehensive clinical data and anterior segment imaging. AEYE analyzes the case and generates a structured report. Separately, an experienced ophthalmologist (blinded to the AI output) reviews the same data and records their clinical decisions.

Key metrics include:

Diagnostic accuracy of keratoconus detection. Agreement in surgical eligibility assessments. Efficiency of workflow execution. Variability in results across different LLMs used in agent roles. Cases with discordant results may undergo adjudication to establish a reference standard.

Innovation and Clinical Relevance.

AEYE represents a novel application of explainable AI in ophthalmology. Its multi-agent design reflects a divide-and-conquer strategy, reducing cognitive load on any single model while enforcing clinical safety through deterministic logic. The system supports scalability, modularity, and integration into electronic health record (EHR) systems. The study will help determine whether AEYE can function as a safe, consistent, and effective assistant in complex diagnostic pathways for corneal ectatic disease.

Inclusion criteria

• Diagnosis of Refractive Error or Keratoconus:

• Patients with a confirmed clinical diagnosis of refractive errors (myopia, hyperopia, or astigmatism) or keratoconus, as determined by a qualified ophthalmologist based on current diagnostic standards.
• Availability of Complete Data:

• Clinical records must include comprehensive demographic data, medical and ophthalmic history, refraction measurements, and high-quality corneal imaging (e.g., corneal topography, tomography, or OCT).
• Eligible for Both AI and Consultant Review:

• Patients must have data sets that allow both the multi-agent AI workflow (AEYE) and an independent consultant ophthalmologist to conduct a complete diagnostic assessment.
• Consent:

• For prospectively enrolled patients, written informed consent must be obtained prior to participation. For retrospective cases, waiver of consent may be granted according to local IRB/ethics committee policies.
• No Restriction on Age or Sex:

• There are no specific age or sex limitations for inclusion in this study. Pediatric and adult cases may be included if data are available and consent is appropriately obtained.

Clinical Documentation Requirements:

• Availability of complete ophthalmic records including:
• Uncorrected and best corrected visual acuity (UCVA, BCVA)
• Manifest refraction (sphere, cylinder, axis)
• History of contact lens use and duration
• Prior ocular surgery or trauma, if applicable
• Family history of keratoconus or corneal ectasia
• Use of medications that may influence tear film stability or corneal biomechanics
• Ocular surface disease documentation if present (e.g., dry eye, blepharitis)

Imaging and Diagnostic Data Requirements:

• Availability of at least one valid and complete set of:
• Scheimpflug-based corneal tomography (e.g., Pentacam or equivalent)
• Anterior segment OCT (AS-OCT), if performed
• Epithelial thickness mapping (optional but desirable)
• Corneal biomechanical data (optional)
• Imaging files must be:
• Of adequate quality as judged by the original technician and reviewer
• Free from significant artifacts or motion blur
• Properly labeled by eye (OD/OS) and imaging modality
• Acquired with standard scan protocols

Diagnostic Spectrum Requirements:

• Patients may fall into any of the following diagnostic categories:
• Normal cornea eligible for laser refractive surgery
• Suspect keratoconus or forme fruste keratoconus
• Established keratoconus of any stage (I-IV or ABCD)
• Post-CXL (cross-linking) cornea undergoing follow-up evaluation
• Post-keratoplasty cornea (e.g., DALK, PKP) undergoing diagnostic review
• Patients disqualified from surgery due to abnormal tomography or other contraindications.

Exclusion criteria

• Incomplete or Poor Quality Data:

• Patients with missing, incomplete, or poor-quality clinical data or corneal imaging that precludes reliable diagnosis by either AI or consultant review.
• Ocular Comorbidities:

• Presence of ocular diseases that could confound the diagnosis or interpretation of refractive error or keratoconus, such as advanced glaucoma, active uveitis, significant retinal pathology, or previous corneal transplantation.
• Severe Systemic Disease Affecting the Eye:

• Patients with systemic diseases known to affect the cornea or refraction (e.g., connective tissue disorders with corneal involvement) will be excluded to avoid confounding effects.
• Inability or Refusal to Consent:

• For prospectively enrolled cases, patients (or guardians, in the case of minors) who are unwilling or unable to provide informed consent will be excluded.
• Participation in Conflicting Studies:

• Patients currently enrolled in other interventional studies that could interfere with the diagnostic process or outcomes measured in this study.

Clinical Documentation Exclusions:

• Missing or incomplete visual acuity data.
• Absence of manifest refraction or subjective refraction data.
• Incomplete or conflicting patient history with unresolvable discrepancies.
• Absence of necessary demographic identifiers (e.g., gender, age, laterality of findings).

Imaging and Data Quality Exclusions:

• Imaging datasets with significant artifacts that preclude AI analysis.
• Files missing key parameters (e.g., Belin/Ambrósio D index, Kmax, pachymetry) required for keratoconus classification.
• Inability to determine laterality (OD/OS) for specific scans.
• Scans acquired with outdated or non-standard imaging protocols.
• Mislabeling or duplication of imaging sets across different patients.

Case-Type Exclusions:

• Cases involving active ocular infection or inflammation at the time of evaluation.
• Cases with active herpetic eye disease, corneal ulcer, or epithelial defects interfering with imaging interpretation.
• Prior laser refractive surgery unless explicitly classified and used as part of post-refractive ectasia evaluation.
• Eyes with concurrent advanced retinal pathology that may confound diagnostic interpretation (e.g., macular edema, retinal dystrophies).

Technology-Specific Exclusions:

• Imaging performed using devices not compatible with the AEYE data ingestion framework (e.g., certain older corneal topographers).
• Non-English documentation incompatible with NLP parsing in the General Ophthalmology Agent.

Other Considerations:

• AEYE must be able to process the case within its defined schema. If key data elements are absent or the case causes schema failure, it will be excluded from quantitative analysis but may be reported qualitatively as part of system performance limitations.
• If multiple attempts at AI processing result in non-deterministic or erroneous outputs, such cases will be logged but excluded from primary outcome evaluation.
• Human reviewers (consultant ophthalmologists) must have completed their diagnostic interpretation independently prior to AEYE input; any cases where human opinion was influenced by AI output will be excluded to preserve blinding.