Assessing the Effectiveness of Large Language Model (LLM)-Enabled Nurse Treatment Planning in 2 Indian Districts

This outcome assesses the quality of clinical care by comparing AI-assisted, nurse-led consultations with standard physician-led consultations. For patients with hypertension or diabetes mellitus, clinical quality is measured using a clinical management rubric with a raw score range of -2 to 7, assessing data review, complication screening, medication adherence, counseling, and treatment planning, with penalties for inappropriate counseling or treatment. For patients presenting with fever, breathlessness, or musculoskeletal pain, clinical quality is measured using a clinical reasoning rubric with a raw score range of -5 to 10, assessing differential diagnoses, final diagnosis, and next steps, with negative scores for harmful recommendations. Consultations are audio recorded, de-identified, and scored by blinded physicians. Higher scores indicate better alignment with evidence-based, context-appropriate care.

Composite patient experience score derived from an 9-item Likert-scale exit survey adapted from the WHO Health System Responsiveness framework and PSQ-18. Items assess communication, understanding, dignity/respect, trust in AI use, and overall satisfaction. Responses are scored 1-5 per item and averaged to generate a composite score (range 1-5), with higher scores indicating more positive experience.

Qualitative assessment of nurse-reported usability, trust in AI recommendations, workflow impact, barriers, facilitators, and willingness to continue use. Interviews are audio recorded and thematically analyzed. Outcomes will be reported as identified themes with representative quotations and frequency of theme occurrence across participants.