Evaluating the Use of Artificial Intelligence to Improve Family Conversations for Intensive Care Patients and Their Families

The primary outcome measure is the documentation quality of family conversations, evaluated using a modified Physician Documentation Quality Instrument-9. Originally validated for progress notes and discharge summaries, Physician Documentation Quality Instrument-9 scores notes on nine attributes (accurate, thorough, useful, organized, comprehensible, succinct, synthesized, consistent, up-to-date) using a 1-5 scale (1 = "not at all," 5 = "extremely"), for a total ranging from 9 to 45. To adapt it for AI-generated text, the investigators removed the "up-to-date" domain (focused on whether the note includes the latest test results/recommendations) and added two new attributes: freedom from hallucinations (unfounded information) and freedom from bias (discriminatory data, algorithms, or heuristics). These additions address potential pitfalls in large language model outputs, such as introducing incorrect content or skewed results.

The time clinicians spend on post-conversation documentation will be measured for both traditional and ambient listening methods. This outcome aims to quantify the efficiency of the ambient listening technology in reducing administrative burden.

Family members' feedback on their satisfaction with the intensive care unit (ICU) communication process. Satisfaction will be measured using a structured questionnaire administered after reviewing a simplified ambient-generated note. The survey includes 16 questions: 5 about demographics (age, education, occupation, and reading skills); 3 adapted from the validated Family Satisfaction with the ICU-24 survey (score range: 1-5, higher scores indicate greater satisfaction) to assess clarity, completeness, and overall satisfaction; and 10 evaluating the summary's readability and utility at a B1 language proficiency level.