Better Real-time Information on Documentation of Goals of Care for Engagement in Serious Illness Communication

Primary outcome: We will assess SIC summary accuracy by reviewing the source documentation for each bullet point of the 5 domains of the SIC summary. The RA will read each bullet point, read the source note for the date cited in the summary and indicate with a 1 (supported) or 0 (not supported) whether the documentation supports the summarized point. If a summarized point is scored as 0, the MPIs will independently review the item and assign a 1 or 0 to indicate whether the unsupported summary could negatively impact patient care. SIC summary accuracy will be determined at the patient level as the proportion of bullet points supported by underlying documentation divided by the total number of bullet points (i.e., a score from 0 to 1). We will determine the average SIC accuracy score across all patients for the study. We will also report the percent of LLM-SIC summary statements that are unsupported and clinically significant and provide qualitative descriptions of these instances.

Feasibility is defined as the successful delivery of the summary email, and will be calculated as the number of participants with successful delivery of the intervention email divided by the total number of intervention participants, with an 80% feasibility threshold.

Clinician perspectives of clinical utility are defined as clinicians' views on the usefulness and relevance of serious illness communication (SIC) and, for intervention-arm clinicians, the LLM-generated SIC summaries for informing participant care. These perspectives will be assessed via structured REDCap surveys sent to both inpatient and outpatient clinicians, with follow-up phone interviews conducted by the research assistants for non-respondents.

Clinician responses to surveys and interview questions will be summarized and stratified by intervention vs control groups. We will compare survey responses between the two study arms using Chisquared and Mann-Whitney U tests for binary and likert items, respectively. Responses to open-ended questions will be qualitatively summarized.

Participant acceptability is defined as participant perceptions of the appropriateness and usefulness of LLM-generated summaries of serious illness conversations (SICs). It will be assessed through structured in-person interviews including the validated "Heard and Understood" questionnaire and open-ended questions regarding the acceptability of SIC summaries and participant-clinician communication.

Participant responses to interview questions will be summarized and stratified by intervention vs control groups. Responses to open-ended questions will be qualitatively summarized.

New documentation of serious illness communication (SIC) during the index hospitalization is defined as any new note or entry created in the electronic health record (EHR) during the current hospital admission that reflects a discussion of the participant's values, goals, preferences, or prognosis, and meets the standardized codebook criteria for SIC.

Length of hospitalization in days for the admission triggering enrollment (data on death during the admission, as well as discharge to hospice and re-admission within 30 days will also be collected)

Number of days in the hospital over the subsequent 90 days

Number hospitalizations over the subsequent 90 days

Number of intensive care unit admissions over the subsequent 90 days

Any occurrence of intubation over the subsequent 90 days

Vital status, including date of death, over the subsequent 90 days