Ambient Scribe in General Practice: a Multi-perspective Before-after Longitudinal Mixed-methods Study

Introduction Over the years the introduction of the electronic health record (EHR) and escalating demands for documentation have led to a mounting burden on GPs. Clinical documentation is a major barrier to efficient patient care as last year more than half the GPs spend more than 20% of their time on administrative duties. This burden leads to reduced job satisfaction and well-being, increased rates of burnout, and employee attrition. The documentation burden also negatively influences patient-provider communication. It leads to GPs making less eye contact, having a more closed body posture, and conveying less information to their patients. The documentation in the EHR however also has positive effects, such as reduced cognitive load and improvements in patient safety and care. Reducing the provider-computer interaction during the consultation may improve patient-provider interaction and provider well-being while retaining the benefits of EHRs.

In fact, studies showed that employing medical assistants for documentation during consultations leads to an increase in face-to-face time and improves patient satisfaction. Moreover, speech-to-text technologies for dictation after the consultation alleviated the documentation burden by increasing documentation speed, patient experience, and provider satisfaction. These interventions may lower burnout and attrition rates, strengthening the well-being of the providers as well as that of the healthcare sector.

The rapid advancement of large language models (LLMs) has opened new avenues to reduce documentation burden. LLMs, such as ChatGPT, are artificial intelligence (AI) models that can interpret and generate text. These models can transcribe and summarize a consultation with the GP in real-time removing the need for medical assistants or dictation after the consultation. However, if the LLM makes mistakes, this may lead to increased administrative workload. The investigators aim to assess the effect of a AI-based transcription and reporting tool in general practice.

Objectives Our primary objective is to assess the effect of a transcription and reporting tool on time spent on clinical documentation in general practice.

Secondary objectives are to assess the effect of a transcription and reporting tool in general practice on

• Total consultation time
• GP experience
• Patient experience
• Documentation length and quality

To assess the usage rates of the tool To assess the acceptability of use of the tool

Design This is a longitudinal before-after study. For each GP the investigators will observe two days of consultations without the tool. This will be done two weeks before implementation of the tool (baseline period). The investigators will also observe two days of consultations with the tool. This occurs two weeks after implementation of the tool (intervention period). Consultations from the intervention period will be compared with the baseline period to assess effectiveness of tool on the various outcomes.

Sample size A sample size simulation showed the investigators would need 30 observations to have enough power for the primary outcome. To improve the generalizability of our research and increase the sample for the qualitative outcomes, the investigators aim to observe in total four days of consultations for 12 GPs from at least five different practices. Practices will be sampled purposively for differences in patient population and practice organization.

Outcomes The investigators will measure time outcomes through continuous observation. An observer will monitor the time spent on various tasks during a consultation, including taking the medical history, conducting the physical examination, explaining the diagnosis or treatment plan, consulting a colleague, clinical documentation, and administrative duties like prescribing or referring.

Patient experience of the consultation will be measured with a validated patient experience questionnaire (PEQ) for general practice. Patient attitude on the use of the tool during the consultation will be expanded upon with a semi-structured interview based on prior research. The experiences of the GP with and impact of the tool on clinical duties will be measured with semi-structured interviews based on prior research. The acceptability and use of the tool by GPs will be assessed according to the unified theory of acceptance and use of technology (UTAUT).

Usage rates of the tool will be measured by assessing the proportion of consultations in which the tool is used. Documentation volume will be measured by determining the length of documentation in words. The documentation's information quantity will be measured by the number of relevant clinical variables in the note.

General practitioners:

Inclusion:

• Planning to implement the AI-based transcription and reporting tool
• Give informed consent for observation and/or interview and/or questionnaire

Exclusion:

• Insufficient knowledge of the Dutch language to be interviewed

Patients:

Inclusion:

• Give informed consent for questionnaire and/or interview
• For questionnaire: had a consultation with the GP
• For interview: had a consultation with the GP in which the tool was used

Exclusion:

• Insufficient knowledge of the Dutch language to be interviewed