AI Timing in Chest X-ray Interpretation Using Eye-Tracking

CREAITED is a behavioural diagnostic reader study examining how the timing of artificial intelligence (AI) decision-support information influences chest X-ray interpretation by healthcare professionals. The study focuses on whether showing AI information before or after a participant first reviews a chest X-ray affects visual search behaviour, interpretation time, clinical decision-making, confidence, and trust in AI support.

Chest X-rays are widely used in clinical practice, but interpretation can be challenging because findings may be subtle and normal anatomy can overlap with abnormalities. AI systems are increasingly used to support image interpretation by highlighting suspected abnormalities. However, less is known about how clinicians use AI information during reporting tasks, including whether seeing AI output early changes attention, reliance, confidence, or interpretation strategy. Current clinical guidance generally supports independent image review before consulting decision-support tools, but direct evidence on AI timing in chest X-ray interpretation remains limited. This study is designed to provide controlled evidence on that question.

The main reader study is a single-site study at University Hospitals of Leicester NHS Trust. It uses a within-subject, counterbalanced, multi-reader, multi-case design. The exposure of interest is the timing of AI decision-support presentation during chest X-ray interpretation. The AI information is used only within the research task and is not used to guide real patient care.

Healthcare professional participants will complete two chest X-ray interpretation sessions in a controlled NHS research setting. Participants will include clinicians and healthcare professionals who interpret, review, check, or act on chest X-ray findings as part of their current or recent professional role. The study is not designed to assess individual competence. Individual participant results will not be used for employment, appraisal, training progression, formal assessment, or performance management.

Each participant will complete both AI timing conditions. In one session, AI information will be shown before the participant reviews the chest X-ray. In the other session, the participant will first review the chest X-ray without AI information, and AI information will then be shown afterwards. The order of these two-timing conditions will be balanced across participants using a pre-generated allocation schedule. The two sessions will be separated by at least four weeks to reduce recall and learning effects.

Each case will use a two-phase interpretation process. In the first phase, only one information source is shown, either the original chest X-ray or the AI output, depending on the timing condition. In the second phase, the alternative information source is introduced so that both the chest X-ray and AI information can be viewed. Participants complete a structured interpretation response after the first phase and may review or revise their response after seeing both information sources before submitting their final response for that case.

During each session, participants will review de-identified chest X-ray images using a dedicated diagnostic reporting workstation in a controlled reporting environment. The reader study is designed for approximately 20 complete participant datasets. Each participant will review 25 study chest X-ray cases per session across two sessions. The same study image set will be used across both sessions, with case order independently randomised for each session. Warm-up cases from a separate image pool will be used at the start of each session to familiarise participants with the task and equipment, and these warm-up cases will not be included in the main analysis.

The image set will include a mix of normal and abnormal chest X-rays and a range of diagnostic difficulty. Cases will be selected from routine clinical imaging and de-identified before use in the study. Ground truth labels and image areas of interest will be defined through a structured clinical review process. These reference data will support analysis of participant interpretation, localisation, and visual search behaviour.

Participants will wear lightweight eye-tracking equipment during the interpretation task. Eye-tracking will record where participants look on the image and how long they spend looking at different areas. These data will be used to assess visual search behaviour, including how attention is distributed across image areas, how participants view areas highlighted by the AI, and whether visual search differs between the AI-first and chest X-ray-first timing conditions.

Study sessions will be self-paced, with optional breaks to reduce fatigue. A researcher will be present to monitor equipment and provide technical support if needed, while minimising interaction that could influence participant behaviour. After completing the reader study sessions, participants will complete a short questionnaire about confidence, trust, perceived usefulness, workflow impact, and the influence of AI timing. Participants will also be debriefed about the study design and the reason for comparing different AI timing conditions.

The study will collect de-identified imaging data, eye-tracking data, structured reporting responses, timing data, localisation information, and questionnaire responses. The primary outcome is case interpretation time, measured for each chest X-ray case. Other outcomes include diagnostic performance, clinical interpretation or management responses, confidence in decision-making, participant perceptions of AI support, and eye-tracking measures of visual search behaviour.

The reader study uses de-identified chest X-ray images from existing examinations. No patients are recruited directly into the reader study, no clinical care is changed, and no clinical decisions will be made from participant study readings. The study does not involve investigational medicinal products, and no biological samples are collected.

Patient and public involvement informed the development of the study. Input from patients, members of the public, healthcare staff, and other stakeholders helped refine the focus on trust, confidence, transparency, human oversight, communication preferences, and the timing of AI information. This input also informed participant-facing materials and the topic areas included in the supplementary survey.

Alongside the main reader study, a separate anonymous supplementary survey will collect wider views about AI use in chest X-ray interpretation. This survey will be administered remotely and will be open to relevant groups, including clinicians, patients, members of the public, and healthcare staff. It will explore views on trust, safety, responsibility, communication, acceptability, governance, and expectations of AI-supported chest X-ray interpretation. The supplementary survey is independent of the reader study sessions and does not influence the image interpretation task.

Study data will be handled using study identifiers and secure NHS systems. Results will be reported in aggregate form so that individual participants are not identified. The findings are intended to help inform safer and more effective use of AI decision support in chest X-ray interpretation.

Inclusion criteria

Main reader study:

• Healthcare professionals aged 18 years or over
• Registered to practise in the UK
• Employed by the NHS or another healthcare service operating in the UK
• Current or recent, within the last 3 years, clinical experience involving chest X-ray interpretation, review, or use in clinical practice
• Able to attend two onsite study sessions at University Hospitals of Leicester NHS Trust at mutually agreed times
• Able and willing to provide written informed consent
• Compatible with the eye-tracking equipment

Supplementary survey:

• Adults aged 18 years or over
• Live in the UK or have used the NHS or another UK healthcare service within the last 5 years
• Able to provide informed electronic consent
• Belong to one of the following respondent groups: healthcare professionals or healthcare staff, patients or carers, or members of the public
• Healthcare professional respondents may include adults involved in requesting, interpreting, checking, or acting on chest X-ray findings in clinical practice

Exclusion criteria

Main reader study:

• Inability to attend both onsite study sessions at University Hospitals of Leicester NHS Trust
• Eye-tracking incompatibility, such as visual, neurological, or physical conditions preventing adequate gaze tracking or participant comfort
• Direct involvement in selection, adjudication, or preparation of the chest X-rays used in the study
• Conflicts of interest, including direct involvement in development of the AI system under evaluation
• Prior participation in a closely related AI chest X-ray study where overlap in image sets or study procedures may compromise validity, assessed on a case-by-case basis

Supplementary survey:

• Aged under 18 years
• Does not live in the UK and has not used the NHS or another UK healthcare service within the last 5 years
• Unable to provide informed electronic consent
• Does not meet one of the eligible respondent groups for the survey