Evaluating an AI-Generated Health Podcast

Background and Rationale Recent advances in digital media have underscored the potential of podcasts as an innovative medium for disseminating healthcare information. Traditional health podcasts, while valuable, often suffer from limitations such as lengthy duration, inconsistent quality, and the use of complex medical jargon that may hinder public understanding. In contrast, artificial intelligence (AI)-driven content generation offers an opportunity to create concise, standardized, and accessible audio content. This study leverages Google NotebookLM to generate a podcast on gum health and prevention using publicly available information from the American Academy of Periodontology (AAP). By doing so, the research aims to explore whether an AI-generated health podcast can effectively enhance public health literacy.

Objectives

The primary objective of this pilot study is to evaluate the feasibility and validity of an AI-generated health podcast as an educational tool for the general public. Specific objectives include:

• Questionnaire Development: Develop and validate a structured questionnaire-adapted from the Questionnaire for Assessing Educational Podcasts (QAEP)-that captures public perceptions regarding podcast accessibility, design, content adequacy, and educational value.
• Reliability and Validity Testing: Assess the reliability and content validity of the newly developed questionnaire through expert evaluation and statistical analysis.
• Public Evaluation of Podcast Quality: Assess the general public's perceptions of the AI-generated health podcast, focusing on its accessibility, design, content quality, and overall educational value, using the validated questionnaire.

Study Design and Methods

This study is designed as a prospective, observational feasibility pilot. It consists of two phases:

Phase 1: Instrument Development and Expert Validation A questionnaire was developed using QAEP as a benchmark and then refined based on feedback from 10 experts (including dental, public health, and communication specialists). Statistical analyses (Cronbach's Alpha, Content Validity Index, and Exploratory Factor Analysis) were conducted to ensure the tool's reliability and validity.

Phase 2: Public Pilot Evaluation The validated questionnaire was administered via an online Google Form to 30 general public participants. Demographic data-including age, gender, educational background, and English audio comprehension-were collected to contextualize the findings.

Expert Group

Inclusion criteria

• Licensed professionals and recognized experts in relevant fields such as dentistry, public health, and health communication
• Minimum of 5 years of professional experience in their respective fields
• Prior involvement in healthcare education, research, or digital health communication
• Willingness to review and provide detailed feedback on the adapted questionnaire
• Ability to complete the survey in English

Exclusion criteria

• Professionals without formal training or relevant expertise in the specified fields
• Individuals with conflicts of interest that might compromise the objectivity of their evaluations
• Experts who are unable to commit the necessary time to provide thorough feedback

General Public Group

Inclusion criteria

• Adults aged 18 years or older
• Individuals fluent in English
• Access to a digital audio device and an internet connection to listen to the podcast and complete the survey
• Consent to participate in the study

Exclusion criteria

• Individuals with significant hearing impairments that might hinder the ability to comprehend the audio content
• Healthcare professionals or individuals with advanced training in dentistry or health communication
• Persons who have participated in similar educational podcast studies previously