The Efficacy of "Inner Peak" an AI-enabled Digital Wellness App on Mental Wellness Among a Sample of University of West Indies (UWI) Mona Students

Background

Since the COVID-19 pandemic, many countries are facing a new epidemic of loneliness, anxiety, and depression. These rates are more prevalent among the more youthful population with recent research surveying 80,000 youth globally indicating that depressive and anxiety symptoms have doubled during the pandemic, with 25% of youth experiencing depressive symptoms and 20% experiencing anxiety symptoms. These statistics are even more stunning in some countries, as in the United States in 2021, 60% of college students surveyed across 370+ schools met the medical criteria to be diagnosed with a mental health challenge. This has a tremendous impact on student graduation rates, as students with a mental health challenge are three times as likely to drop out than students without mental health issues. Universities are losing top-line revenue as a result of student mental health, making mental health a top university concern.

Jamaica is not immune to these mental health challenges as according to the Jamaican Ministry of Health and Wellness, in 2016, 14% of Jamaicans experienced depression (NCPID). Though there have not been reliable reports conducted since the pandemic, if Jamaica followed the average global trend, upwards of 30-35% of the population would have experienced depression. Jamaica's limited economic resources make meeting physical and mental health needs extremely challenging. The patient to psychiatrist ratio in Jamaica is 100,000:1, and there are even fewer psychologists, with 1 for every 300,000 potential patients. Low mental health literacy further compounds these challenges. Further, various demographic factors such as gender, socio-economic factors, accessibility of resources, among others have been found to impact health-seeking behaviours among Jamaicans.

These barriers have necessitated healthcare professionals to consider innovative interventions that are accessible, effective, and tailored to the unique needs of Jamaicans. The University of the West Indies, one of the leading tertiary institutions in the Caribbean, is known for its contribution to healthcare interventions in Jamaica. The university acts as a microcosm of Jamaican society with indications of mental health challenges such as depression, trauma, anxiety, disordered eating behaviours prior to and more so since the pandemic.

The University's Health Center (UHC) is the designated facility to meet both the mental and physical needs of the university population. The UHC at the University of the West Indies, Mona Campus (UWI - Mona) provides a wide range of primary and secondary health care services to members of the university community, including staff and their dependents, retirees and their spouses, students and at times visiting groups.

Traditional face-to-face psychological intervention comes with its own set of challenges, including but not limited to stigma associated with getting psychological treatment; confidentiality; long waitlist or being seen long after the initial request to be seen; financial challenges in accessing care; or a lack of awareness of the resources available and the reliance self to fix one's problems, which is particularly common among young people.

Jamaica faces these problems even more acutely due to a lack of economic resources and mental health providers. UWI Mona Campus is a microcosm of these issues, as the ratio of mental health providers to students is only 1:3,500. This ratio is in practice even worse, as it does not include members of staff, current and those retired, who are also seen by the University of the West Indies, Health Centre. The International Association of College Counselors recommends that campuses maintain minimum staffing ratios in the range of one full-time professional staff member to every 1,000 students. The current cadre of 4 counselors (including the head of the UHC) is short-staffed, leading to long delays in new and follow-up appointments which can be upwards of 6-8 weeks. To meet international standards, UWI Mona would have to hire 10 more full-time counselors, entirely impractical given budget constraints.

Research advocates for using online interventions (which can include digital mental health apps, such as Headspace and Calm) to supplement traditional counseling to meet client needs. These solutions are helpful because they are available 24/7, less costly, and scale to many students at the same time. Further, online intervention programmes have the option to compile data to monitor, evaluate and improve their programme delivery and efficiency. Studies have also indicated that, when these apps are used regularly, they can be effective for users in improving mindfulness, well-being, stress, depressive symptoms, self-harm, eating disorders, sleep disturbances and anxiety symptoms. However, these solutions struggle to keep users engaged and retain them long enough to have an impact. A systematic review of 93 apps with 10,000 installs or more targeting anxiety, depression, or emotional wellbeing showed the median 15-day and 30-day retention rates were 3.9% and 3.3% respectively. Further, these apps are designed to teach a skill in the abstract, not address a specific challenge a client is facing. This suggests that the apps are not situationally relevant and deliver solutions linearly in the same way for each user, or require the user to have the knowledge to search for a specific piece of content to solve an issue.

To successfully meet student mental health needs, the world requires affordable, scalable, and effective interventions that also promote sustained user engagement, ensuring that students utilize the solution sufficiently to derive meaningful benefits. One way to keep students using something is to create accountability through a working alliance. This working alliance between a therapist and patient is the best predictor of success in the therapeutic relationship. Artificial Intelligence (AI) conversational agents can be used in therapeutic capacities to build a working alliance of similar strength to a human therapist working alliance. While this seems counterintuitive at first, AIs at baseline show unconditional positive regard and are non-judgemental, the core attributes of any good patient therapist alliance. Furthermore, AIs are tireless. In fact, researchers found that AI responses to patient questions on an online forum were preferred over actual medical professionals' answers as they rated significantly higher for empathy and quality. Taken together, this research suggests that AI can play a role in human connection and relationships.

"Accountability partner" behavior has its roots in Alcoholics Anonymous' practice of pairing each person with a sponsor with whom they have regular meetings and who encourages their sobriety. The American Society of Training and Development did a study on accountability and found that participants had a 65% chance of completing a goal if they committed to that goal to someone else. If participants had a specific accountability appointment with a person to whom they have committed, they increased their chance of success by up to 95%.

Since AI can mimic a human relationship, this suggests that AI can potentially be used to drive accountability to goals, including continued usage of digital mental health tools. This suggests that AI conversational agents can be used to help keep users engaged. Many young adults and college students are open to seeking anonymous mental health support from a conversational agent. In a recent survey, 55% of 18-29 year olds indicated that they would be more comfortable speaking with an AI conversational agent than a human therapist.

Relying solely on AI has its drawbacks. Because AI technology moves so quickly, there is little academic research identifying the drawbacks. Instead, investigators must turn to examples in the news. One of the examples that evidences the danger of AI is the tragedy of a 14-year-old boy who used a Character AI chatbot that did not have appropriate guardrails in place. When given the leeway to provide the user advice, the chatbot encouraged the boy to take his own life, which he did. Because AI can hallucinate, it is not good at being an expert itself. However, it is very good at identifying patterns and directing a user to the appropriate expert.

The Inner Peak mental wellness app was designed to bridge the gaps in these solutions. The use of digital apps for mental health and wellness is common. The intention of the Inner Peak app is to complement psychotherapy; it is not a replacement for a therapist to treat mental illness. It was designed, like other digital apps, as a guided self-help tool. These other apps are shown to offer scalable, easy-to-access, timely solutions to improve quality of self-care and self-management. It starts from the same core of expert-created digital content as the other digital interventions, but wraps this content in an engaging, personalized, 24/7 accountability partner in the AI coach. AI coaches and therapists have been shown to drive greater engagement over a longer period than using solely videos. A combination of AI coaches with digital content has never been tested academically. The investigators hypothesize that restricting the role of the coaches to active listeners and content curators, as in the Inner Peak app, can prevent some of the dangers of using AI while still realizing the increased engagement, personalization, and accessibility benefits of AI. The reference made to a "guardrailed" AI conversational agent in this study means: using AI only as an active listener as opposed to an advisor or expert. This study will assess whether an AI conversational agent can still achieve desired mental health outcomes if it is only allowed to be an active listener and curator of content.

Justification:

Prior research has tested and demonstrated the effectiveness of using digital therapist-created content in mobile and web applications, but has highlighted that the biggest challenge of this method is encouraging continued usage of the resources. User attrition greatly reduces the effectiveness of these solutions. In order to improve effectiveness and reduce attrition, apps that involve a relational element (generally human, but sometimes AI) drive greater usage. Given that researchers are unable to use humans because of limited resources, an app that uses AI in this context can address this challenge. AI on its own however can be harmful if it makes a recommendation to a user that is incorrect. An app which has combined AI with content already created by expert therapists is deemed to be the best solution for driving higher engagement and effectiveness.

Research has also indicated that when people engage with human therapists or AI therapists, they are much more likely to continue engagement over longer periods of time allowing greater potential for impact. While it may be preferential to use humans over AI, AI becomes a more viable interim alternative especially in lower-income countries or places with lower concentrations of therapists and lack of resources to meet the mental health needs of its population. An observational study of 10 apps that offer support and treatment for a variety of mental health concerns with a built-in chatbot feature concluded that "chatbots have great potential to offer social and psychological support in situations where real-world human interaction is not preferred or possible to achieve. To build on this finding, the investigators propose an experimental design to test the actual effectiveness of the Inner Peak chatbot beyond just user reviews."

Because AI technology is advancing so quickly, there is limited research testing different ways of applying it. In this study, the investigators will test whether AI that is limited in its role to solely being an active listener and content curator can still drive better continued usage and outcomes while reducing risk of AI providing inaccurate advice that might be dangerous to users.

Given the lack of resources, along with the long wait period to gain support from a therapist, AI may provide some support for college students in this low and middle income country. Additionally, the investigators would like to ensure that the type of app used is one that is deemed to address any previous limitations found with other app. This study therefore seeks to bridge this gap by examining this newly generated AI mental wellness support app among a sample of university students, their use of it and how they engage with it. Findings from this study may provide useful insight into other forms of support, especially among populations that have been unable or unwilling to receive mental health support from mental health professionals.

Hypothesis:

Using a guardrailed AI conversational agent that curates therapist-created video content in a mental wellness app will improve mental health attitudes and outcomes in UWI students.

Objectives:

i. To describe the mental wellness challenges students are experiencing (and the frequency of these challenges) that prompt them to utilize a new guardrailed AI-powered mental wellness resource.

ii. To measure whether students enjoyed using guardrailed AI for their mental health and felt more emotionally supported and safer by having access to the app.

iii. To assess whether giving students access to a guardrailed AI-powered mental wellness resource can increase student understanding of their mental health and confidence in their ability to improve their own mental health on their own or with AI-powered tools.

iv. To determine if using guardrailed AI to support student mental health can improve wellbeing outcomes as measured by self-reported clinically validated scales.

Inclusion criteria

• University of West Indies students aged 18+ who have signed our consent form.
• Have access to an Android or Apple Device
• Have a UWI email account

Exclusion criteria

• Participants will be excluded if they

• Are below the age of 18
• Have not given written consent to participate
• Do not attend UWI at the time of consenting to participate