Testing a Measurement Feedback App to Improve Data Quality, Supervision & Outcomes in Behavioral Health

A behavioral screening tool used to assess psychological attributes in children and adolescents, leading to a total score of 0 to 40. A higher total score indicates greater difficulties in emotional and behavioral functioning, while lower scores suggest fewer difficulties.

an aggregate of the numbers of intervals scheduled, and the numbers of those intervals in which data was entered (% real time data)

for intervals in which data was entered, the time lag between occurrence and data entry

intervals classified as complete or incomplete according to whether the aide collected their planned metrics as required by their agency (% goals targeted)

Top problems identified during intake meeting. Severity of each problem is rated from 0 (not a problem) to 4 (a very big problem). Severity ratings can be tracked for each problem individually or as an overall mean severity score created by averaging ratings from the same informant at the same measurement occasion

The investigators adapted scales to measure mechanisms affecting data collection, including intentions, using social psychology methods (Armitage & Connor, 2001). The intentions measure had one item, which was rated from 1=strongly disagree to 7=strongly agree. Strongly agree meant stronger intentions to take data. Aides completed measures at baseline and post-trial (3 weeks). Data reported are changes over time.

The investigators adapted scales to measure mechanisms affecting data collection, including attitudes, using social psychology methods (Armitage & Connor, 2001). The attitudes measure had one item, which was rated from 1=good to 7=bad (meaning that taking data would be bad). Attitudes had 2 items. These items were averaged to create the scale score which could range from 1 to 7 for each measure. Aides completed measures at baseline and post-trial (3 weeks). Data reported are changes over time.

The investigators adapted scales to measure mechanisms affecting data collection, including perceived norms, using social psychology methods (Armitage & Connor, 2001). The perceived norms measure had one item, which was rated from 1=strongly disagree to 7=strongly agree. Strongly agree meant stronger influence of norms on taking data. Aides completed measures at baseline and post-trial (3 weeks). Data reported are changes over time.

The investigators adapted scales to measure mechanisms affecting data collection, including descriptive norms, using social psychology methods (Armitage & Connor, 2001). The perceived norms measure had one item, which was rated from 1=strongly disagree to 7=strongly agree. Strongly agree meant stronger influence of norms on taking data. Aides completed measures at baseline and post-trial (3 weeks). Data reported are changes over time.

The investigators adapted scales to measure mechanisms affecting data collection, including self-efficacy, using social psychology methods (Armitage & Connor, 2001). The self-efficacy measure had one item, which was rated from 1=strongly disagree to 7=strongly agree. Strongly agree meant stronger self-efficacy about taking data. Self-Efficacy had 2 items. These items were averaged to create the scale score which could range from 1 to 7 for each measure. Aides completed measures at baseline and post-trial (3 weeks). Data reported are changes over time.

The investigators used the System Usability Scale (SUS; Brooke, 1996) to measure aides' reactions to various statements regarding the app's usability with 10 items that use a 5-point scale from (1) strongly disagree to (5) strongly agree. The SUS has high internal consistent reliability (Cronbach's alpha = .91) and demonstrated sensitivity to change (Lewis, 2018). The investigators adapted the original questionnaire to replace "system" with "app" across all items. The SUS is calculated as a proportion score out of 100, with higher scores indicating higher usability (≥85 = Excellent; ≥71 = Good; ≥51 = Okay (Bangor et al.).

The investigators will compare the groups on combined frequency and content of the communications by using a three-level ordinal response for each period (0=no communication, 1=communication, with discussion of client, 2=communication, discussion of client, and discussion of treatment strategy changes).