The Annenberg Institute posted an RCT of Choose to Change, a group therapy and mentoring program for at-risk youth. Quick take: Despite study claims of effectiveness, it found no discernible impact on the primary outcome: average number of youth arrests over 3 years (the claimed effects on other outcomes are only suggestive).
Program and Study Design:
Choose to Change (C2C) is a 6-month behavioral health program of intensive mentoring and group therapy for at-risk Chicago youth age 13-18.
The study randomized 2,074 youth to C2C vs control, and was well-conducted based on our careful review (e.g., low attrition, baseline balance).
Findings:
The study found no discernible impact on the primary prespecified outcome - average number of youth arrests - measured over 3 years. (C2C group averaged 1.091 arrests vs 1.175 for controls; the difference wasn't statistically significant).
On secondary outcomes: 1 of 16 impacts was statistically significant.
On other outcomes: The study found a significant effect on the percent of youth arrested over 3 years (33.4% C2C vs 38.6% control). But this was a post-hoc outcome - not prespecified as primary or secondary - and one of multiple post-hoc outcomes that the study measured (along with crime victimizations, police stops, etc).
Thus, the finding on percent arrested is only suggestive. It could be a true effect, but isn't reliable under established standards (FDA, IES) until confirmed in future studies. Such confirmation is needed to rule out the possibility that it's a chance finding resulting from the study's measurement of many outcomes.
Comments:
Unfortunately, the study abstract (below) does not mention the lack of effect on the study's primary outcome, and makes strong claims of effectiveness based on the suggestive (post-hoc) findings.
