EBP Exchange - UMB School of Social Work

That’s not the title of this RAND report, but it should be. The report compares model programs to standard programs and finds that on most outcomes there is no difference–12/60 have statistically significant differences (you would expect only 3 out of 60 by chance, alone) and these 12 are split with half favoring the model programs and half favoring the standard programs. The RAND analysis concludes that there are too many intervening factors to be able to intepret the outcomes–and sites differences in case mix and community supports, and the like.  They then proceed to suggest that we should use quality of care indicators.  What they don’t say is why case mix challenges to interpretable findings can’t be managed using methods like propensity score matching but they also don’t indicate how you would find quality of care indicators if you don’t know whether the model programs have any overall benefit.  I can surmise that there are some effective programs in the study and that they might have common (quality) program indicators, but its a long stretch for me to conclude that we have enough science to select quality indicators–in the absence of much evidence from this study, or any other, to indicate what works in teen substance abuse treatment.  Any one else see the logic in their conclusion or a way to get from the starting point of little or no program benefit to determining what indicates a quality service? See the report at:

http://www.rand.org/pubs/research_briefs/2007/RAND_RB9269.pdf