Interrupted time series designs in health technology assessment: Lessons from two systematic reviews of behavior change strategies

Objectives: In an interrupted time series (ITS) design, data are collected at multiple instances over timebefore and after an intervention to detect whether the intervention has an effect signi?cantly greaterthan the underlying secular trend. We critically reviewed the methodological quality of ITS designsusing studies included in two systematic reviews (a review of mass media interventions and a review of guideline dissemination and implementation strategies).

Methods: Quality criteria were developed, and data were abstracted from each study. If the primarystudy analyzed the ITS design inappropriately, we reanalyzed the results by using time series regression.Results: Twenty mass media studies and thirty-eight guideline studies were included. A total of66% of ITS studies did not rule out the threat that another event could have occurred at the pointof intervention. Thirty-three studies were reanalyzed, of which eight had signi?cant preinterventiontrends. All of the studies were considered “effective” in the original report, but approximately half of thereanalyzed studies showed no statistically signi?cant differences.

Conclusions: We demonstrated that ITS designs are often analyzed inappropriately, underpowered,and poorly reported in implementation research. We have illustrated a framework for appraising ITS de-signs, and more widespread adoption of this framework would strengthen reviews that use ITS designs.