Interrupted Time Series Design: A Useful Approach for Studying Interventions Targeting Participation

. Comparative effectiveness research (CER) strives for evidence to inform clinical decisions for specific clients in typical health care settings. While the randomized controlled trial (RCT) is well-aligned with the objectives of CER, this design may not be feasible or applicable to all research questions, particularly those pertaining to clinical decision making and individually based change. It is important, therefore, to consider alternative approaches, especially when studying complex and subjective outcomes such as children’s participation. We propose the use of interrupted time series (ITS) quasi-experimental design for its potential application in determining the effectiveness of participation-focused interventions. In this perspective, ITS design is described, ascertaining its advantages and limitations, and suggestions are provided to overcome challenges to implementation. Results of a case study using the ITS approach to evaluate changes in participation of an adolescent with physical disabilities are described. Finally, strategies to implement this design in practice are suggested.

Participation, defined as "involvement in life situations" or "engagement in daily activities" (WHO, 2001), has become an important outcome in neurodevelopmental rehabilitation and has gained increased attention in research and clinical communities.Engaging in leisure activities outside of school plays a key role in a child's development and quality of life, particularly when it reflects the child's preferences and interests (Majnemer et al., 2010).Through participation, children acquire skills and competency, achieve physical and mental health, and develop a sense of meaning and purpose in life (Larson & Verma, 1999).
(Received 9 April 2013; accepted 5 November 2013) Research indicates that children with neurodevelopmental disorders experience significant restrictions to their participation in comparison to their typically developing peers (Engel-Yeger et al., 2009;King et al., 2010;Law et al., 2011), suggesting an important area of interest for targeting interventions.Nevertheless, most research has focused on identifying the factors that contribute to participation and limited systematic research exists on how participation can be improved.
With accumulating knowledge it is clear that participation is a complex concept.It is influenced by a multitude of factors related to the child (e.g., abilities, preferences), family (e.g., family functioning, values) and environment (e.g., lack of accessibility to buildings; availability of programs and services) (King et al., 2006).There is also evidence illustrating how participation changes over time.For example, in a longitudinal study of 136 children and youth following an acquired brain injury, Anaby and colleagues (2012) found that participation rates changed over a period of one year.Changes were related to severity of injury, family income and family functioning.Another study that tracked participation changes of 402 children and youth with physical disabilities over the course of 3 years found that factors related to the child (e.g., communicative functioning, preferences), family (e.g., family cohesion), and environment (e.g., attitudes, policies) predicted the rate of change of participation (King et al., 2009).These predictors depended on the type of activity and varied as a function of the child's age and gender (King et al., 2009).
There is much debate in the literature about what is considered 'optimal' or 'adequate' participation, due to its variance across individuals.As such, no norms exist regarding participation as a concept and some argue that in this case, norms simply may not be appropriate."Optimal" participation, therefore, is a subjectively determined construct led by one's own criteria and expectations (Rochette et al., 2006).
Given the complexity of participation and interventions to address it, selecting the appropriate research design is critical.Interventions may focus on one or more of the aforementioned factors (e.g., child, family, environment), yet, in the case of participation, a dynamic interaction exists across factors (Palisano et al., 2012) and, hence, accounting for other factors becomes important.Protocols should therefore be flexible enough to address various conditions and real-life situations that pertain to participation.It is important to choose a design that allows for rigorous testing of an intervention in a manner that respects the individuality and context of each client (i.e., accounting for variability and change within participants) and, thus, is highly applicable to practice.
The purpose of this perspective is to discuss the use of the interrupted time series (ITS) quasi-experimental design with multiple baselines and replications for its potential application in testing the effectiveness of participation-focused interventions.We begin with an overview of current approaches for obtaining clinical evidence.The ITS approach is then presented, its advantages and limitations are appraised, and suggestions are made for managing challenges, using an illustrative example.

Towards Pragmatic Clinical Trials
The Institute of Medicine in the United States has acknowledged the need for more pragmatic clinical trials that can provide applicable clinical evidence and have suggested the application of the Comparative Effectiveness Research (CER) approach (Iglehart, 2009).This approach reflects the real-world clinical setting and accounts for the variability of participants and environmental characteristics that occur in frontline practice.The CER approach asks whether a treatment works, for whom it works, and under what conditions it works, rather than asking if, under optimal conditions, a treatment "can work".It captures in-depth, comprehensive information about participants' characteristics, processes of care, outcomes and the contribution of individual processes on outcomes.Such an approach encourages the study design to reflect routine and day-to-day experiences of clinicians and hence can assist clinicians in making informed decisions (Horn & Gassaway, 2007).It is important, therefore, to identify rigorous methods and designs that meet CER objectives.
Randomized controlled trials (RCTs) fall on a continuum of efficacy (explanatory) to effectiveness (pragmatic), the latter being well aligned with comparative effectiveness research.RCTs are considered the gold standard in clinical research, particularly for evaluating the effectiveness of interventions.Rehabilitation researchers have applied this design when studying outcomes such as gross motor activities (Salem & Godwin, 2009) and performance of functional tasks related to mobility and self-care (Law, Darrah, et al., 2011) among children with cerebral palsy.RCTs are also associated with several challenges, such as the cost of selection and randomization of participants, as well as feasibility related to recruitment of sample size.In addition, within traditional RCTs there is also an issue regarding the extent to which findings are applicable to the clinical setting.This is due in part to strict inclusion and exclusion criteria during recruitment as well as to tightly controlled intervention protocols that may be difficult to replicate and deliver in the context of routine care (Silverman, 2009).While pragmatic RCT designs do propose solutions to address many of the aforementioned criticisms increasingly associated with efficacy studies (Luce et al., 2009;Patsopoulos, 2011), they remain limited in their ability to account for intra-individual change.This is especially relevant when studying the complex and subjective outcome of participation and targeting heterogeneous populations, i.e., children with physical disabilities (Østensjø et al., 2003).
In light of patient-oriented research (Sacrist án, 2013) that provides specific quantifiable answers for individual cases, it is important to account for individual differences in outcomes rather than simply reporting the effect on the "average individual."Designs that account for within-subject change, including the history of change and the underlying secular trend before the intervention is introduced are therefore needed.The ITS with repeated replications within and across subjects, is one such design (Shadish et al., 2002).

What is Interrupted Time Series (ITS) Design?
ITS is a quasi-experimental design, also known as a Repeated Outcome design, where data is collected at multiple instances before, during, and after the intervention, in order to established and compare trends in outcomes.ITS can be a particularly powerful design when it involves multiple baselines (Shadish et al., 2002).As such, the effect of the intervention is tested in a number of different contexts (e.g., individual-based behaviors/goals) using separate baselines.The timing in which the Generalizability of the effect intervention is introduced is manipulated ("interrupted"), and subjects act as their own controls.
In pediatric rehabilitation interventions, the multiple baseline approach has been successfully used in the context of single-subject design.For example, Lammi and Law (2003) examined the effect of Family Centred Functional Therapy on occupational performance across three tasks in the home setting (e.g., donning pants, self-feeding) of three young children with cerebral palsy.Interventions were introduced for each task in a staged manner and the child's performance was monitored daily using the Canadian Occupational Performance Measure (COPM) scale before, during and after intervention, over a course of 46-61 days.Results indicated a positive and significant change in performance for each child in most tasks.
The proposed ITS design relies on the principle of single subject design, especially because the individual serves as the unit of analysis.This design, however, is comprised of additional elements; for example, it involves a greater number of observations (i.e., data points) over time as well as a greater number of participants in order to systematically replicate the effect.Such a design employs analytical approaches that combine individually based data and consequently can identify sub-groups most affected by the intervention-an advantage that is not inherent in simple single-subject designs.Table 1 describes the key elements of the proposed design.

How is the ITS Design Implemented?
This design has been used extensively in epidemiology to monitor medication use within states (Wagner et al., 2002), policy change across communities (Gilligan et al., 2010), and within the field of health care for evaluating implementation of practice guidelines in different clinical settings (Ramsay et al., 2003).This design, therefore, can be applied across different units of analysis and we propose to use it for testing change in different participation goals in which the effect of the intervention is  The clinician and the client set at least 3 participation goals and the performance of each goal is monitored across all phases of the study.A minimum of 3 goals is needed to allow for testing the replication of the intervention effect in different situations (Logan et al., 2008).An intervention is introduced for each goal and a delay of a certain period of time (e.g., 4 weeks) is set between each intervention.As illustrated in Figure 1, three participation goals are monitored throughout the entire study using the performance scale of the COPM (Law et al., 2005).If an observed change in performance of goal 1 follows an intervention that is coupled with an absence of change of goal 2 and 3, both of which have not yet been addressed, then we can conclude with high level of confidence that the intervention was the cause of the change.This conclusion is strengthened if the effect is replicated across individuals.
While there is no rule of thumb indicating the required sample size, Jones and Nesselroade (1990) argue that 30 replications are considered powerful and result, in our proposed design, in 90 trajectories (i.e., 3 goals x 30 replications).Systematic replication of the effect across randomly selected participants is valuable as it can help us identify the profiles of youth for whom the treatment is effective or less effective, as well as the conditions under which the effect is observed or partially observed.Guidelines for detecting similarities and differences in patterns of change across participants have been developed by Hawkins et al. (2007) and continue to be explored by others.Specific analytical approaches to evaluate change across participants are described in detail in the following section.

Data Analysis and Interpretation
Testing the effect of the intervention and the replication of the effect in different conditions (i.e., across three participation goals) further allows for internally valid conclusions to be drawn.There are ways to determine if a positive effect was observed at the level of the individual as well as the magnitude of this effect.Examining change that is both statistically and clinically significant is essential.When considering the COPM, a measure that can capture change in performance of participation goals, an increase of at least 2-points in performance/satisfaction scales is considered clinically significance (Law et al., 2005).Determination of statistical significance involves a combination of visual analysis and statistical approaches.Visual analysis involves the following procedure; for each participation goal, a series of data points that represent level of goal performance are plotted and analyzed to detect change.This visual inspection of data allows for comparison of patterns of change (e.g., level, variability and trend) between baseline and intervention data.Visual analysis of graphical data allows for the timing of change to be examined; specifically, whether change in performance occurred immediately after the intervention began, more gradually, or did not occur at all.Statistical approaches include the Celeration line and C-statistic; both account for serial dependency or autocorrelation across responses (Ottenbacher, 1986).The Celeration line is a method that calculates the proportion of data points falling above and below the line.No treatment effect is indicated when 50% of observations are above the line and 50% below the line, whereas significantly more observations falling above the line indicates a change in goal performance.This proportion is then compared to a probability table to determine if change is statistically significant.A minimum of 24 data points is needed in order to apply statistical tests.Accounting for serial dependency across data points where sequential responses are highly correlated is also important.To date, statistical tests such as Bartlett's test (Huitema & McKean, 1991) and data transformation techniques (Ottenbacher, 1986) have been developed to effectively detect and treat such issues and, hence, avoid overestimation of treatment effect.
Testing the replication of the effect in a variety of circumstances or participation goals increases confidence that the treatment caused the outcome; moreover, it is possible to set an additional participation goal where the intervention is not targeted, yet performance is monitored throughout the entire study.Such manipulation can test the generalization of learned skills and strategies with a different participation goal, resulting in stronger evidence.
Testing intervention effect across participants is guided by a systematic replication approach.This approach relies on a qualitative review of patterns of change across individuals, rather than on formal statistical inference (Velicer & Fava, 2003).For example, two independent experts can detect trends within each participant following a set of guidelines and can then cluster participants based on aspects of change that reflect the extent of the intervention effect (e.g., number of significant replicated effects across goals).Once clusters are formed, group members can be profiled to represent specific sub-groups to which the intervention is most or least effective.Formal analytical strategies to combine treatment effects from aggregated single individual data are suggested in the literature.These include the pooled time series analysis (Velicer & Fava, 2003) and meta-analysis of single-system designs (Shadish & Rindskopf, 2007), where the former has been successfully used among children with autism (Hoeppner et al., 2007).Another technique that focuses on analyzing intra-individual variability over time is dynamic factor analysis which can detect rates of change in factor structure of targeted outcome over time (Nesselroade & Ram, 2004).An examination of these intra-individual structures can then be performed to allow generalizability of individual effects.Such analysis; however, can be employed in specific conditions; for example, related to number of observations as well as the use of outcome measures that encompass various items to allow for factor analysis.

Advantages of the ITS Design
ITS design has several advantages; it is highly applicable to practice and easy to replicate in the clinical setting.ITS, in many cases, mirrors typical rehabilitation practice in which more than one goal is attended to and the clinician often solves a series of problems sequentially rather than at one point in time.Thus, findings based on such a design are directly applicable to the clinical setting and to clinicians' day-to-day questions.In multi-baseline designs, a series of multiple data points are collected through all phases of the study.The data not only allows observation of change but also the nature and timing of occurrence.This type of design can assist clinicians in developing well-planned treatment programs involving an optimal duration of time and resources.Finally, the proposed ITS design builds on a nonconcurrency principle-it does not require starting the data collection with all subjects at the same time, a procedure that reflects real life situations and is easy to replicate in practice.

Determining Cause-and-effect Relationships
Another advantage of ITS design is the ability to determine if the intervention is truly responsible for change in participation; it has high internal validity that allows for causal inference.If done properly, ITS can reassure, to a certain degree, that the intervention was the cause of change in the outcome (i.e., participation) rather than change being the result of other factors, known as confounders.
Having a stable baseline is a prerequisite for a well-designed experimental study and ITS takes this principle a step further by varying the length of baseline across participation goals.By doing so, extraneous variables such as maturation and carryover effects are controlled for, which in turn strengthens the internal validity of the study (Ramsay et al., 2003).Hawkins and colleagues (2007) developed guidelines that examine causality in order to increase confidence for casual inferences related to the effectiveness of the intervention.An observation of the timing in which change has occurred is another important indicator that can strengthen causal inferences; observed effects that are temporarily consistent across participation goals (e.g., immediate change occurring following implementation of interventions), further support the decision of causality (Hawkins et al., 2007).
Randomizing the order of the goals is another strategy for increasing the internal validity of the study and hence increases confidence in causal inference (Hawkins et al., 2007).In other words, each participation goal can be randomly assigned to the order of the intervention ensuring that the intervention effect is not due to a co-occurring event.In order to classify the strength of such evidence, Ramsay et al. (2003) developed quality criteria for ITS designs.These criteria evaluate study design in terms of internal validity and aspects of data collection, such as number and spacing of data points, use of appropriate outcome measures and techniques to analyze the effect, among others.

Disadvantages and Ways to Overcome Them
At first glance, like in many individual-based designs, generalizability is one methodological limitation.This design, notably, allows for testing the replication of the intervention effect across three different goals within each participant and across participants, providing a degree of generalizability of results.Replicating the effect across a representative sample can further support generalizability of findings.
The need for multiple data points (a minimum of 24) might be perceived as another challenge.Using a measure that is easy to administer and complete can alleviate this need.For example, the COPM scale has participants rate their performance using a simple scale from 1 (unable to perform) to 10 (performs extremely well).Another potential issue may be relatedness or dependency between the selected participation goals.The more interrelated the goals, the more likely they are to be influenced by the interventions, regardless of the timing in which they were introduced; such an issue weakens the internal validity of the study.Thus, it is important to ensure that goals are independent of each other and involve different circumstances/contexts, as well as to ensure that baseline data are stable before each intervention is introduced.The clinician must carefully attend to this issue when setting participation goals with the child/family.Finally, in the case of participation where goals are individualized and involve various types of challenges/barriers, the intervention is guided by principles of a certain approach rather than by an exact protocol.To increase confidence in clinical effectiveness, fidelity (i.e., the extent to which the intervention is delivered as planned) can be assessed and monitored.The Paediatric Rehabilitation Observational Measure of Fidelity (PROF; Di Rezze et al., 2013) can be used to identify intervention principles and attributes.

A Case Example
Mark is a 20-year-old male with a complex physical disability involving cognitive and communication difficulties (cerebral palsy, GMFCS level IV).His family was recruited from a children's rehabilitation centre and a family network in Hamilton, Ontario.His mother was involved in the assessment, goal planning and intervention phases of the study.

Procedure
Two student occupational therapists met with Mark and his mother, and, using the COPM, collaboratively set 3 participation goals that Mark would like to participate in, yet finds difficult.An environment-based intervention was then introduced at different time points, with a 4-week delay between goals, resulting in a 12-week intervention phase.In this intervention, environmental barriers of participation were identified and strategies to remove them were implemented.Over the course of 20 weeks-including baseline, intervention and follow up phases-the performance of each participation goal was repeatedly measured, i.e., twice a week, using the COPM (Law et al., 2005).The COPM is a valid and reliable tool that is sensitive to change (Verkerk et al., 2006).The Children's Assessment of Participation and Enjoyment (CAPE) and Preferences for Activities of Children (PAC), a psychometrically sound assessment (King et al., 2004;King et al., 2007) that measures preferences, diversity, intensity (ranging from 1 to 7) and enjoyment (ranging from 1 to 5) of participation in 55 out-of-school activities, was administered pre-intervention to evaluate participation patterns and to assist in the process of goal setting.A series of data points (32 data points per goal) that represent level of goal performance were plotted and a visual analysis was performed where an increase of at least 2points on the COPM scale indicated a clinically significant change (Law et al., 2005).The celeration line technique was then performed to detect statistically significant change.

Results
Three participation goals were identified by Mark and his mother: (a) attending a Hamilton Tiger Cats football game; (b) participating in an inclusive social dance; and (c) returning to be an active church member.Barriers encountered by the participant included: lack of knowledge and access to information, lack of inclusive programming, lack of parental knowledge on how to successfully advocate for child inclusion, physical inaccessibility of community buildings, and lack of knowledge of accessibility.For example, barriers towards Mark's goal of attending a Tiger Cats game included the institutional barrier of additional cost of attending the game with a support worker and the physical barrier of unknown accessibility, including mom's concerns regarding how Mark will participate with the potential difficulty of  entering and exiting the stadium due to crowds.An intervention was developed to address these barriers.It included contacting the stadium to determine accessible areas and availability of discounted tickets for support staff, creating a plan with Mark's mom to arrive 45 min early to avoid crowds, staying seated until the crowds eased and bringing a noisemaker to cheer in order to increase Mark's participation.Table 2 describes in detail the intervention strategies that were used.Improvements in COPM scores were clinically significant for all three participation goals and statistical analysis using the celeration line indicated a significant treatment effect, i.e., all data points fell above the celeration line (see Figure 2).Notably, change occurred immediately following the intervention, coupled with an absence of change in the goals yet to be addressed.There were clinically significant changes in performance scores for all identified goals, with an average change of 7.67 points from initial assessment to reassessment at follow-up.

Considerations for Implementation
Implementing a ITS design as part of practice involves careful consideration.Relatively stable life circumstances of both the child and family are important; preferably when no major changes (e.g., during acute recovery, illness in the family) are anticipated.Change is monitored at multiple times points, therefore it is also important to choose a measure that can be easily completed (e.g., performance scale of COPM) and easily collected, for instance, via text messages or e-mail.Documenting the intervention by including contextual backgrounds that might effect treatment (e.g., family events) is also warranted as it can further explain change that is effective or less effective.
Collecting ongoing data in the clinical setting can build evidence over time; it can assist in program evaluation, and can inform policies related to practices (e.g., optimal duration of intervention for specific clients, release time, number of sessions and cost effectiveness).Finally, this approach can improve researcher-practitioner collaborations-an important partnership for the development of evidence-based practice as well as for effective, on-going and integrated knowledge translation initiatives.
Our case study example illustrates that this method is applicable and feasible in studying interventions that target the outcome of participation.Further studies using ITS designs are needed, particularly those involving a larger sample that represents families of different background contexts.With existing guidelines (Hawkins et al., 2007) and analytical strategies for examining this design across subjects (Velicer & Fava, 2003), it is possible to conduct a systematic replication of a specific intervention effect aimed at increasing children's participation.This will result in new and rigorous knowledge, pertinent to the clinical setting.

CONCLUSIONS
Individual-based frameworks that involve ITS design are gaining recognition as a sound methodology for clinical evidence (Hawkins et al., 2007;Johnston & Smith, 2010;Velicer & Fava, 2003).ITS design is particularly appropriate for the purpose of detecting individual-based change in complex outcomes using flexible intervention protocols.In our case, the use of an environment-based intervention to increase participation in the child's preferred activities is put forth.Such a design is not only rigorous and allows for cause-and-effect inferences to be drawn but is also highly applicable to the clinical context.It can be easily incorporated into practice without interfering with clinical routine, while providing the therapist with a true indicator of the impact of their day-to-day individualized-based interventions-all this with minimal additional cost.ITS design allows for client variation and can accommodate complex cases, which is beneficial in rehabilitation.As clinicians meet clients over time, they can record session-to-session change in performance.Finally, ITS across goals does not require all participants to begin the baseline phase simultaneously; a more accurate reflection of 'real life' practice.
Figure 1 illustrates an ITS design with one participant and multiple baselines across participation goals or activities.It involves several phases per goal: baseline, intervention and follow-up.

FIGURE 2 .
FIGURE 2. Mark's performance scores (COPM) for Goal 1 (participating in a football game), Goal 2 (taking part in dance classes), and Goal 3 (attending church), respectively.Note: Vertical line indicates start of intervention, horizontal line represents the celeration line.