JOURNAL OF APPLIED BEHAVIOR ANALYSIS

2013, 46, 296300

NUMBER

1 (SPRING 2013)

TRAINING RESIDENTIAL STAFF TO CONDUCT TRIAL-BASED FUNCTIONAL ANALYSES JOSEPH M. LAMBERT, SARAH E. BLOOM,
AND

S. SHANUN KUNNAVATANA

UTAH STATE UNIVERSITY

SHAWNEE D. COLLINS
CHRYSALIS

AND

CASEY J. CLAY
UTAH STATE UNIVERSITY

We taught 6 supervisors of a residential service provider for adults with developmental disabilities to train 9 house managers to conduct trial-based functional analyses. Effects of the training were evaluated with a nonconcurrent multiple baseline. Results suggest that house managers can be trained to conduct trial-based functional analyses with a high degree of procedural fidelity. Key words: functional analysis, staff training, treatment integrity

Adults with intellectual or developmental disabilities served in staffed residential settings are more likely to engage in problem behavior than those who live independently or with family members (Lowe et al., 2007). When addressing problem behavior, it is standard to conduct a functional analysis (FA; Iwata, Dorsey, Slifer, Bauman, & Richman, 1982/1994) prior to developing an intervention (Mace, 1994). Previous research has shown that people with various backgrounds and levels of experience can be taught to conduct FAs (e.g., Iwata et al., 2000; Phillips & Mudford, 2008; Wallace, Doney, Mintz-Resudek, & Tarbox, 2004). However, given that residential service providers are often underfunded (Reinke, 2009), it is possible that some of these organizations do not have the
All training materials are available from the second author. We thank Hayley Halverson for her assistance conducting this study. Address correspondence to Sarah E. Bloom, Department of Special Education and Rehabilitation, Utah State University, 2865 Old Main Hill, Logan, Utah 84332 (e-mail: sarah.bloom@usu.edu). doi: 10.1002/jaba.17

resources to conduct FAs of problem behavior regularly. One derivation of the FA that may be a viable alternative in residential settings is the trial-based FA (Bloom, Iwata, Fritz, Roscoe, & Carreau, 2011; LaRue et al., 2010; Sigafoos & Saggers, 1995). In a trial-based FA, people who already interact with the subject (e.g., direct-support staff) can conduct brief experimental trials throughout the day during normally scheduled activities in the subjects natural environment. Despite not having access to controlled environments, LaRue et al. (2010) and Bloom et al. (2011) demonstrated 60% to 80% correspondence between the outcomes of trial-based and traditional FAs (depending on how trials are conducted and how data are analyzed). Having direct-support staff conduct trial-based FAs during regularly scheduled activities might decrease the demand on human resources sometimes required for FAs. Doing so also increases the number of people in an organization who are capable of assessing problem behavior, allowing behavior specialists to allocate time to more challenging cases without neglecting the needs of clients, family, and staff members who encounter

296

TRAINING STAFF TO CONDUCT FAS problem behavior on a regular basis. In addition, allowing staff to conduct trial-based FAs may provide them with opportunities to experience how their own behavior contributes to the maintenance of problem behavior. This experience may generate support when staff members are then asked to implement function-based interventions. In several prior studies, teachers implemented trialbased FAs successfully during regularly scheduled classroom activities (e.g., Bloom, Lambert, Dayton, & Samaha, 2013; Lambert, Bloom, & Irvin, 2012; Sigafoos & Meikle, 1996). The purpose of the current study was to evaluate a train-the-trainers program designed to teach house managers to conduct and collect data from trial-based FAs. METHOD Subjects Behavioral supervisors. Six supervisors who worked for a community residential service provider for individuals with intellectual or developmental disabilities were taught to conduct trial-based FAs and then were taught to train others to conduct trial-based FAs. Each supervisor had a masters degree in social work or counseling. Two supervisors were Board Certified Behavior Analysts (BCBA). The other four supervisors had limited exposure to applied behavior analysis. Prior to participating in this study, each supervisor had demonstrated 100% procedural fidelity, graphing accuracy, and data interpretation after training on trial-based FA methodology. House managers. Nine house managers participated. Managers were selected based on the following criteria: (a) They had been employed by the residential service provider for at least 1 month, (b) they worked with a client who engaged in problem behavior and who might benefit from an FA, (c) they had a high school diploma or a GED, and (d) they had no formal academic training in behavior analysis or any previous exposure to any form of FA. Each

297

manager conducted trial-based FA trials that targeted problem behavior of a client who resided in the home that they managed. All had previously been trained on how to protect themselves, others, and property from dangerous and destructive behavior. Clients. Nine adults who had been diagnosed with a developmental disability participated. Clients communicative repertoires ranged from nonverbal to highly verbal. All clients were ambulatory. They were selected because their problem behavior was clinically relevant, occurred at high rates, required an intervention, and was not severe (i.e., it did not pose a serious risk to self, others, or property). Examples of problem behavior included swearing and screaming and mild forms of self-injurious behavior and property destruction. Target Response House manager fidelity to trial-based FA procedures was our dependent variable. Fidelity data were recorded in response to questions outlined in checklists specific to each trial-based FA condition. All questions could be answered yes or no (e.g., Did the house manager ignore all client behavior during this segment?). Data collectors scored (a) whether the putative reinforcer was present during control segments of each trial and absent during test segments, (b) whether house managers provided the correct consequences for behavior, and (c) whether appropriate materials were present and inappropriate materials were absent, and (d) whether house managers collected data during each trial. House manager fidelity was determined by dividing the number of yes scores by the number of yes and no scores for each trial and converting the result to a percentage. Training Fidelity Supervisors conducted training sessions according to a rubric. We assessed fidelity to this rubric. During each session, data collectors answered yes or no to questions about whether supervisors completed the tasks outlined in their rubric. If a

298

JOSEPH M. LAMBERT et al. manager to collect data on the clients problem behavior while conducting each of the trial types outlined by Bloom et al. The research assistant did not provide feedback. House Manager Training Supervisors met with house managers in supervisors offices to conduct the training sessions. First, supervisors provided house managers with 15 min to review brief written descriptions of each trial-based FA condition. Next, they described and modeled each of the trial segments in each condition of the trial-based FA with house managers acting as clients during role play. Supervisors then used a completed trialbased FA data-collection sheet to describe and model data-collection procedures. Finally, supervisors played the roles of clients and house managers played the roles of therapists during role-play scenarios in which clients engaged in both targeted and nontargeted problem behavior at prescribed times according to scripts. Therapists conducted trial-based FA trials and collected data on target behavior. If house managers made errors (e.g., responded to nontargeted problem behavior, failed to collect data), the supervisor waited until the house manager completed the trial and then provided feedback by stating the error, describing and modeling the correct procedure, and providing the house manager with another opportunity to conduct the trial. Role-plays continued until house managers conducted all trial types with 100% fidelity. Posttraining We conducted posttraining observations in an identical manner to baseline, with the following exceptions: Research assistants did not ask house managers to read Bloom et al. (2011) and did provide feedback (in the manner described in Phase 3 of training) after trials conducted with less than perfect fidelity. Note that because feedback was provided, only the first trial of each type conducted in this condition reflects the effect of house manager training alone.

supervisor did not complete a task, research assistants recorded a no on the fidelity checklist and then prompted the supervisor to complete it. Training fidelity was evaluated by dividing the number of yes scores on the rubric by the total number of yes and no scores and converting the result to a percentage. Four of the six supervisors conducted their training sessions with 100% fidelity. The mean fidelity score across all nine house manager training sessions was 97% (range, 85% to 100%). Reliability A second observer independently collected data on house manager fidelity to trial-based FA procedures and on supervisor fidelity to the training rubric during baseline and posttraining conditions. We calculated reliability by dividing the number of agreements (between primary and reliability data collectors) by the number of agreements plus disagreements and converting the result to a percentage. The mean reliability score for house manager fidelity to the ignore condition was 97% (range, 83% to 100%) and was calculated during 42% of all ignore trials. The mean reliability score for the attention condition was 95% (range, 71% to 100%) and was calculated during 47% of all attention trials. The mean reliability score for the tangible condition was 97% (range, 86% to 100%) and was calculated during 38% of all tangible trials. The mean reliability score in the escape condition was 97% (range, 75% to 100%) and was calculated during 44% of all escape trials. The mean reliability score for supervisor fidelity to the training rubric was 96% (range, 85% to 100%) and was calculated during 33% of all house manager training sessions. Baseline Prior to house manager training, we asked house managers to read Bloom et al. (2011) at least 24 hr before the scheduled assessment. A research assistant then visited a target clients house or day program and asked the house

TRAINING STAFF TO CONDUCT FAS RESULTS AND DISCUSSION Figure 1 shows the fidelity of the house managers. During baseline, fidelity was below 50% in all conditions. After training, subjects performance improved across all conditions. This improvement occurred during the first trial of each condition (i.e., the trial that occurred before any feedback from the research assistant). All but one house manager (Manager 4) required

299

BL
100 75 50 25 0 100 75 50 25 0 100 75 50 25 0 100 75 50 25 0 100 75 50 25 0 100 75 50 25 0 100 75 50 25 0 100 75 50 25 0 100 75 50 25 0 5 10 Ignore Escape * * *

Post-Training
Supervisor 1 (Non-BCBA) HM #1 * * Supervisor 2 (BCBA) HM #2 * Supervisor 3 (Non-BCBA) HM #3 Supervisor 4 (non-BCBA) Tangible HM #4 * * Supervisor 4 (non-BCBA) HM #5 Supervisor 2 (BCBA) HM #6 Attention * * Supervisor 3 (Non-BCBA) HM #7 * * * * Supervisor 5 (Non-BCBA) HM #8 * * Supervisor 6 (BCBA) HM #9 15 20 25 30

TRIALS
Figure 1. Percentage of house managers actions that were correct during baseline and posttraining. Asterisks indicate trials that were followed by feedback.

feedback in at least one condition during posttraining trials to achieve 100% fidelity. These results demonstrate the effectiveness of our program for teaching behavioral supervisors to train house managers to conduct trial-based FA trials with high fidelity. It should be noted that all the participating house managers previously had been trained to protect themselves, others, and property from problem behavior and that institutional safety protocols were in place throughout the study. Given the potential danger posed by problem behavior, it may not be appropriate to train all entry-level staff to conduct trial-based FAs, or to ask trained staff to conduct trial-based FAs on dangerous problem behavior. Staff-conducted trial-based FAs may be most appropriate when staff members are adequately trained and problem behavior is socially important but not dangerous. Also, we recommend ongoing supervision by a professional with extensive training in applied behavior analysis. A few limitations should be noted. First, we provided feedback to house managers during some posttraining trials, thus obscuring the effect that training alone had on performance. Therefore, we suggest considering only the first posttraining trial of each condition to determine the effect of training alone on fidelity. Of note, an increase in performance was observed during each of the first observations conducted. Similarly, it is possible that feedback provided in one condition may have affected subsequent performance in other conditions. However, each condition called for different therapist behavior, so feedback on performance in one condition was not necessarily relevant to performance in another. Also, problem behavior did not occur in every trial of every condition because fidelity was assessed using actual client behavior. However, we programmed problem behavior to occur in every trial segment of every trial across all conditions during training. Thus, house managers had an opportunity to perform with high fidelity both in role-play scenarios in which the occurrences of problem behavior were controlled but contrived and in

PERCENTAGE CORRECT

300

JOSEPH M. LAMBERT et al.

Behavior Analysis, 33, 181194. doi: 10.1901/ jaba.2000.33-181 Lambert, J. M., Bloom, S. E., & Irvin, J. (2012). Trial-based functional analysis and functional communication training in an early childhood setting. Journal of Applied Behavior Analysis, 45, 579584. LaRue, R. H., Lenard, K., Weiss, M., Bamond, M., Palmieri, M., & Kelley, M. E. (2010). Comparison of traditional and trial-based methodologies for conducting functional analyses. Research in Developmental Disabilities, 31, 480487. doi: 10.1016/j.ridd.2009. 10.020 Lowe, K. K., Allen, D. D., Jones, E. E., Brophy, S. S., Moore, K. K., & James, W. W. (2007). Challenging behaviours: Prevalence and topographies. Journal of Intellectual Disability Research, 51, 625636. doi: 10.1111/j.1365-2788.2006.00948.x Mace, F. C. (1994). The significance and future of functional analysis methodologies. Journal of Applied Behavior Analysis, 27, 385392. doi: 10.1901/jaba. 1994.27-385 Phillips, K. J., & Mudford, O. C. (2008). Functional analysis skills training for residential caregivers. Behavioral Interventions, 23, 112. doi: 10.1002/ bin.252 Reinke, T. (2009). Community living and housing options for adults with disabilities. Exceptional Parent, 39, 33 35. Retrieved from http://www.eparent.com/ Sigafoos, J., & Meikle, B. (1996). Functional communication training for the treatment of multiply determined challenging behavior in two boys with autism. Behavior Modification, 20, 6084. doi: 10.1177/01454455960201003 Sigafoos, J., & Saggers, E. (1995). A discrete-trial approach to the functional analysis of aggressive behaviour in two boys with autism. Journal of Intellectual and Developmental Disability, 20, 287297. doi: 10.1080/ 07263869500035621 Wallace, M. D., Doney, J. K., Mintz-Resudek, C. M., & Tarbox, R. F. (2004). Training educators to implement functional analyses. Journal of Applied Behavior Analysis, 37, 8992. doi: 10.1901/jaba.2004.37-89 Received June 12, 2012 Final acceptance November 15, 2012 Action Editor, Joel Ringdahl

applied settings in which the occurrence of problem behavior was less predictable but real. Finally, we used a nonconcurrent multiple baseline and decided on a brief (single data point per condition) baseline length a priori for our short baseline. Although nonconcurrent multiple baselines have been used in previous research, it is unfortunate that we did not include longer baselines. Future researchers could include a component analysis to determine whether training was necessary or whether providing in situ feedback alone would have been sufficient. In addition, future research could evaluate this training protocol by assessing fidelity only for trials in which problem behavior was observed or by demonstrating that training leads to effective interventions in group homes. REFERENCES
Bloom, S. E., Iwata, B. A., Fritz, J. N., Roscoe, E. M., & Carreau, A. B. (2011). Classroom application of a trial-based functional analysis. Journal of Applied Behavior Analysis, 44, 1931. doi: 10.1901/jaba. 2011.44-19 Bloom, S. E., Lambert, J. M., Dayton, E., & Samaha, A. L. (2013). Teacher-conducted trial-based functional analysis as the basis for intervention. Journal of Applied Behavior Analysis, 46, 208218. Iwata, B. A., Dorsey, M. F., Slifer, K. J., Bauman, K. E., & Richman, G. S. (1994). Toward a functional analysis of self-injury. Journal of Applied Behavior Analysis, 27, 197209. doi: 10.1901/jaba.1994.27-197 (Reprinted from Analysis and Intervention in Developmental Disabilities, 2, 320, 1982) Iwata, B. A., Wallace, M. D., Kahng, S., Lindberg, J. S., Roscoe, E. M., Conners, J., Worsdell, A. S. (2000). Skill acquisition in the implementation of functional analysis methodology. Journal of Applied