About Me. My Background Behavior Analyst since 1991 Children s Seashore House FBA in Hospital Settings 6 month admissions Joe Lalli University of Iowa Hospitals & Clinics FBA in Clinic Settings Outpatient Clinic for Low functioning children with behavior problems Outpatient Clinic for High functioning children with behavior and learning problems Inpatient Program modeled after Seashore House Inpatient Pediatric Feeding Disorders Program Dave Wacker & Linda Cooper-Brown
About Me. My Background Behavior Analyst since 1991 Bancroft NeuroHealth FBA in Neuro-Stabilization Unit One of 2 Ph.D. Level Behavior Analysts (I was now the Joe Lalli) MelMark FBA in Residential School & Outpatient Programs Translated this work to residential school settings One of 5 Behavior Analysts for a 12 classroom school program (enrollment of around 60-65 students) Penn State University Teaching teachers how to do FBA In Home Feeding Grant FBA Training at the County Level
About Me. My Role at the State State Consultant 1 st State BCBA Consultant To address the challenging behaviors in the state My role at the state level is to: Provide training and support with issues revolving around challenging behavior. We are not very good at addressing challenging behaviors in the state.
About Me. My Role at the State 10 Year Plan Years 3-5: Train School Consultants whose job was to address challenging behavior FBA technologies (i.e., Train) Years 4-6: Develop BCBA programs in the state (i.e., Sustain) Years 7-10: Develop Day TX and Inpatient programs for most difficult children (i.e., develop Full Continuum of services) Goal of the CBS Project : To eliminate false positives from practicing who were addressing challenging behavior by creating true positives who use FBA s to inform their BIPs
Range of FBAs Functional Behavioral Assessments Indirect Methods Naturalistic Assessments Experimental Analysis Behavioral Checklists Behavioral Interviews ABC Analysis Descriptive Analysis Antecedent Analysis Functional Analysis
Indirect Assessments Behavior of concern is not directly assessed. An informant is used Care-provider behaviors Child Asking them about
Direct Observation Assessments Behavior of concern is directly observed. An informant is not used Care-provider behaviors Child This relationship is directly observed (Naturalistic Assessments)
Experimental Analyses Behavior of concern is directly observed. An informant is not used Care-provider behaviors Child This relationship is directly observed and manipulated
What do we do for Challenging Behavior? In Education, we are required to use: Evidence Based Practices Functional Behavioral Assessments (FBAs) Functional Behavior Assessments are mandated for children w/ disabilities whose behavior is determined to be a manifestation of their disability. Thus, if a child engages in challenging behavior in the school, LEAs are responsible for conducting an appropriate FBA to develop an effective Behavior Intervention Plan (BIP) that can reduce the future occurrences of the child s specific challenging behaviors. School must do this
The Law that Specifies FBA FBA is required in Special Education for children who present challenging behaviors. 300.530 Authority of school personnel. LEAs must supply qualified personnel to do proper FBA s There is little guidance specifying the parameters of Who (can or) should do them
Overall Plan Goal: 2 Experts in each AEA (n=9) with the capacity to: Fluently Conduct the full range of FBA s Train others in the AEA to do appropriate FBA s Triage children with Challenging Behaviors Green: No, not our problem; Deal! (e.g., eye rolling, protesting) Yellow: Low to High Level FBA s used to develop BIP s Red: Problem exceeds capacity of AEA staff due to Time Resources 3-5 year plan to provide training to all 9 AEA s The initial project has been used to develop a 2 nd project specifically with school districts
Area Education Agencies (AEA): Providing Specialized Services in Iowa Schools
Iowa vs New Jersey - Demographics Land Mass Virginia is 25% smaller Population (2015) Virginia (14 th ) = 8.4 Million 202 per Square Mile Iowa (30 th ) = 3.1 Million 56 per Square Mile
Stats of Behavior in Iowa Functional Behavior Assessments completed: 2010 6,897 2014 13,417 Students placed out-of-district: more than 650 students for behavior/mental health reasons Don t know current numbers Students placed out-of-state more than 130 for behavior/mental health reasons 86 currently (33% decrease)
FBA, a Path to a Smoother BIP...
To Accomplish the Goal: We needed goals for trainees: Developed 3 Levels of Expertise (NIH) L1: Novice Can do FBA w/ Assistance L2: Intermediate Can do FBA w/o Assistance L3: Advanced Fluent users of FBA, Can train others (Roll out a credentialing system once goal is achieved)
Procedures (IV) University of Iowa CDD (Hands-On Training) Clinic (to train skills) School Visits (to generalize skills) Department of Education (Didactic) Basic Course Sequences through the Online Delivery Scope & Sequence
The DVs Essential Skills Data Collection 10 second Partial Interval Graphing Percentage of Interval (FA/SA) Percentage of Choice (PA/COA) Preference Assessment Forced Choice Free Operant Descriptive Analysis ABC Structured Functional Analysis Brief Extended Reversal Design Multi-element Data Analysis/Decision Making Visual Inspection Hagopian Analysis
The DVs Complimentary Skills Goal: Is to have at least 2 Experimental Analyses in each Advanced Level s CBS repertoire. Functional Analysis plus any of the following: Structural Analysis Antecedent Assessment Concurrent Operants Assessment Choice Assessments
Assessing our DV s: Multi-Model Assessments Screening Tools Self-assessment survey Do you think you have expertise in the area of FBA? Knowledge Exam If you think you can do FBA, you should be able to answer questions about FBA. Direct Measure: Direct Observation of skill Show us that you can do FBA. Lagging Measure: Evaluation of FBA & BIP data: Appropriate match to function Level of sophistication
1) That people reported that they didn t very know much about FBA (Self-Assessment) 2) Knowledge Examination and Direct Observation confirmed the above statement
Dependent Measure #1 Screening Tool
AEA 1 AEA 267 AEA 8 AEA 9 AEA 10 AEA 11 AEA 12 AEA 13/14 AEA 15/16 Experts By Year - Baseline Total L3: Advanced 0 L2: Intermediate 1 1 (3%) L1: Novice 2 1 2 1 6 (19%) Fundamental Awareness 5 3 3 5 1 4 1 22 (71%) N/A 1 1 2 (6%) N=31 trainees
AEA 1 AEA 267 AEA 8 AEA 9 AEA 10 AEA 11 AEA 12 AEA 13/14 AEA 15/16 Experts By Year End Y1/Begin Y2 Total L3: Advanced 0 L2: Intermediate 1 1 (2%) L1: Novice 2 1* 5 7(1) (17%) Fundamental Awareness 5 6* 5* 1* 3* 3 2 5 2* 32 (78%) N/A 1 1 (2%) N=41 trainees
AEA 1 AEA 267 AEA 8 AEA 9 AEA 10 AEA 11 AEA 12 AEA 13/14 AEA 15/16 Experts By Year End Y2/Begin Y3 Total L3: Advanced 0 L2: Intermediate 1 2 5 2 10 (20%) L1: Novice 3 2 1 1 3 1 1 12 (24%) Fundamental Awareness N/A 6* 4 3 5* 3 1 3 9* 34 (57%) N=56 trainees
AEA 1 AEA 267 AEA 8 AEA 9 AEA 10 AEA 11 AEA 12 AEA 13/14 AEA 15/16 Experts By Year End Y3/Begin Y4 Total L3: Advanced 2 2 4 (8%) L2: Intermediate 1 1 4 1 1 1 9 (18%) L1: Novice 5 2 1 1* 2 1 3 1 16 (32%) Fundamental Awareness N/A 3 3 1 4* 1 9 21 (42%) N=50 trainees
AEA 1 AEA 267 AEA 8 AEA 9 AEA 10 AEA 11 AEA 12 AEA 13/14 AEA 15/16 Experts By Year End Y4/Begin Y5 Total L3: Advanced 2 2 2 2 6 2 3 1 20 (46%) L2: Intermediate 1 2 1 2 2 1 1* 10 (23%) L1: Novice 5 1 1 1 1 9 (21%) Fundamental Awareness N/A 3 1 4 (9%) N=43 trainees
AEA 1 AEA 267 AEA 8 AEA 9 AEA 10 AEA 11 AEA 12 AEA 13/14 AEA 15/16 Experts By Year End Y5/Begin Y6 Total L3: Advanced 5 2 5 1 7 2 3 2 27 (49%) L2: Intermediate 1 1 2 4 (7%) L1: Novice 5 1 1 1 8 (14%) Fundamental Awareness 3 1 8 2 1 15 (27%) N/A 1 1 (2%) N=55 trainees
AEA 1 AEA 267 AEA 8 AEA 9 AEA 10 AEA 11 AEA 12 AEA 13/14 AEA 15/16 Experts By Year End Y6 Total L3: Advanced 5 2 5 2 8 2 2 2 28 L2: Intermediate 1 1 2 3 2 1 10 L1: Novice 5 1 1 3 3 13 Fundamental Awareness N/A 3 1 1 1 1 1 8 N=59 trainees
Dependent Measure #2 Screening Tool
AEA 1 AEA 267 AEA 8 AEA 9 AEA 10 AEA 11 AEA 12 AEA 13/14 AEA 15/16 Experts By Year Exam End Y3 Total L3: Advanced 3 3* 1* 2* 6 2 17 (3) (31%) L2: Intermediate 1 1 3 ++ 5 (2) (13%) L1: Novice 3 + 1* 2 ++ 1 1 2 1 + 11 (5) (15%) Fundamental Awareness 1* 1 3 +++ 1 1 7* +++ +++ 14 (11) (22%) N/A 2 1 2 5 (19%)
Exam Score Exam Scores as a Cohort 100% Score on FBA Exam by Years Involved 90% 80% 70% 82 76 71 60% 50% 40% 30% 33% 93% 70% 82% 84% 20% 10% SD = 10.5 SD = 5.6 0% Non-Experts Experts 1 Year 2 Years 3 Years
AEA 1 AEA 267 AEA 8 AEA 9 AEA 10 AEA 11 AEA 12 AEA 13/14 AEA 15/16 Experts By Year Exam 1 Year Total L3: Advanced 0 L2: Intermediate 2 2 (13%) L1: Novice 1 2 1 4 (27%) Fundamental Awareness N/A 3 6 9 (60%) N=15 trainees
AEA 1 AEA 267 AEA 8 AEA 9 AEA 10 AEA 11 AEA 12 AEA 13/14 AEA 15/16 Experts By Year Exam 2 Years Total L3: Advanced 1 1 1 3 (30%) L2: Intermediate 0 L1: Novice 1 1 (10%) Fundamental Awareness 1 1 2 (20%) Dropped Out 1 3 4 (40%) N=10 trainees
AEA 1 AEA 267 AEA 8 AEA 9 AEA 10 AEA 11 AEA 12 AEA 13/14 AEA 15/16 Experts By Year Exam 3 Years Total L3: Advanced 3 2 1 6 2 14 (44%) L2: Intermediate 1 1 1 3 (9%) L1: Novice 2 1 1 2 6 (19%) Fundamental Awareness 1 1 1 3 (9%) N/A 2 1 3 6 (19%) N=32 trainees
AEA 1 AEA 267 AEA 8 AEA 9 AEA 10 AEA 11 AEA 12 AEA 13/14 AEA 15/16 Experts By Year Exam End Y4 Total L3: Advanced 3 4 2 4 2 8 2 5 2 32 (83%) L2: Intermediate 1 1 (3%) L1: Novice 3 + 1 4 (10%) Fundamental Awareness N/A 1 1 + 2 (5%)
AEA 1 AEA 267 AEA 8 AEA 9 AEA 10 AEA 11 AEA 12 AEA 13/14 AEA 15/16 Experts By Year Exam End Y5 Total L3: Advanced 3 4 3 4 3 16 3 4 3 43 (84%) L2: Intermediate 1 1 2 (4%) L1: Novice 3 3 (6%) Fundamental Awareness 1 1 2 (4%) N/A 1 1 (2%)
AEA 1 AEA 267 AEA 8 AEA 9 AEA 10 AEA 11 AEA 12 AEA 13/14 AEA 15/16 Experts By Year Exam End Y6 Total L3: Advanced 3 4 3 6 3 16 3 4 3 45 (86%) L2: Intermediate 1 1 (2%) L1: Novice 3 1 4 (8%) Fundamental Awareness N/A 1 1 2 (4%)
Dependent Measure #3 Direct Observation Measure
AEA 1 AEA 267 AEA 8 AEA 9 AEA 10 AEA 11 AEA 12 AEA 13/14 AEA 15/16 DO Score End of Year 1 (2010) Total L3: Advanced 0 L2: Intermediate 0 L1: Novice 0 Fundamental Awareness N/A 8 3 4 1 8 2 5 1 32
AEA 1 AEA 267 AEA 8 AEA 9 AEA 10 AEA 11 AEA 12 AEA 13/14 AEA 15/16 DO Score End of Year 2 (2011) Total L3: Advanced 0 L2: Intermediate 1 1 (2%) L1: Novice 0 Fundamental Awareness N/A 8 6* 5* 1* 5* 8 2 5 1* 41 (10) (98%)
AEA 1 AEA 267 AEA 8 AEA 9 AEA 10 AEA 11 AEA 12 AEA 13/14 AEA 15/16 DO Score End of Year 3 (2012) Total L3: Advanced 1 2 6 1 10 (20%) L2: Intermediate 0 L1: Novice 0 Fundamental Awareness 9 * 4 * 4 5 * 2 2 5 10 * 42 (15) (80%) Dropped Out 1 1 3 5 (4)
AEA 1 AEA 267 AEA 8 AEA 9 AEA 10 AEA 11 AEA 12 AEA 13/14 AEA 15/16 DO Score End of Year 4 (2013) Total L3: Advanced 2 2 1 2 7 2 4 2 22 (72%) L2: Intermediate 2 1 1 4 (12%) L1: Novice 2 2 (6%) Fundamental Awareness Dropped Out 1 1 1 3 (9%)
AEA 1 AEA 267 AEA 8 AEA 9 AEA 10 AEA 11 AEA 12 AEA 13/14 AEA 15/16 DO Score End of Year 5 (2014) Total L3: Advanced 2 2 5 7 2 4 2 22 (73%) L2: Intermediate 2 4 (13%) L1: Novice 2 (7%) Fundamental Awareness 1 1 3* 9 3 2 2 (7%) Dropped Out 1 2 1
AEA 1 AEA 267 AEA 8 AEA 9 AEA 10 AEA 11 AEA 12 AEA 13/14 AEA 15/16 DO Score End of Year 6 (2015) Total L3: Advanced 3 2 5 2 7 2 3 2 26 (73%) L2: Intermediate 1 1 (13%) L1: Novice 1 1 (7%) Fundamental Awareness Dropped Out 1 1 1* 9 3 2 3 20
Dependent Measure #4 Self-Assessment, Exam Results and Direct Observation Measures Lowest of the 3 Measures Combined
AEA 267 AEA 8 AEA 9 AEA 10 AEA 11 AEA 12 AEA 13/14 AEA 15/16 Combined Results End of Year 2 (Self-Assessment & Direct Observation) Total L3: Advanced L2: Intermediate L1: Novice 1 1 Fundamental Awareness 5 2 2 7 2 5 23
AEA 267 AEA 8 AEA 9 AEA 10 AEA 11 AEA 12 AEA 13/14 AEA 15/16 Combined Results End of Year 3 (Self-Assessment, Exam, & Direct Observation) Total L3: Advanced 1 2 4 1 8 L2: Intermediate 2 2 L1: Novice 0 Fundamental Awareness 4 2 5 1 2 5 19
AEA 267 AEA 8 AEA 9 AEA 10 AEA 11 AEA 12 AEA 13/14 AEA 15/16 Combined Results End of Year 4 (Self-Assessment, Exam, & Direct Observation) Total L3: Advanced 1 2 1 2 7 2 3 1 19 L2: Intermediate 3 1 1 1 6 L1: Novice 1 1 2 Fundamental Awareness 1 2 1 4
AEA 267 AEA 8 AEA 9 AEA 10 AEA 11 AEA 12 AEA 13/14 AEA 15/16 Combined Results End of Year 5 (Self-Assessment, Exam, & Direct Observation) Total L3: Advanced 3 2 4 7 2 3 2 23 L2: Intermediate 1 1 2 4 L1: Novice 1 1 Fundamental Awareness 3 3
AEA 267 AEA 8 AEA 9 AEA 10 AEA 11 AEA 12 AEA 13/14 AEA 15/16 Combined Results End of Year 6 (Self-Assessment, Exam, & Direct Observation) Total L3: Advanced 3 2 5 2 7 2 3 2 25 L2: Intermediate 1 2 L1: Novice 1 1 Fundamental Awareness 1 1 1 9 3 3 2 21
Advanced Level Practitioners for Functional Analyses - Baseline 0 (2) 0 (2) 0 (5) 0 (8) 0 (7) 0 (2) 0 (5) 0 (1)
Advanced Level Practitioners for Functional Analyses - Currently 3 (3) 1 (1) 3 (1) 0 (9) 8 (8) 2 (1) 3 (3) 1 (3) 5 (1)
BCBAs That Trained Through This Project 0 1 0 0 0 8 0 1 2
Statistical Analysis of the Work: Engagement Mattered FTE Attendance - Number of children evaluated Discipline and Years of Experience did not
Evaluation of CBS Data Attrition Those that did not finish included 0.0 FTE to 0.20 FTE (up to 1 day) 18 Dropped Out 95% 1 Finished 0.41 FTE to 0.79 FTE (2+ days 2 Dropped Out 33% 4 Finished 0.21 FTE to 0.40 FTE (1+ days) 4 Dropped Out 44% 5 Finished 0.8 FTE to 1.0 FTE (4+ days) 7 Dropped Out 31% 16 Finished *Note 0.20 FTE = 1 day/week
Evaluation of CBS Data We ran simple correlations between the data we collected and each IV and DV pairing. Anything that came back significant was later entered into a regression analysis to yielded the best prediction equation for determining higher scores on each DV.
Results Self-Assessment Average FTE and Years in the Project were the best predictors of performance on this measure Knowledge Exam Years in the Project, Direct Observation Score, and Degree (School Psychologists) were the best predictors on this measure. Direct Observation Cumulative Attendance and Exam score were the best predictors of performance on this measure
to the AEA CBS teams and cost savings?
Number of Assessments Conducted Per Year 100 90 Number of Experimental Analyses Conducted per Year by CBAT Members at Advanced Level (n = 6) August 2007 to May 15, 2013* Training YEAR 3 92 YEAR 4 98 80 70 60 50 Training YEAR 2 Preference Assessment 40 30 Training YEAR 1 34 20 10 0 Baseline 11 4 2007-2009 2009-2010 2010-2011 2011-2012 2012-2013* Academic School Year
Number of Assessments Conducted Per Year 100 90 80 70 60 50 40 30 20 10 0 Number of Experimental Analyses Conducted per Year by CBAT Members at Advanced Level (n = 6) August 2007 to May 15, 2013* Baseline Training YEAR 1 4 2 3 11 Training YEAR 2 2007-2009 2009-2010 2010-2011 2011-2012 2012-2013* 34 15 Academic School Year Training YEAR 3 92 24 YEAR 4 98 56 Preference Assessment Functional Analysis
Number of Assessments Conducted Per Year 100 90 80 Number of Experimental Analyses Conducted per Year by CBAT Members at Advanced Level (n = 6) August 2007 to May 15, 2013* Training YEAR 3 92 YEAR 4 98 Preference Assessment 70 60 50 Training YEAR 2 56 Antecedent Analysis 40 30 20 10 0 Baseline 4 2 3 1 Training YEAR 1 34 11 5 19 17 2007-2009 2009-2010 2010-2011 2011-2012 2012-2013* 15 Academic School Year 24 36 Functional Analysis
Number of Assessments Conducted Per Year 100 90 80 70 60 50 40 30 20 10 0 Number of Experimental Analyses Conducted per Year by CBAT Members at Advanced Level (n = 6) August 2007 to May 15, 2013* Baseline 11 4 5 7 2.52 3 1 Training YEAR 1 Training YEAR 2 34 92 19 17 15 5 98 38 37 2007-2009 2009-2010 2010-2011 2011-2012 2012-2013* Academic School Year Training YEAR 3 24 YEAR 4 56 36 Preference Assessment Antecedent Analysis Concurrent Operants Assessment Functional Analysis
Percentage of On-Site Direct Support Cases Outcomes of On-Site Direct Support Student Cases August 2009 to March 2014 6 5 Training Year 1 Training Year 2 Training Year 3 Training Year 4 Year 5 5.3 5.3 5 4 Out of District Placement 3 3.2 2 1.7 1 0 2009-2010 2010-2011 2011-2012 2012-2013 2013-2014* Academic School Year
Translated This reflects a 68% reduction in out of district placements. Assuming $50K per out of district placement (conservatively), if scaled, this would equate to saving the state of Iowa $22M+. 650 currently kids in placement Would be 210 kids. Imagine the savings in New Jersey!!!
In Short We successfully accomplished our goal of creating True Positives to address Challenging Behavior in our school system. Additionally, we know have a credentialing system that helps ward off false positives. Data were used to maximize the chances that CBS s personnel in the project are configured to achieve project goals in 2-3 years.
Iowa Department of Education
What s next for the State? Years 4-6 Developing Pre-service training programs Briar Cliff University (Sioux City) started in 2013 Drake University (Des Moines) starting in 2017 University of Iowa (Iowa City) hopefully starting someday The notion is that they will produce trained people to permeate our systems: AEAs Schools Behavior Teams Teachers Private Consultants Residential program staff Clinic staff
Future Plans Years 7-10 Develop Day School/Residential School programs for children with most significant challenging behaviors. Yoking these programs with the Universities that would provide some of the best training opportunities for trainees anywhere.
When people rely on weak FBA data
In Summary Addressing Challenging Behavior at the State Level has not been: Fast or Easy; And there is No Magic Wand for behavior It takes time to teach people the needed skills to remedy challenging behaviors. We have put in: Training programs for AEA and LEA personnel Developed College Prep programs to teach future
Q & A Comments? Questions? You can also contact me at: Sean.Casey@iowa.gov 1.515.281.5447