Evaluation of the Positive Behavior Interventions and Support (PBIS) program as it relates to tardiness at the high school

Submitted to: Jeriamy Jackson Assistant Principal, Monroe High School Chris Medenwaldt Principal, Monroe High School Joe Monroe Director of Pupil Services, Monroe School District Cory Hirsbrunner Superintendant, Monroe School District

Submitted by: Nathanael Cannon

1|Page

Summary Monroe High School and the School District of Monroe, located in Monroe, WI, have used the Positive Behavior Interventions and Support (PBIS) over the last four years. It was originally implemented at the high school level as a response to the problem of excessive student tardiness during 1st and 5th hour classes. Following the integration data provided by an outside company called School Wide Information System (SWIS) seemed to indicate a decrease in the overall amount of lateness throughout the entire school day. After a more thorough examination of the data from SWIS, it appears that the goal of reducing student tardiness to less than 5% has been achieved. However, after reviewing the current 2013-2014 school year data through November 26, 2013 (or approximately one-third of the school year) using the online grade book/attendance program Skyward, the data provided by SWIS might not be as accurate. In the current school year, over 45% of the student population has already accumulated at least one tardy. Further evaluation is required to determine which set of data is correct because SWIS and Skyward share the same raw data set. Description of the program evaluated The intended user-base of the Positive Behavior Interventions and Support (PBIS) system are all twelve of the Cooperative Educational Service Agencies (CESA), along with the Wisconsin Department of Public Instruction. PBIS is associated with the Response to Intervention (RtI) program which addresses school-wide behavioral issues in a systematic and proactive manner. Because all twelve CESAs are participating in PBIS, every school district in the state is therefore using the program. Addressing the relevant issues within the different educational levels of a school district is critical to the success of these interwoven programs. Each school will have their own unique challenges; for example timeliness is a major issue at the high school level that needs to be addressed with the students as they prepare to enter the work force in the upcoming years. When PBIS was first implemented at the high school during the 2009-2010 school year, there were two key target objectives: School-wide behavior with a systematic and proactive approach Improve student punctuality, focusing especially on the 1st and 5th hour classes

To assist administrators and educators, several resources were made available. These include us of an electronic grade book/attendance system called Skyward, special SWIS-provided website access to monitor behavioral problems, and a paper form (also provided by SWIS) that would be filled out whenever a behavioral problem was encountered. This form was prepared in triplicate, with one copy saved by the administration, one copy given to the parent/guardian of the student, and the last copy given to the teacher who wrote it. Copies of these reports and a sample form is given in the Appendices.
2|Page

Evaluation Method There were several individuals involved in this evaluation report. Jeriamy Jackson, Assistant Principla Monroe High School, agreed to gather the information that was available to the administrators through the SWIS website. He also provided background information on the methodology and reasons for implementing the RtI and PBIS programs. School Secretary Mary Grinnell provided tardy totals through November 26, 2013 for the current 2013-2014 school year. Director of Pupil Services Joe Monroe supplied the electronic SWIS form that is made into triplicates for teachers to report student behavioral incidents. The Superintendent of the School District of Monroe should be advised of the PBIS program status as it progresses within her district, as well as any staff members from the high school who are interested in the results. Past and current students reported behavior was the driving force for the data in this report. This reporting by the staff will allow the high school to be proactive in their approach for the incoming students. Because a school does not exist in a vacuum, the use of the Discrepancy Model is the best approach to evaluate the PBIS program. As it relates to this situation, invested members of the high school will evaluate the causes and effects of student behavior to construct reasonable assumptions and adjust the program as needed. Recall that PBIS started out at the high school as a program to decrease the number of student hours being tardy from 10% to 5%, and to optimistically close to 0%. This would be a classic example of the Goal-Based Method of evaluation. However, with PBIS being used concurrently with RtI, the other Behavioral Problems criteria on forms and the website reports came along with the program as demonstrated by the charts and tables in the Appendices. During the evaluation period which ran from the September 2 November 26, 2013, the school functioned normally. Staff where required to enter their attendance via Skyward within the first 5-10 minutes of class or at their earliest convenience if they were a traveling teacher or an encore subject area. Exceptions to this requirement were made by administration on a case-by-case basis. Substitutes were not required to enter this information into Skyward but instead were instructed to keep track of those who were or were not in class and inform the school secretaries at the end of the school day; Skyward attendance records would be updated accordingly. None of the current 735 students were interviewed during this evaluation process because their behavior might change if they became aware that they were being evaluated more than usual. One staff member from each department was blindly and randomly selected to take part in a survey asking them three questions about PBIS.

3|Page

Results The following table reports the SWIS data as it relates to tardiness in the high school for the school years 2009-2014. 2009-2010 Not available 441 602 73.3% 180 1440 Not available Not available 716 1015 1333 76.1% 180 1440 1,031,040 0.10% 698 704 1021 69.0% 180 1440 1,005,120 0.07% 709 803 1464 54.8% 180 1440 1,020,960 0.08% Number of students Number of reported tardy reports according to SWIS Number of reported SWIS forms Percent of reported SWIS forms that were about tardiness Number of school days Total number of school hours (8-hour day) Total number of school hours of ALL students Percent of school hours where students have been tardy Number of students Number of reported tardy reports according to SWIS Number of reported SWIS forms Percent of reported SWIS forms that were about tardiness Number of school days Total number of school hours (8-hour day) Total number of school hours of ALL students Percent of school hours where students have been tardy Number of students Number of reported tardy reports according to SWIS Number of reported SWIS forms Percent of reported SWIS forms that were about tardiness Number of school days Total number of school hours (8-hour day) Total number of school hours of ALL students Percent of school hours where students have been tardy Number of students Number of reported tardy reports according to SWIS Number of reported SWIS forms Percent of reported SWIS forms that were about tardiness Number of school days Total number of school hours (8-hour day) Total number of school hours of ALL students Percent of school hours where students have been tardy
4|Page

2010-2011

2011-2012

2012-2013

2013-2014

735 Number of students Not available Number of reported tardy reports according to SWIS Not available Number of reported SWIS forms Not available Percent of reported SWIS forms that were about tardiness 180 Number of school days 1440 Total number of school hours (8-hour day) Not available Total number of school hours of ALL students Not available Percent of school hours where students have been tardy Due to turnover of administrators and secretarial staff at the high school, glitches in reporting resulted in SWIS data being unavailable at the time of this evaluation. Skyward data was used instead.

The following table reports the data collected by Skyward as it relates to tardiness in the high school for approximately one-third of the 2013-2014 school year. Following this table is a graphical representation of the data. Raw Skyward data may be found in the Appendices.
2344 7.0 735 61 488 358,680 45.2% 3 1 9 251 75.4% 0 56 0.7% 1.4% 0.0% 11.5% 0.6% 0.2% Total number of hours tardy Average number of tardies per tardy student Number of students Number of school days Total number of school hours (8-hour day) Total number of school hours of ALL students Percent of students who have been tardy Median of number of hours tardy Mode of number of hours tardy Standard deviation of number of hours tardy Number of tardy students at or below the standard deviation Percent of tardy students who are at or below the standards deviation Minimum number of tardies for a student (p.14) Maximum number of tardies for a student (p.22) Percent of school hours where students have been tardy Percent of the number of school hours missed by the average tardy student Percent of the number of school hours missed by the minimum tardy student Percent of the number of school hours missed by the maximum tardy student Percent of the number of school hours missed by the median tardy student Percent of the number of school hours missed by the mode tardy student

5|Page

The following table reports results from the random survey that was given with blind to one member of each department at the high school. The survey asks the following three Yes/No questions as it relates to PBIS. 1. Do you know what the PBIS program is? 2. Do you know what the PBIS program was designed to do at the high school? 3. From your observations as an educator, do you think the program is working at the high school? Eight responses were received; one core and one encore subject were not represented. Therefore, the Director of Curriculum & Instruction completed the survey for these two departments. Staff survey number Totals Question #1 Yes 100.0% No 0.0% Question #2 Yes 100.0% No 0.0 % Yes 25.0% Question #3 Somewhat/ No Maybe 50.0% 12.5%

I don't know 12.5%

6|Page

Discussion The purpose of this study was to determine the effectiveness of the PBIS program at the high school level as it relates to tardiness with the students. The original goal for PBIS at the high school was to decrease the number of student hours being tardy from 10% to 5%, and optimistically close to 0%. According to the SWIS data for the 20092013 school years, this goal has been achieved. At least 50% of the reported SWIS forms concerned tardiness with students but the percentage of school hours where students have been tardy was never greater than 0.1% of the entire school year. With that being said, there was no break down available for individual students since there could have been students without any tardies and some with multiple tardies throughout the school year. Over the summer of 2013, there was significant personnel turnover in the administration and secretarial staff. Because of this turnover, there was a lapse in the coordinating of data from the Skyward system to SWIS. With the understanding from administration that SWIS and Skyward use the same raw data, Skyward data was used to assess the tardiness data one-third of the way through the 2013-2014 school year. Based on previous data, it was extrapolated that the current tardies at Monroe High School would be in the 200-300 range. When the actual numbers came through from Skyward there were 2,344 tardies and over 45% of the student population had at least one recorded tardy. Even with this drastic increase from the previous years, this is still only corresponds to 0.7% of the total school hours where students were tardy. Another interesting result from the Skyward data on the 2013-2014 school year was that around 75% of the tardy population had less that the standard deviation of 9-tardies for the school year. There were just 95 students (approximately 13% of the student population) with just one tardy while only 28 students (approximately 4% of the student population) had 20 or more tardies so far. Such a drastic change and discrepancy between data generated by two different programs which share the same raw data is cause for concern. These numbers ought to be closer together than they really are. One possibility is that the data is not synchronized properly. If possible, previous years school year from Skyward should be compared to what SWIS has in their system. Another possible reason for this sudden increase in tardiness could be that the change in the high schools bell system: previously, it rang at 8:05 am, but this changed to 8:00 am for this current school year. This could have had a more profound effect upon tardies than originally predicted by administration. A final way to determine a cause for this jump in tardiness is to investigate Skyward data for each graduating classs tardy totals. Over a minimum of four years, this could provide a more equalized view as to when in the students high school career the tardies begin to become an issue. The staff survey was given with randomly and with blind selection to one member of each department at the high school. The survey asks the following three Yes/No questions as it relates to PBIS.

7|Page

1. Do you know what the PBIS program is? 2. Do you know what the PBIS program was designed to do at the high school? 3. From your observations as an educator, do you think the program is working at the high school? There were eight responses that were received. One core area and one encore subject were not represented. Therefore, the Director of Curriculum & Instruction completed the survey for these two departments. While all the staff surveys reported that they knew what the PBIS program was and what it was designed to do at the high school, the opinions of whether it was working or not were mixed. 62.5% reported that they thought the program was somewhat working or not working based on their observations. While there has been personnel turnover within the administration which could have contributed to this negative response to the program, there could be other reasons and further investigation is recommended. Overall, additional evaluation is needed to determine which data reporting system is correct. Regardless of which data system is being used, the total amount of instructional time for a school year that student tardiness affects is still less than 1%. This is encouraging that most of the educational time is being used efficiently between the staff and the students. However, determining why Skyward says that over 45% of the student population has a tardy this school year when SWIS says in previous years we have had less than 5% is cause for concern and further evaluation is strongly recommended. Project Cost Personnel Principal Investigator Travel Mileage 100 miles/day @ $0.56/mile 61 days @ $56.50/day $56.50 $3,446.50 TOTAL: $33,946.50

61 days @ $500/day

$30,500

This bill has been broken down into the 61 school days that I was at the High School observing, gathering data, and evaluating the information provided about PBIS. The mileage charge reflects a daily roundtrip commute from my offices in Madison to Monroe High School each day; the rate was established by the U.S. General Services Administration for privately-owned automobiles and was current as a January 1, 2013.

8|Page

Appendices SWIS data

9|Page

10 | P a g e

SWIS form

11 | P a g e

Skyward data

12 | P a g e

13 | P a g e

14 | P a g e

15 | P a g e

16 | P a g e

17 | P a g e

18 | P a g e

19 | P a g e

20 | P a g e

21 | P a g e

22 | P a g e

23 | P a g e

24 | P a g e

25 | P a g e

26 | P a g e

27 | P a g e

28 | P a g e

29 | P a g e

30 | P a g e

31 | P a g e

32 | P a g e

33 | P a g e

34 | P a g e

35 | P a g e