An Evaluation of the Academic Assistance Program at AHS.

Submitted by Daniel McIlhenney

Submitted to AHS Principals Advisory Board
12/8/14

Summary:
This is an evaluation of the Academic Assistance Program (AAP). The program is offered
to all students in the high school. It meets twice a week for an hour after school. Students
are given a quiet environment to work on homework, use the media lab, and receive
tutoring from teachers. The decision to place a student in the program is a team decision
made by the parents, teachers, administrators and students. Typically, the program is
used by students that are failing some or all of their courses. The goal of program is to
reduce missed assignments (no more than 2 a week total) and for students to receive
passing grades in their core classes. The motive behind evaluating the program was to
have evidence to support the administration in deciding whether to expand the program
or revise the structure of the program.
The purpose of the evaluation was to determine the effect the program has on student
performance in core classes. The goal of the evaluation was to gather quantitative data on
a sample group of the students that use the program in order to determine if the program
is meeting the two main objectives (reduce missing assignments and for students to pass
core classes). In order to make these determinations, grades and missing assignments in
English, math, social studies, and science were tracked. First, the baseline grades and
number of missing assignments of students in the program were charted. Every other
week, student data was recorded to view how it had changed. In addition, students, staff,
and parents were surveyed to gain insight into the attitudes towards the program from key
stake holders. The evaluation contains both quantitative data on student performance as
well as qualitative data on personal experience with the program.
Two main groups of students were evaluated. The first group was small and it was the
group of students that had begun the year in AAP. The second group was the students that
were placed in AAP due to poor performance in the current school year. The first report
cards went out six weeks into the school year and it resulted in a large amount of new
enrollees. The evaluation tracked their number of missing assignments and grades as they
signed-up and progressed through the second marking period.
The data collection concluded at the end of November to coincide with the end of the first
trimester. The outcome was a sample of data that provided evidence indicating that the
Academic Assistance Program was not successful and was in need of either revision or
elimination. On average, students in the program continued to have failing grades in
many core classes and continued to have more than 2 missing assignments per week.
Students that began the year in the program, tended to have less missing assignments, but
struggled to earn passing grades. Students that entered the program mid year often saw a
temporary boost in both objectives. However, a regression typically followed. The survey
reports indicated general dissatisfaction and misunderstanding with the program. If the
program is to continue, it is highly recommended that the AAP undergo significant
changes for the 2015-16 school year.

Program Description:
The Academic Assistance Program (AAP) has been in existence at the high school in its
current form for eights years. The program is intended as a safety net for students that
struggle with homework completion or lack access to an adequate study environment.
The program meets every Monday and Thursday for 1 hour after school. AAP meets in
the cafeteria. It is staffed by two teachers for every 20 students. Students are often
allowed access to a media lab during this time period.
Students can be recommended for the program by teachers or a parent can independently
sign their student up. Either way, parent and student must fully commit to attending the
program. If students miss 2 sessions without excusal from their parent or guardian, then
they are removed from the program. Students must find a way to occupy their time. They
may study or complete homework during the sessions. Students must work quietly and
independently. Students stay in the program until they are removed by the parent due to
success or they are removed by administration due to lack of attendance.
The goal of the program is offer students a supported study environment after school. The
objectives are:
- To reduce the number of missing assignments to a maximum of 2 a week
- To improve student grades to at least passing all core classes
- To increase student access to educational technology
The typical student using the program is a first or second year high school student.
Students using the program tend to be failing some or all of their core classes. The
students that attend usually suffer from poor homework completion and low test scores.
Most students using the program have between 3 and 4 core classes (math, language arts,
science, social studies). Students using the program represent a diverse group of students.
There are students from in-district and school of choice, students of various races,
students with disabilities and students of varying social-economic status.
The teachers involved in administering the program are a math and science teacher with
15 years experience, a special education teacher with 10 years of experience , a social
studies teacher with 6 years of experience and an English teacher with 3 years of
experience.
The program is free to use. The school advertises the program in various ways. There are
flyers available in the office. The school website lists It under academic programs.
Teachers are encouraged to recommend it to students and parents. All teachers are given
packets of information on the program so that they can pass it a long to those that it may
benefit. There is no formal process for promoting the program.

Evaluation Method:
The evaluation was done in two parts. The first part was a quantitative evaluation. In this
portion, student data was tracked. The students were broken into two groups. Group A
were students that began the year in AAP. Group B were students that joined AAP during
the school year. Over the course of the trimester, both groups of students had two key
data points tracked. Their overall grades in their core classes was charted. In addition,
their weekly number of missing assignments was charted.
The quantitative data was used to determine if the program was able to reduce the
number of missing assignments amongst students and if it was able to assist students in
earning passing grades in core classes. Since the school is on trimesters, the tracking
periods can be broken into two marking periods: the first six weeks (MP1) and the second
six (MP2). All grades and assignments reset at the start of a new marking period. For
students in Group A missing assignments and grades for both MP1 and MP2 were
tracked. The majority of students in Group B entered near the end of MP1. Only 4
students entered after MP2 had begun. Therefore, with Group B, only missing assignment
and grade data for MP2 has been tracked.
In addition, qualitative data was obtained used a survey with a Likert scale. Parents,
students, and staff were surveyed. The goal was to obtain a general attitude from those
three stakeholder groups. Students were surveyed in areas such as quality of access to
technology and quantity of 1:1 academic assistance provided. Staff were surveyed in
areas such as compensation and available resources for the program. Parents or guardians
were surveyed in areas such as awareness of the program, clearness of the
procedures/outcomes of the program and satisfaction with the communication form the
program officials.
All students in the program were tracked and surveyed. A survey was sent to the parent of
every student in the program. Every teacher involved in the program was surveyed. Since
the program contained less than a 100 students, families and teachers sampling was not
appropriate.
Results:
# to use the program
# to finish the
program
% of students that
did not remain in
AAP through the
end of the trimester
Average number of
students lost per
week due to success

Group A
11
7

Group B
54
38

Total
65
45

36%

30%

31%

Average number of
0.3
students removed
from program due to
lack of attendance

1.3

1.6

The above table indicates that regardless of whether students request to enter the program
at the beginning of the year or are recommended for it as the year goes on, approximately
3 of every 10 students that uses the program is removed from the program due to poor
attendance.

QuickTime and a
decompressor
are needed to see this picture.

QuickTime and a
decompressor
are needed to see this picture.

The previous graph reveals that missing assignments remain a serious problem for
students in the Academic Assistance Program. Students that chose to enter at the
beginning of the year did not have as many missing assignments. However, after 6
weeks, they still averaged more than 2 missing assignments per week. Students from
Group B fared even worse. They averaged 3.5 missed assignments per week.
% of Core Classes Passed for Students in AAP
MP1 % of Core MP2% of Core
Classes Passed Classes Passed
Group A
20/32= 63%
14/25= 56%
Group B
N/A
78/154= 51%
Total
92/ 179= 51%
The table above indicates that students in both groups were passing just over half of their
core classes.

Parent Survey (Likert Scale 1= Strongly disagree, 5= Strongly agree)

Avg.
Answer
The rules of the AAP have been clearly

4.2

% of
Surveys
unreturned
51%

explained to me.
The rules of the AAP are fair and
reasonable.
The students are given significant access to
technology at AAP.

3.8

51%

4.2

51%

Teacher Survey (Likert Scale 1= Strongly disagree, 5= Strongly agree)

Avg.
Answer
The pay for AAP Is fair.
The hours for AAP are fair.
I am satisfied with AAP.
The students take advantage of their time at
AAP.

5
5
2
2.5

% of
Surveys
unreturned
0%
0%
0%
0%

Student Survey (Likert Scale 1= Strongly disagree, 5= Strongly agree)

Avg.
Answer
The rules of the AAP have been clearly
explained to me.
The rules of the AAP are fair and
reasonable.
I am satisfied with AAP.
The students are given significant access to
technology at AAP.
There is a significant amount of time to get
1 on 1 assistance from teachers.
I take advantage of my time in AAP.

% of
Surveys
unreturned
15%

2.2

15%

2
1.5

15%
15%

4.3

15%

3.7

15%

Discussion:
The program sought to reduce student missing assignments (no more than 2 a week) and
get students to pass all of their core classes. In addition, it sought to improve student
access to technology. Looking at the results, the program seems to be failing at every
metric. There seems to be four major issues with the program.

- The first issue is that 30% of the students that enter, are removed for lack of attendance.
The school should examine ways to improve program retention. While parents mostly felt
that the rules were fair and explained, the response rate for low. Perhaps, the parents that
did not respond had different opinions on the fairness of the rules and the clarity with
which those rules were explained. Multiple attempts were made to reach those parents
without success. This could reflect a level of dissatisfaction with the program. In
addition, the students were very adamant about their disagreement with the rules of the
program. Also, the students felt less clear on the rules. Going forward, the district may
want to re-examine the attendance policy and re-examine how it presents the program
rules to parents and especially students.
- The next major issue is with missing assignments. As the marking periods go on, the
missing assignments are adding up for students of both groups. AAP has been unable to
reach their objective of 2 missing assignments per week. Teachers have indicated that
students do not seem to utilize their time in AAP. Could their be adjustments to the way
the actual time in AAP is structured to help the students be more efficient?
- The greatest issue is that the students in the program are on average only passing half of
their core classes. This means that many of these students will have to repeat courses, be
reclassified, take summer school, and not graduate on time. Teachers have felt that the
schedule and pay is fair. Is there the possibility that the district could expand the
program? Instead of two days a week, what about three or four? Instead of an hour, could
the program be lengthened?
- Finally, teachers and students seem to agree that the program is not providing a
significant access to technology. Are there computer labs available after school that could
be used instead of the cafeteria? Using a media lab would provide technology to every
student. It may also help increase on task behavior, improve assignment completion and
overall grades.
The quantitative data reveals a clear gap between the objectives of the Academic
Assistance Program and the results. As the marking periods progress, missing
assignments grow beyond an acceptable number. The students that voluntarily enter the
program do tend keep the missing assignments lower than students recommended for the
program mid year. Students in Group A are able to sometimes maintain the minimum
number of missing assignments. On average, students in Group B are never on track for
the objective. However, this ability for Group A to come close to their missing
assignment goal does not carry over to their overall grades. Students in Group A are still
failing just above half of their core classes. Still, the results do seem to favor the students
that volunteer themselves for the program at the beginning of the year.
The qualitative data provides some interesting insight and some lack of insight. The
parents that returned the surveys were satisfied. The missing half of the survey data may
tell a different story. It would be recommended that in the future, the program develops a
strategy to survey every family that exits the program. The evaluation made multiple

attempts to reach out, but perhaps administration could devise a plan that was more
successful. That missing data is important. As for teachers, they are generally satisfied.
However, they do not see the students as taking advantage of the time. Adjusting the
program to maximize student effort is critical going forward. Students seemed to find the
program unfair and were not happy with it. It is recommended that the administration
speak to students in the program to get more detailed information as to how the program
could be improved.
Ultimately, the program is not meeting its objectives. There are a few options going
forward. The school should revise the program so that it can successfully meet its goals.
The school could research an alternative program/strategy and implement it in place of
the Academic Assistance Program. Finally, the school could consider eliminating the
program altogether. The budget for the program is a significant factor in the decision.
This evaluation was not made privy to the budget expenditures for the program. Without
that knowledge, the evaluation can not completely recommend which direction the school
should move forward with. The evaluation can only say that a change is necessary.

Project Cost:
In order to carry out the evaluation for the Academic Assistance Program,
quantitative and qualitative data will need to be collected, sorted, and analyzed. The
school administration will need to sit down with evaluation team to outline the program
and key objectives. Student data ranging from grades to missing assignments must be
collected and graphed. Surveys will be drafted and distributed. The surveys will be
collected. There will be follow-up contact made to those that do not respond the surveys.
After all information is collected, the data will be analyzed. The school will receive
information on the ability of the program to meet the established objectives. A report will
be published and future recommendations will be made.
Schedule:
1. Interview with program administration 2 hours
2. Observation of program 1 hour
3. Review of program documents, current data- 2 hours
4. Drafting, production and distribution of surveys- 3 hours
5. Collection, follow-up and tabulation of surveys- 6 hours
6. Bi-weekly quantitative data collection- (6 times, 3 hours a piece) 18 hours
7. Quantitative data tabulation and analysis- 4 hours
8. Production of final report- 4 hours
9. Presentation of results -1 hour
Total estimated hours: 77
Daily Rate of $500 ($62.50 per hour)
Total estimated cost: $4,812.50

10

Dear Parent/Guardian,
We are conducting an evaluation of the Academic Assistance Program. The goal is create
a program that can offer the most benefit to the students at our school. Your opinion is
important to us and would be very helpful in shaping the future of the program. It may
remain anonymous. Please answer the 3 questions below. When completed, you or your
student can drop it off at the office.
Thank you again for your time,
Daniel McIlhenney
Program Evaluator
Indicate as follows:
1- Strongly Disagree
2- Disagree
3- Undecided
4- Agree
5- Strongly Disagree

The rules of the AAP have been clearly

explained to me.
1

The rules of the AAP are fair and

reasonable.
1

The students are given significant access to

technology at AAP.
1

11

Dear Teacher,
We are conducting an evaluation of the Academic Assistance Program. The goal is
determine the impact of the program and guide administration in making informed
decisions for the future of the program. In addition, this is part of a project for a class of
mine at Boise State University. In no way is this being used as an evaluation towards the
performance of the staff. It may remain anonymous When the survey is completed, it can
be returned to the office.
Thank you again for your time.,
Daniel McIlhenney
Program Evaluator
Indicate as follows:
1- Strongly Disagree
2- Disagree
3- Undecided
4- Agree
5- Strongly Disagree

The pay for AAP Is fair.

1

The hours for AAP are fair.

1

I am satisfied with AAP.

1

The students take advantage of their time at

AAP.
1

12

Dear Student,
We are conducting an evaluation of the Academic Assistance Program. The goal is
determine how successful the program has been and help administration make informed
decisions for the future of the program. In addition, this is part of a project for a class of
mine at Boise State University. In no way is this being used against any student. It may
remain anonymous. When the survey is completed, it can be returned to the office.
Thank you again for your time.,
Daniel McIlhenney
Program Evaluator

Indicate as follows:
1- Strongly Disagree
2- Disagree
3- Undecided
4- Agree
5- Strongly Disagree

The rules of the AAP have been clearly

explained to me.
1
2
3
4
5
The rules of the AAP are fair and
reasonable.
1
2
3
4
5
I am satisfied with AAP.
1
2
3
4
5
The students are given significant access to
technology at AAP.
1
2
3
4
5
There is a significant amount of time to get
1 on 1 assistance from teachers.
1
2
3
4
5
I take advantage of my time in AAP.
1
2
3
4
5

13