Evaluation of Destination Success Learning Systems

Prepared for: Florida County Public School #274

Prepared by: Angela Banks

B&B Associates, Jacksonville Florida May 7, 2013

Summary This evaluation was conducted to determine the educational value of the online software program Destination Success Learning Systems. Destination Success contains an entire collection of software for Reading and Math ranging from Kindergarten to 12th grade. It is widely used in the County School system to supplement classroom instruction and enhance student learning of reading and math skills. Reading strands covered are grammar, phonics, word study, genres, research and comprehension skills, text structure and writing. Many of the activities throughout this program are designed alongside the county curriculum, ensuring that the necessary skills for each learning level are covered. This also makes it easier for teachers to assign activities for students, as they merely need to search for the topic of that week, as opposed to each skill. The evaluation is to determine if the Destination Success activity directions parallel classroom instruction and if this program increases student learning. Information was gathered from student performance and teacher surveys to better understand what the program is offering academically and how it pertains to our learning standards. These findings will help teachers better plan for instruction throughout the year to ensure all the necessary standards are being addressed and students are prepared for the next grade level. After completing the surveys and analyzing data, it has been found that most teachers find the Destination Success Learning System, is a highly valuable tool for classroom instruction. All of the kindergarten through Second grade teachers who completed surveys said they use the software program daily or weekly in the classroom. It is used for students to practice the skills recently taught and for those who have yet to grasp skills previously taught. About half of the teachers login every week to assign new activities and check student progress. The other half of teachers login every few weeks or monthly. None of the early elementary teachers use the pre and post-tests available within the program, nor did they know about the student progress report. Most teachers like the easiness of assigning activities to students, and how the program parallels the curriculum. Common dislikes among teachers are students becoming bored with the babyish activities, students ability to click through games instead of working through them, and some activities provide less practice in some skills than others or they are explained poorly. The teachers do like the educational value of Destination Success and that is mostly entertaining to the students. Teachers also appreciate the ease in differentiating the activities for students varying levels of mastery. It was also found that Third grade teachers use the Destinations program much less than k-2 teachers. The upper elementary students take the State assessment (FCAT) and use FCAT Explorer software program to better meet their skill levels. This program also has a student progress report and teachers check it at least weekly to biweekly. Program Description Destination Success is an online educational software program adopted by our county school system several years ago and is a curriculum requirement for all students. Destination Success contains an entire collection of software for Reading and Math ranging from Kindergarten to 12th grade. The program was designed to parallel school curriculum, so there are enough

activities for the entire school year. Destination Success allows students extra practice on reading and math skills at their own pace. Reading strands covered are grammar, phonics, word study, genres, research and comprehension skills, text structure and writing. It uses interactive cartoon characters, colorful graphics, and fun sound effects to keep students engaged and interested in the activities. The teacher accounts on Destination Success, allow instructors to assign various activities for students based on individual needs and skills taught in class. Teachers can also monitor student completion of each activity individually or as a class. Another aspect of this program is that, teachers are able to assign activities on any skill level. All skill levels are accessible to all teachers, to assign for their students learning level. The Destinations program has been a valuable asset to classroom instruction, but due to recent changes in standards and curriculum, its relativity is now questioned. The process of evaluation of this program is based on several program objectives: 1. The software learning standards and state learning standards should correlate with few discrepancies. 2. Skills should progress easily from one level to the next. 3. Each skill level should be addressed and practiced in depth to allow sufficient practice and mastery. 4. Skills should be explained state standards language, to decrease confusion from instructor lessons. 5. Program provides student growth and progress charting for each skill or activity.

Evaluation Method The discrepancy model was conducted for this evaluation because it compares what is actually being accomplished to the original product purpose. The goal was to determine the variations between the programs stated purpose and what is actually happening. The software claims to cover specific learning standards, but this evaluation will determine to what degree those standards are actually covered. One major factor in this evaluation is the countys new standards implementation. Because the Common Core Standards are much more in depth on a given topic than the previously used Sunshine State standards, the evaluator will determine if the Destination Success program still meets the standards. Another prime concern is student interest in the software. Teachers are concerned that students are clicking through the activities and not taking the time to work through them and learn the skill. As this program is mostly used by early elementary teachers, the evaluation will focus on the kindergarten through 3rd grade standards and skills. Using the stated objectives, the following activities were conducted to evaluate the program: 1. Teacher or evaluator observations of students playing the same games during class hours. a. Teachers were asked to observe students participating in the Destination Success activities over one weeks time. They were to write down student behaviors such

2.

3.

4.

5.

6.

as frustration, clicking through games, lack of interests, ease of navigation, etc. This will also be done by the evaluator; however students behave differently when they are aware of another person watching them. At least one teacher (from each grade level) Kindergarten through 3rd grade will play and give a review of each game. a. Teachers were asked to play the game for themselves and make notes of how the skill was taught and how extensive the lesson and practice was. Aspects to take note of were; explanation of the skill, time spent in that activity, amount of practice, number of chances for correct answers, and responses for correct/incorrect responses. Have students verbally explain what they thought about the game (what was taught, what they learned, how was it explained). a. After playing a reading activity, the student is asked to explain what they did and learned from the activity to the evaluator or their teacher. Comparing state standards with software standards to determine correlations. a. The evaluator will compare and contrast the standards of Destination Success reading activities, 2003 Sunshine State Standards, and the 2012 Common Core Standards Comparing the standards with the activity during play to determine the extent of its coverage. a. The teachers and evaluator will play an activity or carefully observe the students during play, to see how well the program covers the standards in an activity. The evaluator is looking to see how much time is spent on that skill and how well it is explained. Teacher survey and interviews of overall thoughts of Destination Success program. a. The surveys will be emailed to all k-3rd grade teachers allowing them to complete within two weeks. b. Interviews will be conducted after surveys are completed. They will be one on one, or in small groups depending on teacher availability and preference.

Results The evaluator began this process by giving the 29 kindergarten through third grade teachers at Florida County Public School #274 a survey to complete. The survey consisted of 11 questions with scaled responses, and 3 open ended questions. The evaluator also asked for teachers to provide a copy of student progress forms from the Destination Success report section and a copy of pre and post test results of 2 skill areas. All k-3 teaches were emailed this survey, but only 14 were completed and returned. Of the 14 teachers that responded, 3 are kindergarten teachers, 4first grade teachers, 5 second grade teachers and 2 third grade teachers. Only one teacher provided student data from Destination Success. This was a second grade teacher. Below is the survey response from the 14 teachers:

1.

What grade do you teach?

KG 21% Daily 72%

1 grade 29% Weekly 7%

st

2 grade 36% Monthly 21%

nd

3 grade 14% Never 0

rd

2.

How often do your students use the Destination Success software program in class? How often do you assign activities for your students? How often do check your students progress on Destinations? To what extent do you feel Destination Success meets the new Common Core standards? To what extent do you feel Destination Success covers the reading skills you teach in class? How similar are the explanations in Destinations to your teaching of the skills? How often do you observe your students playing the activities on Destinations? How often do you observe your students clicking through the activities?

3.

Daily 0 Daily 0 Exactly 0 Exactly 0

Weekly 43% Weekly 21% Mostly 50% Mostly 43%

Monthly 57% Monthly 43% Somewhat 50% Somewhat 57%

Never 0 Never 36% Not at all 0 Not at all 0

4.

5.

6.

7.

Exactly 0

Mostly 64%

Somewhat 36%

Not at all 0

8.

All the time 0

Most of the time 43% Most of the time 43% 70-89% N/A

Sometimes 50%

Never 7%

9.

All the time 0

Sometimes 43% 50-69% N/A

Never 14% Less than 50% N/A

10. Using the Reports section of the teacher 100-90% account, what percentage of activities N/A are 100% completed by your class?

11. How often do you assign the pre and post assessments in Destination Success?

Weekly 0

Monthly 14%

Occasionally 0

Never 0

This survey provided a good overview on the student and teacher usage of Destinations Success, and also provides a better focus for the evaluation. 72% of teachers are using the Destination software on a daily basis. From the individual surveys, this 72% consists of k-2 teachers. The third grade teachers use the program les often, as they mostly us the FCAT Explorer software program. The third grade teachers feel this program more adequately addresses the skills and standards for their grade level, and better prepares the students for the State test. All 14 of the teachers assign activities for their students regularly. Using the ability to assign start and finish times for each activity, just over half of teachers prefer to assign groups at a time, while 43% prefer to sign up weekly.

Some discrepancies occurred within the student progress section of Destination Success. 64% of teachers said they check student progress weekly or monthly. However, when asked about the student reports section, none of the teachers knew what this was. During interviews, the teachers explained that on their main login page, there is an overview of all the activities assigned to the students. On this page it shows what percentage of each activity has been completed. The 64% that have been monitoring student progress have been using this page for information and not the student report section that provides more detailed information. The survey also shows that are about half and half about the content coverage in Destination Success. 100% of the teachers feel that Destinations at least somewhat of the new Common Core standards. 50% find it mostly covers the new standards and 50% think somewhat covers the standards. Upon visiting the Destination Success website, it is noted *Destination Reading does
not align to all standards. Those standards are not shown on this document. This document is a correlation of Destination Reading, to the Florida 2007 Sunshine State Standards.* Please see the appendix for the link

to full report of standards correlation. Of the 40 Sunshine State reading standards for 2nd grade, Destination Success covers 53% of them. When comparing the 30 Common Core reading standards to the Destination standards, Destination covers 60%. Though the program seems to address more of the Common Core standards, the activities only provide surface level practice. For example one vowel activity addresses the /oo/ sound as in pool, but does not address the second /oo/ sound as in book. When teachers observed the students participation in Destination activities, they provided more information during evaluator interviews. Students that were observed were chosen by skill levels by their teachers. Teachers selected one student from the lowest, average, and highest performance ranges. 86% of the teachers said they observe some students clicking through the games instead of working through them. During separate interviews and an open ended questionnaire, teachers explained that it was a mixture of students and circumstances involved in rushing through an activity. Every teacher gave very similar responses to the evaluator questions. The lower performing students sometimes click through an activity when they do not understand the activity or skill. They become frustrated and want to get it done quickly so they can move on to something more fun. Some higher performing students are observed rushing through activities when they become bored with the games and find it too easy. Activities that are designed to take 5-8 minutes to complete, students are finishing in 1-2 minutes. The 14% of teachers that said they never observe students rushing through activities said it is because they are either not watching the students, or merely have not seen at that exact time. In order to evaluate progress that resulted from participation, students knowledge of material covered in the course was to be measured by a pre- and post-test, and verbal survey after playing the game. Only the 2 third grade teachers had assigned pre and post-tests to their students. However, they did not provide the evaluator with their test results. The kindergarten through 2nd grade teachers had not assigned any test. Due to lack of data this section of the evaluation could not be completed. In the questionnaires and interviews, teachers provided more detailed information about the software program. When playing the games teachers noted some consistencies among each activity. The program gives a brief explanation of the skill, provides an example, and has the students practice before completing the game independently. A concern among teachers is that the program does not explain the skill independently of the activity. Teachers said in the games

they played, the skill was explained within the activity directions. This leads them to think the students are learning more about the game than the skill itself. One 1st grade teacher shared an experience with one of her students that she observed. The student told her how to move the ice blocks together to make a new word and the character dances, as opposed to telling her what compound words are. Though this skill was taught before the game was assigned, the students focus became fixated on playing the game instead of understanding the skill.

Discussion Many teachers feel that Destination Success learning is a valuable supplement and practice with teaching. It provides students with fun and entertaining educational activities that are directly related to the skills and topics being taught in the classroom. Teachers like how the activities are aligned with the curriculum and the amount of skill games that are provided. They also like how Destinations allows access to all levels of skills and activities, making it easier for teachers to differentiate for student needs. After discovering the Student Report section, the teachers were thrilled to see more detailed explanations of student progress in each activity. Some negatives mentioned were the students ability to click through activities as they want to. Teachers notice that some students become bored with the easiness of games or feel it is too babyish for them. Older students are more competitive and find Destinations to be too slow and lacking motivation. Academically, the program covers only about 60% of the grade skill requirements, which was much less than originally thought. A strong recommendation is to provide teacher training on the Student report section of Destination Success. All the teachers participating in this survey are efficient in navigating the majority of the site and can easily assign activities for the varying needs of their students. However, most were unaware of what else the program has to offer. Some teachers and the evaluator looked more closely at the report sections and were able to pull up progress reports by class, skill, activity, or individual students. Each report provides the activity, the date it was started and finished by the student, how much time was spent on that activity and its completion rate. There were areas for correct and incorrect responses, but all spaces were empty. Using this report could help teachers in planning instruction and determining individual student needs. Teachers that participated did mention other online programs that are used in class to support reading instruction. Programs such as Starfall and FCAT Explorer are the more common names that were discussed. The use of all programs may be the best recommendation to help with standards coverage and practice until the county becomes more familiar with the new Common Core Standards. This will provide more time for teachers to experiment with Destinations tests and reports to monitor student progress and better address their needs. Another recommendation is for administration to allow teacher collaboration time to find and create supplement activities to cover skills not addressed in Destinations. The evaluator is very aware, as she was told several times, of minimal planning and preparation time teachers have. If more time was allotted for teachers to work together on student activities, it could have a positive impact on learning gains.

Project Cost Personnel Angela Banks: 25 days @ $500 a day Brandy Smith: 10 days @ $250 a day Total Personnel Travel Gas and mileage Per Diem expenses Total Travel Communications Telephone Fax Total Communications Supplies and Materials Photocopies and Supplies Total Supplies and Materials Total Budget $ $ 200 200 $ 250 $ 50 $ $ 300 400 700 $12,500 $ 2,500 $15,000

$ 300

$16,200

The full evaluation of this program was completed in 25 days. The lead evaluator, Angela Banks, completed student observations, most of the teacher interviews and the survey. Her assistant Brandy Smith researched and compared the standards, and conducted some teacher discussions. Both evaluators are locally based so travel expenses are limited to vehicle maintenance and time spent on site. Communications between the school, teachers, and evaluators was mainly done over the use of cell phones, landlines and faxes. Supplies and materials consists of printing surveys and standards from all three sources, copies of teacher observation notes, and providing copies of final report to involved parties.

Appendix
Teacher Survey
Directions: Please circle the answer that describes your situation.

1.

What grade do you teach?

2.

How often do your students use the Destination Success software program in class? How often do you assign activities for your students? How often do check your students progress on Destinations? To what extent do you feel Destination Success meets the new Common Core standards? To what extent do you feel Destination Success covers the reading skills you teach in class? How similar are the explanations in Destinations to your teaching of the skills? How often do you observe your students playing the activities on Destinations? How often do you observe your students clicking through the activities?

Daily

Weekly

Monthly

Never

3.

Daily

Weekly

Monthly

Never

4.

Daily

Weekly

Monthly

Never

5.

Exactly

Mostly

Somewhat

Not at all

6.

Exactly

Mostly

Somewhat

Not at all

7.

Exactly

Mostly

Somewhat

Not at all

8.

All the time

Most of the time

Sometimes

Never

9.

All the time

Most of the time 70-89%

Sometimes

Never

10. Using the Reports section of the teacher 100-90% account, what percentage of activities are 100% completed by your class?

50-69%

Less than 50%

11. How often do you assign the pre and post assessments in Destination Success?

Weekly

Monthly

Occasionally

Never

Open-ended Questionnaire Directions: Please answer the questions below. Provide as much detail as possible. 1. Do your students take the time to work through each activity or do they rush and click through the games? Please explain. 2. What are some things you like about Destination Success reading activities? 3. What are some things you do not like about Destination Success reading activities? 4. What comments or thoughts do your students have about Destination Success reading activities?

Riverdeep Destination Reading Aligned to Florida Sunshine State Standards January 2007 Grade K http://downloads.hmlt.hmco.com/EdSchool/LMS4Resources/States/FL/2007/Rea ding/Florida%20Grade%20K%20Correlation%20to%20DR.pdf Grade 1 http://downloads.hmlt.hmco.com/EdSchool/LMS4Resources/States/FL/2007/Rea ding/Florida%20Grade%201%20Correlation%20to%20DR.pdf Grade 2 http://downloads.hmlt.hmco.com/EdSchool/LMS4Resources/States/FL/2007/Rea ding/Florida%20Grade%202%20Correlation%20to%20DR.pdf Grade 3 http://downloads.hmlt.hmco.com/EdSchool/LMS4Resources/States/FL/2007/Rea ding/Florida%20Grade%203%20Correlation%20to%20DR.pdf

CPALMS- Florida Teacher resource website Search engine for Common Core Standards by grade levels and subjects http://www.cpalms.org/Standards/FLStandardSearch.aspx

Correlation of 2nd grade 2007 Sunshine State standards, Destination Success reading standards, and Common Core 2012 standards https://docs.google.com/file/d/0B3_Wob67Pm9rTnJ2eWQzVmNoVGc/edit?usp=s haring