Escolar Documentos
Profissional Documentos
Cultura Documentos
Prepared for
ETR 531 - Fall 2014
Professor Richter
November 12, 2014
EXECUTIVE SUMMARY
Project Lead the Way Gateway is an introductory course designed to bring engineering and creativity within
one classroom for Junior High Students. This integrated design encourages higher-level learning and creates confidence
within the young students who will use new skills to solve complex problems.
The Project Lead the Way Gateway program at Lincoln Middle School seeks to create an innovative and
cutting-edge environment for their students. In order to accomplish this goal, it is determined that a learning
management system could help to streamline the learning process by utilizing a flipped classroom.
Canvas explains that choosing an LMS comes down to three questions:
Will it get used?
Will it adapt to your needs?
Is it reliable? (Instructure, n.d.)
The evaluation of Canvas within the context of the PLTW classroom will help Lincoln Middle School answer
those questions, and more, as they consider a future implementation of a 1:1 program. This evaluation has been
designed to determine the effectiveness of the introduction of Canvas learning management system into PLTW at
Lincoln Middle School. This evaluation seeks to address questions from a number of stakeholders, including students,
teachers, and School District 57 administration.
TABLE OF CONTENTS
I.
II.
Executive Summary
Program Description
A. PLTW Background
B. Evaluation Need
C. Evaluation Purpose
III.
IV.
Evaluation Design
A. Evaluation Questions
B. Evaluation Plan Matrix
C. List of Measures/Instruments
V.
VI.
VII.
PROGRAM DESCRIPTION
Project Lead the Way Background
This program evaluation will focus on a course at Lincoln Middle School in Mount Prospect, Illinois, called
Project Lead the Way Gateway. Project Lead the Way, or PLTW for short, originated in 1986 when a high school
teacher in New York began offering pre-engineering classes to his students in response to a shortage of engineering
employees. Eleven years later, the teacher partnered with the Charitable Leadership Foundation to establish PLTWs
high school engineering program. It later grew to include middle school courses, and this year began offering programs
to elementary schools. In over 5,000 schools across the country, PLTW is now the leading provider of science,
technology, engineering, and mathematics (STEM) education for K-12 students.
The PLTW Gateway program at Lincoln Middle School started in the 2010-2011 school year. The Technology
Education teacher, Mrs. Elizabeth Straczek, had been asked to look for a new curriculum to use in her classes. The old
curriculum consisted of 15 learning stations that students rotated through during their three years at the school.
Unfortunately, the materials were outdated, and many of the modules were not in working order. Elizabeth learned
about PLTW and made a proposal to the Board of Education, who approved the curriculum as a three-year pilot.
The middle school program, called PLTW Gateway, consists of two foundational units and six specialization
units. Lincoln Middle School offers the foundational units, Design & Modeling and Automation Robotics, to sixth and
seventh graders respectively; eighth grade students take Environmental Engineering as their specialization. The five
other specialization units PLTW Gateway offers include Green Architecture, Magic of Electrons, Flight and Space,
Science of Technology, and Medical Detectives. Elizabeth is certified to teach the three Lincoln Middle School
courses, and last year she completed requirements to be a PLTW Master Teacher. She is one of five Master Teachers
at the middle school level in the state of Illinois.
PLTW Gateway has been successful at Lincoln; however, this year the organization has introduced a new
component to the program. The curriculum has moved online, using the learning management system Canvas, with the
intention of creating a blended learning environment. Now that PLTW Gateway has proven to be successful even
beyond its initial three-year pilot, the Board of Education wants to evaluate the impact of Canvas LMS on the course.
PLTW Gateway is considered a Broad Experience course at Lincoln, which means it is on a quarterly rotation
with computer education, art, and music. This allows all 700 students to experience Canvas, and it also give Elizabeth
the opportunity to adjust her teaching methods when using the LMS with different groups. This broad scope will create
sufficient data to be considered at the end of the 2014-2015 school year.
The purpose of this formative evaluation is to determine the impact of Canvas on the PLTW Gateway learning
environment, focusing on teacher and student reactions, instructional applications, academic performance, and student
engagement statistics. Evaluation methods will include stakeholder analyses (administrators, students, teachers, and
parents), user data collection (attendance, academic performance, student engagement), and interviews with Elizabeth
and her students. These analyses will be used to predict the maximum benefit of the program. The results will provide
information to Lincoln Middle School and the Board of Education in the consideration of future LMS implementation
beyond the PLTW Gateway program. This evaluation will help to determine if Canvas will be a viable LMS system
and whether it enhances the effectiveness of the PLTW Gateway design.
Evaluation Need
PLTW Gateway has been successful at Lincoln; however, this year the organization has introduced a new
component to the program. The curriculum has moved online, using the learning management system Canvas, with the
intention of creating a blended learning environment. Now that PLTW Gateway has proven to be successful even
beyond its initial three-year pilot, the Board of Education wants to evaluate the impact of Canvas LMS on the course.
Evaluation Purpose
The purpose of this formative evaluation is to determine the impact of Canvas on the PLTW Gateway learning
environment, focusing on teacher and student reactions, instructional applications, academic performance, and student
engagement statistics. The results will provide information to Lincoln Middle School and the Board of Education in
the consideration of future LMS implementation beyond the PLTW Gateway program. This evaluation will help to
determine if Canvas will be a viable LMS system and whether it enhances the effectiveness of the PLTW Gateway
design.
Activities
Human: Board
of Education,
Principal,
PLTW teacher,
students, parents
Financial:
-PLTW Grant
funding for
Canvas LMS
and teacher
training
-Technology
budget for
classroom set of
Dell computers
-School district
budget for
teacher salary
Organizational:
-Classroom
-Desks and
chairs
-Laptop cart
with chargers
-Internet access
Community:
The PLTW
Canvas page
will be made
available
anywhere with
internet access
Discussion Boards
Embedded video
Hyperlinked
resources and online
activities
Announcements
Reflection journal
Mastery Badges
Online Quizzes
Download files
Interactive apps
Portfolio
Grades
Outputs
6th
grade projects on
design process,
measurement,
orthographic &
isometric sketching,
3D CAD modeling
7th
grade projects on
gears & gear ratio,
increasing speed &
torque, build
mechanisms,
programming
8th
grade projects on
types & forms of
energy, sustainable
energy, wind
turbine design, heat
transfer
Student
s will navigate
Canvas LMS
skillfully
Student
s will know where
to locate online
class resources
Outcomes
Student
data for making a
s will know how to
decision for future
submit assignments
LMS implementation
online
Impacts
LJHS
students
will go to
high school
better
prepared to
utilize an
LMS
Enrollment
in high
school
engineering
classes will
increase
LJHS will
become an
even more
notable
exemplar
as a PLTW
school
(currently
tied for
22nd
middle
school in
Illinois)
Individuals, groups,
or agencies needing
the evaluation
findings
School Board
LJHS Administration
1:1 Committee
PLTW Teacher
To provide
input to
evaluation
To react
Parents
For interest
Students
Technology Coach
Taxpayers
Lincoln Teachers
Stakeholder Prioritization
In terms of the weight of stakeholders, if we prioritize the value of each party based on the importance of the
Canvas evaluation for taking action on policy and operational decisions to meet long-term goals, the results will be
most valuable to Lincoln Junior High Schools administration and the School Board. While the PLTW teacher and
students are the most immediately impacted by the use of Canvas, evaluation findings will not significantly impact the
viability of the program since PLTW has already been successful for four years. If the results indicate improved
student performance, the administration will consider Canvas for possible implementation in other classes. The school
district has already formed a 1:1 committee to determine the need of digital learning tools for instruction at LJHS; the
evaluation results will provide significant data for their consideration. If the evaluation results are favorable and the
use of Canvas is recommended by the 1:1 committee, the Lincoln administration would then present the findings to the
School Board, who give the final vote of approval.
Our secondary stakeholder would be the students. Canvas needs to be evaluated on its merits as an LMS system
in consideration to its main consumer. While the school district may make the ultimate decision to implement this
system, it will be essential to determine whether students are able to learn and communicate effectively through this
software before moving forward. Its important to note that students will not have the power to decide whether the
system is used, but their opinions and outcomes will be vital in assessing Canvas and determining what functions were
most valuable in the PLTW classroom and to consider additional possibilities for future students.
The staff at Lincoln would come next, including the PLTW teacher, technology coach, and other classroom
teachers. If Canvas might be implemented school-wide, it will be important to have teacher buy-in and support for the
LMS. The staff will need to understand the positive impacts that blended learning can have on student growth, and so
it will be necessary to share the results with teachers and to consider their input. Furthermore, once teachers buy in to
using new technology in their classrooms, they can act as representatives within the community to encourage further
innovation.
Parents and taxpayers are valuable, but would be the final stakeholder group. Those whom take a vested
interest in decisions made by LJHS and the School Board would find formative data valuable in shaping opinions about
money and efforts spent on digital tools, such as learning management systems. Some may track closely the costs of
such technology initiatives in comparison to the instructional benefits. The majority of parents and taxpayers may
appreciate being informed along the way, but will not feel part of this process until their child comes home and logs
into an LMS on a school provided device.
EVALUATION DESIGN
The following Evaluation Purpose Statement describes the focus and anticipated outcomes of the evaluation:
The purpose of this formative evaluation is to determine the impact of Canvas on the PLTW Gateway learning
environment, focusing on teacher and student reactions, instructional applications, academic performance, and student
engagement statistics. The results will provide information to Lincoln Middle School and the Board of Education in the
consideration of future LMS implementation beyond the PLTW Gateway program.
Evaluation Questions
Evaluation Question
Rationale
Information Required
Stakeholder feedback
Information Source
Method
Survey
Sampling
Schedule
Analysis Procedures
Evaluation Question #2
Information Required
Achievement data
Information Source
Method
Pre-Post Tests
Sampling
Gather standard pre-test and post-test scores from the three years from
PLTW teacher, this years with an LMS and the last two years without
Collect pre-test and post-test scores from the previous two years from
the PLTW teacher by November 15, 2014
Collect pre-test and post-test scores from each quarter of the 2014-15
school year from the PLTW teacher within two weeks following each
quarter.
Assess scores and report findings by June 30, 2015
Schedule
Analysis Procedures
Appendix A1
Appendix A2
Appendix A3
Appendix A4
Appendix A5
Appendix A6
Appendix A7
Appendix A8
Appendix A9
Resources/People
A S O N D J F M A M J
u e c o e a e a p a u
g p t v c n b r r y n
t
Evaluation Team
Evaluation Team
Evaluation Team
1
/
2
Evaluation Team
Evaluation Team
?
%
Evaluation Team
Lead Evaluator
Evaluation Team
?
%
Evaluation Budget
Personnel
Evaluation Team: Board of Education member, Lincoln administrator, 1:1
Committee member, Lincoln reading teacher, Technology Coach
Evaluation Consultant
20 Hours at $35/hour
48 Hours
8 Hours
Data analysis
15 Hours at $50/hr.
Total Personnel Time, No additional cost 181 Hours
Total Personnel Cost $1450.00
Incentives $250.00
Direct Cost
Participant support cost (Interview)
$250.00
$150.00
$0.00
Total Direct Cost $400.00
Total amount of the request $1850.00
REFERENCES
Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2011). Program evaluation: Alternative approaches and practical
guidelines. Boston: Pearson.
Instructure. (n.d.). Learning management system | LMS | canvas by instructure. Retrieved from
http://www.instructure.com/
Project lead the way, inc. (2014). Project lead the way. Retrieved from https://www.pltw.org/
Robotics academy. (2014). Robotics academy. Retrieved from http://www.education.rec.ri.cmu.edu/
APPENDICES
A. Evaluation Measures
A1. PLTW Student Survey
Pseudocode
startMotor(LeftMotor, -63);
/*
startMotor(RightMotor, -63);
/*
turnLEDOn(RedLED);
//
wait(2);
//
turnLEDOff(RedLED);
//
stopMotor(RightMotor);
//
stopMotor(LeftMotor);
//
1.
2.
When your project is ready to be graded, turn this paper in AND WRITE DOWN YOUR ROBOT NUMBER HERE
__________
When your project is ready to be graded, turn this paper in AND WRITE DOWN YOUR ROBOT NUMBER HERE
__________
When your project is ready to be graded, turn this paper in AND WRITE DOWN YOUR ROBOT NUMBER HERE
__________
When your project is ready to be graded, turn this paper in AND WRITE DOWN YOUR ROBOT NUMBER HERE
__________
When your project is ready to be graded, turn this paper in AND WRITE DOWN YOUR ROBOT NUMBER HERE
__________
Data Analysis/
Scientific Method
Analytical Reflection
(3) Proficient - B
(2) Basic - C
Robot is built
accurately with no
mistakes, following
all directions
Robot is
programmed
accurately with no
errors, following all
directions
Journal entries
indicate reasonable
predictions based
on prior knowledge
All data is correctly
gathered and
recorded
Chart or graph is
accurately
constructed
illustrating results
Journal entries
indicate predictions
based on prior
knowledge
Most data is
correctly gathered
and recorded
Chart or graph
illustrating results
is constructed with
few errors
Reflection
includes:
A topic statement
accurately
presenting
information on the
mission
A detailed
explanation of how
the robot was
constructed and the
trial process
A concluding
statement
thoroughly
summarizing the
end result
Reflection
includes:
A topic statement
accurately
presenting
information on the
mission
An explanation of
how the robot was
constructed and the
trial process
A concluding
statement
summarizing
results
Journal entries
indicate predictions
are irrational and
not based on prior
knowledge
Some data is
correctly gathered
and recorded
Chart or graph
illustrating results
is constructed with
some errors
Reflection
includes:
A topic sentence
presenting basic
information on the
mission
Basic explanation
of the construction
and trial process
A concluding
statement
summarizing some
results
Journal entries do
not indicate
predictions
Data is not
gathered or
recorded correctly
Chart or graph is
inaccurate
Reflection
includes:
A topic statement
that does not
explain the mission
(or no topic
statement)
Very little or no
explanation of the
construction and
trial process
A concluding
statement that does
not summarize the
results (or no
concluding
statement)
B. Evaluator Credentials