Running Head: KidSmart Evaluation Proposal

KidSmart Evaluation Proposal

Heather Cover, Carlie Houchen, Chelsea Whitfield
HEA 625- Program Evaluation
December 2015

KidSmart Evaluation Proposal

SECTION 1: CONCEPTUALIZATION OF EVALUATION

Evaluation Purpose
The purpose of this formative evaluation is aimed at improving the KidSmart program design
and performance. Formative evaluations allow for the development of a better understanding of
the process of change, finding out what works, what doesnt, and why. It should be viewed as a
valuable investment that improves the likelihood of achieving a successful outcome through
better program design.
Broad Questions
1. Are there evidence based strategies/methods identified in the curriculum?
2. According to the KidSmart facilitators, what parts of the curriculum are useful, fit and
appropriate? Why or why not?
3. To what degree do teachers in participating Forsyth County schools refer program to
others and are they satisfied with length of program?
4. How can the program be represented in a logic model?
5. What additional needs does the client/program have?
Key Stakeholders
Novant Health Forsyth Medical Center; Community Wellness and Education Department
program manager, coordinator, and facilitators; participating Forsyth County Schools; school
teachers; students; and parents.
Assumptions

KidSmart Evaluation Proposal

The existing implementation of the KidSmart program operates under several assumptions. First
of all, the evaluation will assume that all students will be present for all four sessions. This will
not always be possible as there is no way to ensure perfect attendance or that students will still be
at the same school for the final session that occurs several months after the initial three.
Secondly, it is assumed that parents are willing, able, and interested in being involved with their
childs education and health. In a process evaluation, it would be possible to consider different
implementation strategies to encourage parent involvement, but that is not the purpose here.
Thirdly, this curriculum assumes that nutrition and physical activity is not being adequately
taught in schools through the standard curriculum.
Major Contextual Factors
There are several important contextual factors to consider in this evaluation. The various
demographics in the schools throughout Forsyth County are important to consider. This can
include but is not limited to socioeconomic status, social interactions/norms, race/ethnicity, and
cultural background. The willingness of teachers, students, and parents to be involved in the
program could additionally alter outcomes. Specifically, the willingness of teachers to participate
could be influenced by their past experience with KidSmart, information they have heard from
other teachers about the program, or their own personal belief in its value. Student participation
may be shaped by the program delivery, the perceived relevance of the content, and support from
teachers, parents and peers. Willingness of parents to participate could hinder on their decision to
actively pursue healthy behaviors, their perceptions about a healthy weight, their self-efficacy in
their ability to confidently execute these behaviors, their employment status, and/or the number

KidSmart Evaluation Proposal

of additional children in their home. Lastly, the participation of multiple Forsyth County Schools
is another contextual factor that could affect the expansion of the program.
SECTION 2: PROGRAM DESCRIPTION
Program Purpose
KidSmart is a branded trademark interactive healthy lifestyle program developed by Novant
Health Forsyth Medical Center to increase knowledge and awareness about healthy eating and
physical activity among children and adolescents. The program promotes and increases the
understanding of practices regarding healthy food choices and encourages families to take an
active role in their health. Furthermore, the program provides opportunities for children to
practice fun and physical activities such as jumping jacks, sit ups, and active play. Parents are
engaged through take-home newsletters and a parent-specific web portal to support the efforts of
the health behaviors learned during the program. Broadly speaking, the purpose of this program
is to instill healthy lifestyle habits in school-age children and their families in a way that is
positive and fun with the hope of decreasing prevalence of childhood obesity.
Program Goals
Primarily, the goal of the program is to reduce and prevent the onset of youth adopting behaviors
that can contribute to them becoming an obese adult. The KidSmart program is working to start
the change of small improvements in a childs lifestyle that will ultimately alter intentions about
healthy behaviors, introduce personal goals for the future, and generate a lifetime of health
benefits. This program is partnering with schools in Forsyth County, North Carolina to assist in

KidSmart Evaluation Proposal

teaching students how to prevent or reduce the onset of childhood disease through focusing on
awareness of choices about what they are eating and how to get their bodies moving.
Objectives
Short term objectives include providing nutrition education to promote and encourage healthy
food choices while also increasing knowledge about physical activity that improves childrens
overall health and wellness. Long term objectives are to increase nutritional and physical activity
behavioral intentions; encourage parent engagement in the program to promote the adaptation of
health behaviors in the home; and improve decision-making skills.
Target Audience
The target audience is youth students in kindergarten to sixth grade. While children are the
program participants, parents are an important secondary audience.
Program Location
The program is delivered in Forsyth County school systems via three different program types: an
assembly style, classroom style, and an afterschool/summer program. The assembly style
program is conducted during school hours with multiple classes coming together to learn about
KidSmart. Classroom style is delivered by a facilitator teaching a KidSmart session during a
class period, typically during health or physical education classes. The afterschool and summer
KidSmart sessions are delivered outside of traditional school hours, either in the afternoon when
school has been dismissed or during the summer months. The initial three classes are provided
during the first half of the school year (August to January) with a final session on what I

KidSmart Evaluation Proposal

learned in the second half of the school year typically in March. Each KidSmart session is
designed to deliver:

Interactive nutrition and healthy lifestyle lessons:

Class 1- MyPlate, Dietary Guidelines

Class 2- Food labels, Serving Sizes

Class 3- Food Consumption and Physical Activity

Class 4- KidSmart Final Project

A KidSmart curriculum including lesson objectives and activities

Hands on home-based activities and family access to www.KidSmartkid.org

Monthly KidSmart newsletter for parents

Previous Evaluations
This program has never been evaluated and the program manager and coordinator would like to
see the program thriving not just surviving like it has been for the last seven years.
Logic Model Description
As listed in the logic model found in appendix A after the references of this paper, the first goal
is to educate youth about healthy eating and physical activity. Before this objective can be
implemented, schools in Forsyth County that are willing to participate in the program are

KidSmart Evaluation Proposal

recruited. Students in these participating schools then complete three consecutive classes in the
fall to learn about healthier eating practices and physical activity. A fourth session is delivered in
the latter half of the school year to reiterate what the students have previously learned. The result
of this goal is that it increases positive attitudes around the topic of healthy food consumption
and exercise. Evidence of the activities performed can be found in the lesson plan and handout
materials that are used. Results of program outcomes are then evaluated through reviewing the
curriculum content on healthy eating and physical activity. The second goal of the program is to
improve decision-making skills among the participants through activities performed in the four
classes. The result of these activities help youth gain knowledge and empowerment to make
informed decisions regarding their health. Lesson plans created to bring about this specific skill
help to provide evidence to monitor this goal. Youth will report their confidence with how to
make sound choices in a survey to help establish outcomes. The last goal described in the model
is the adaptation of health behaviors in the home. The activities for this goal include sending
homework materials home with children to involve their parents. As a result of this activity,
children and parents are able to learn something new together. Additionally, this enhances
parents ability to support healthier behaviors and aids in the practice of eating smarter and
moving more at home. Homework materials are turned back into facilitators to be utilized as
forms of evidence for this activity. To further the evaluation portion of this goal, content of the
homework material will be reviewed.
SECTION 3: EVALUATION PLAN
Evaluation Questions
1.

Are there evidence based strategies/methods identified in the curriculum?

KidSmart Evaluation Proposal

a. In what way is the curriculum content evidence based?

b. In what way is the delivery platform appropriate?
c. In what way is the content age appropriate?
a. According to the KidSmart facilitators, what parts of the curriculum are useful, fit and
appropriate? Why or why not?
a. How does the session flow affect the intended audiences ability to learn the
subject lesson?
b. In what ways is the amount of instructional content available adequate to
complete the activities?
c. In what ways is instruction based on the curriculum?
d. In what ways is it being implemented the way it was intended?
e. In what ways do barriers affect implementation?
f. In what ways is KidSmart culturally appropriate, competent/sensitive, and at the
accurate reading level to target audience?
g. In what ways is parent involvement included and measured?
h. In what ways could the program be improved?
a. To what degree do teachers in participating Forsyth County schools refer program to others
and are they satisfied with length of program?
a. How does the session flow affect the intended audiences ability to learn the subject
lesson?
b. How does the length of the program affect satisfaction with the program?

KidSmart Evaluation Proposal

c. In what way is the content age appropriate?

d. In what ways are students engaged with program activities and facilitators?
e. In what ways could the program be improved?
f. How does the delivery of the program affect your desire to recommend program to
other teachers or schools?
g. How do the facilitators of the program affect your desire to recommend program to
other teachers or schools?
h. In what ways are other concerns affecting your desire to recommend the program to
other teachers and schools?
a. How can the program be represented in a logic model?
b. What additional needs does the client/program have?
Experimental Design
This evaluation is a non-experimental design because the evaluation is not trying to generalize or
determine validity. The evaluation will consist of gathering and analyzing information on the
feasibility of the program. This will include demonstrating a need for the program and evidence
to justify program components.
Identify Measureable Variables
The measures in this evaluation will be largely qualitative. Reviews of existing literature will
seek to justify curriculum components. Interviews and questionnaires with key stakeholders will
provide information on the need for the program and the appropriateness for the target audience.

KidSmart Evaluation Proposal

10

Data Instruments/Tools
Methods:

Instruments:

1. Interviews

Develop interview questions/ interview protocol

2. Observation

Observation protocol

3. Survey

Develop questionnaires/ questionnaire protocol

Data Collection Plan

To support the steps of the evaluation, data collection through the use of interview questions,
observation protocol and questionnaire development will be implemented. It is anticipated that
these instruments will capture a multitude of perspectives and information to generate a complete
picture of the KidSmart program curriculum and to determine areas of improvement. The
purpose of the data collection plan is to gather insight from those involved in KidSmart. First
interview questions will be developed based on inquiry from stakeholders and then expanded to
retrieve more in-depth understanding. Once complete, voice recorded interviews will be
conducted by the evaluator with all respected program personnel roles as an effective technique
for providing information about KidSmart that may not be addressed by secondary data such as
health statistics. A standardized observation protocol for every class session will be developed by
the evaluator. Facilitators will know prior to the observation of when these observations will
occur. By using observation as a data collection method, information can be gathering by
watching behavior, events, or noting physical characteristics of facilitators and participants in
their natural setting (Centers of Disease Control and Prevention, 2008). However, the actual
observations will not be able to take place until the fall of 2016. Lastly, the questionnaire

KidSmart Evaluation Proposal

11

protocol to find out if participating past and present Forsyth County School teachers would refer
KidSmart to others and if they are satisfied with length of program will be developed with the
collaboration of program personnel. Since they already have a relationship with the teachers in
the participating schools in which they serve, working together to develop this survey tool to
address concerns about the program will be most effective. Program personnel may also have
further insight on if teachers would prefer an electronic version such as that from SurveyMonkey
or a paper copy. Once this survey is developed and after the last session of KidSmart, the
evaluator will disseminate them to participating Forsyth County School teachers to capture a
sample of the population. Upon collection the data will be analyzed.
Data Analysis
Descriptive statistics will be run on survey and other quantitative data. A thematic analysis will
be done for the interview and observational data. This will then be organized into a concept map.
Evaluation Crosswalk
Please refer to appendix B.
SECTION 4: REPORTING PLAN
The data collected will be analyzed and condensed into a final report that will be delivered at the
end of April 2016. This report will consist of condensed and summarized findings from
interviews and surveys. Additionally, the report will include information gathered on evidencebased programs and initiatives with similar goals. Data included in the one page brief will be an
analysis of patterns and common concerns from teachers and facilitators. The findings and

KidSmart Evaluation Proposal

12

results of the report will be presented to all program personnel. This personnel includes the
program manager, coordinator, and both facilitators. During the presentation a one-page brief of
the report will be provided in print to ensure that all program personnel have knowledge of the
evaluation data analysis. This meeting will be held to review recommendations to improve and
strengthen the KidSmart program and to answer any questions that program personnel may have
pertaining to the presented information. There will be no formal publications from this
evaluation, as it is the first phase of several evaluations to come.
SECTION 5: DETAILED BUDGET
Please refer to appendix C.
SECTION 6: DETAILED TIMELINE
The various activities being performed during the evaluation will begin and end according to the
Gantt Chart found in appendix D. This timeline helps to schedule sufficient time for critical
components of the internship that need to be researched, collected and summarized for the final
data analysis and reporting at the end of the KidSmart evaluation. This organized technique to
manage a project is read from the left starting with the sequencing of activities and then along
the top to show the duration of time for what has to be done and when (Gantt.com, 2015). The
evaluator is responsible for all activities described and the majority of time spent will be during
implementation.

KidSmart Evaluation Proposal

13

References
Centers of Disease Control and Prevention. United States Department of Health and Human
Services. (2008, December). Data collection methods for program evaluation:
observation. Retrieved from http://www.cdc.gov/healthyyouth/evaluation/pdf/brief16.pdf
Gantt.com. (2015). What is a Gantt chart?. Retrieved from http://www.gantt.com/

KidSmart Evaluation Proposal

Appendix A: Logic Model

14

KidSmart Evaluation Proposal

Appendix B: Evaluation Crosswalk

15

KidSmart Evaluation Proposal

Appendix C: Detailed Budget

16

KidSmart Evaluation Proposal

Appendix D: Gantt Chart

17

KidSmart Evaluation Proposal

18