What do we mean by collecting data?

What do we mean by analyzing data?

Why should you collect and analyze data for your evaluation?
When and by whom should data be collected and analyzed?
How do you collect and analyze data?

In previous sections of this chapter, weve discussed studying the issue, deciding on a
research design, and creating an observational system for gathering information for your
evaluation. Now its time to collect your data and analyze it figuring out what it means so
that you can use it to draw some conclusions about your work. In this section, well examine
how to do just that.

What do we mean by collecting data?

Essentially, collecting data means putting your design for collecting information into
operation. Youve decided how youre going to get information whether by direct
observation, interviews, surveys, experiments and testing, or other methods and now you
and/or other observers have to implement your plan. Theres a bit more to collecting data,
however. If you are conducting observations, for example, youll have to define what youre
observing and arrange to make observations at the right times, so you actually observe what
you need to. Youll have to record the observations in appropriate ways and organize them so
theyre optimally useful.
Recording and organizing data may take different forms, depending on the kind of
information youre collecting. The way you collect your data should relate to how youre
planning to analyze and use it. Regardless of what method you decide to use, recording
should be done concurrent with data collection if possible, or soon afterwards, so that nothing
gets lost and memory doesnt fade.
Some of the things you might do with the information you collect include:

Gathering together information from all sources and observations

Making photocopies of all recording forms, records, audio or video recordings, and
any other collected materials, to guard against loss, accidental erasure, or other
problems
Entering narratives, numbers, and other information into a computer program, where
they can be arranged and/or worked on in various ways
Performing any mathematical or similar operations needed to get quantitative
information ready for analysis. These might, for instance, include entering numerical
observations into a chart, table, or spreadsheet, or figuring the mean (average),
median (midpoint), and/or mode (most frequently occurring) of a set of numbers.
Transcribing (making an exact, word-for-word text version of) the contents of audio
or video recordings
Coding data (translating data, particularly qualitative data that isnt expressed in
numbers, into a form that allows it to be processed by a specific software program or
subjected to statistical analysis)
Organizing data in ways that make them easier to work with. How you do this will
depend on your research design and your evaluation questions. You might group
observations by the dependent variable (indicator of success) they relate to, by

individuals or groups of participants, by time, by activity, etc. You might also want to
group observations in several different ways, so that you can study interactions among
different variables.
There are two kinds of variables in research. An independent variable (the intervention) is a
condition implemented by the researcher or community to see if it will create change and
improvement. This could be a program, method, system, or other action. A dependent
variable is what may change as a result of the independent variable or intervention. A
dependent variable could be a behavior, outcome, or other condition. A smoking cessation
program, for example, is an independent variable that may change group members smoking
behavior, the primary dependent variable.

What do we mean by analyzing data?

Analyzing information involves examining it in ways that reveal the relationships, patterns,
trends, etc. that can be found within it. That may mean subjecting it to statistical operations
that can tell you not only what kinds of relationships seem to exist among variables, but also
to what level you can trust the answers youre getting. It may mean comparing your
information to that from other groups (a control or comparison group, statewide figures, etc.),
to help draw some conclusions from the data. The point, in terms of your evaluation, is to get
an accurate assessment in order to better understand your work and its effects on those youre
concerned with, or in order to better understand the overall situation.
There are two kinds of data youre apt to be working with, although not all evaluations will
necessarily include both. Quantitative data refer to the information that is collected as, or
can be translated into, numbers, which can then be displayed and analyzed
mathematically. Qualitative data are collected as descriptions, anecdotes, opinions, quotes,
interpretations, etc., and are generally either not able to be reduced to numbers, or are
considered more valuable or informative if left as narratives. As you might expect,
quantitative and qualitative information needs to be analyzed differently.

Quantitative data
Quantitative data are typically collected directly as numbers. Some examples include:

The frequency (rate, duration) of specific behaviors or conditions

Test scores (e.g., scores/levels of knowledge, skill, etc.)
Survey results (e.g., reported behavior, or outcomes to environmental conditions;
ratings of satisfaction, stress, etc.)
Numbers or percentages of people with certain characteristics in a population
(diagnosed with diabetes, unemployed, Spanish-speaking, under age 14, grade of
school completed, etc.)

Data can also be collected in forms other than numbers, and turned into quantitative data for
analysis. Researchers can count the number of times an event is documented in interviews or
records, for instance, or assign numbers to the levels of intensity of an observed event or
behavior. For instance, community initiatives often want to document the amount and
intensity of environmental changes they bring about the new programs and policies that
result from their efforts. Whether or not this kind of translation is necessary or useful depends

on the nature of what youre observing and on the kinds of questions your evaluation is meant
to answer.
Quantitative data is usually subjected to statistical procedures such as calculating the mean or
average number of times an event or behavior occurs (per day, month, year). These
operations, because numbers are hard data and not interpretation, can give definitive, or
nearly definitive, answers to different questions. Various kinds of quantitative analysis can
indicate changes in a dependent variable related to frequency, duration, timing (when
particular things happen), intensity, level, etc. They can allow you to compare those changes
to one another, to changes in another variable, or to changes in another population. They
might be able to tell you, at a particular degree of reliability, whether those changes are likely
to have been caused by your intervention or program, or by another factor, known or
unknown. And they can identify relationships among different variables, which may or may
not mean that one causes another.

Qualitative data
Unlike numbers or hard data, qualitative information tends to be soft, meaning it cant
always be reduced to something definite. That is in some ways a weakness, but its also a
strength. A number may tell you how well a student did on a test; the look on her face after
seeing her grade, however, may tell you even more about the effect of that result on her. That
look cant be translated to a number, nor can a teachers knowledge of that students history,
progress, and experience, all of which go into the teachers interpretation of that look. And
that interpretation may be far more valuable in helping that student succeed than knowing her
grade or numerical score on the test.
Qualitative data can sometimes be changed into numbers, usually by counting the number of
times specific things occur in the course of observations or interviews, or by assigning
numbers or ratings to dimensions (e.g., importance, satisfaction, ease of use).
The challenges of translating qualitative into quantitative data have to do with the human
factor. Even if most people agree on what 1 (lowest) or 5 (highest) means in regard to rating
satisfaction with a program, ratings of 2, 3, and 4 may be very different for different
people. Furthermore, the numbers say nothing about why people reported the way they
did. One may dislike the program because of the content, the facilitator, the time of day, etc.
The same may be true when youre counting instances of the mention of an event, such as the
onset of a new policy or program in a community based on interviews or archival
records. Where one person might see a change in program he considers important another
may omit it due to perceived unimportance.
Qualitative data can sometimes tell you things that quantitative data cant. It may reveal why
certain methods are working or not working, whether part of what youre doing conflicts with
participants culture, what participants see as important, etc. It may also show you patterns
in behavior, physical or social environment, or other factors that the numbers in your
quantitative data dont, and occasionally even identify variables that researchers werent
aware of.
It is often helpful to collect both quantitative and qualitative information.

Quantitative analysis is considered to be objective without any human bias attached to it

because it depends on the comparison of numbers according to mathematical
computations. Analysis of qualitative data is generally accomplished by methods more
subjective dependent on peoples opinions, knowledge, assumptions, and inferences (and
therefore biases) than that of quantitative data. The identification of patterns, the
interpretation of peoples statements or other communication, the spotting of trends all of
these can be influenced by the way the researcher sees the world. Be aware, however, that
quantitative analysis is influenced by a number of subjective factors as well. What the
researcher chooses to measure, the accuracy of the observations, and the way the research is
structured to ask only particular questions can all influence the results, as can the researchers
understanding and interpretation of the subsequent analyses.

Why should you collect and analyze data for your

evaluation?
Part of the answer here is that not every organization particularly small community-based
or non-governmental ones will necessarily have extensive resources to conduct a formal
evaluation. They may have to be content with less formal evaluations, which can still be
extremely helpful in providing direction for a program or intervention. An informal
evaluation will involve some data gathering and analysis. This data collection and
sensemaking is critical to an initiative and its future success, and has a number of advantages.

The data can show whether there was any significant change in the dependent
variable(s) you hoped to influence. Collecting and analyzing data helps you see
whether your intervention brought about the desired results

The term significance has a specific meaning when youre discussing statistics. The level
of significance of a statistical result is the level of confidence you can have in the answer you
get. Generally, researchers dont consider a result significant unless it shows at least a 95%
certainty that its correct (called the .05 level of significance, since theres a 5% chance that
its wrong). The level of significance is built into the statistical formulas: once you get a
mathematical result, a table (or the software youre using) will tell you the level of
significance.
Thus, if data analysis finds that the independent variable (the intervention) influenced the
dependent variable at the .05 level of significance, it means theres a 95% probability or
likelihood that your program or intervention had the desired effect. The .05 level is generally
considered a reasonable result, and the .01 level (99% probability) is considered about as
close to certainty as you are likely to get. A 95% level of certainty doesnt mean that the
program works on 95% of participants, or that it will work 95% of the time. It means that
theres only a 5% possibility that it isnt actually whats influencing the dependent variable(s)
and causing the changes that it seems to be associated with.

They can uncover factors that may be associated with changes in the dependent
variable(s). Data analyses may help discover unexpected influences; for instance, that
the effort was twice as large for those participants who also were a part of a support
group. This can be used to identify key aspects of implementation.
They can show connections between or among various factors that may have an
effect on the results of your evaluation. Some types of statistical procedures look

for connections (correlations is the research term) among variables. Certain

dependent variables may change when others do. These changes may be similar
i.e., both variables increase or decrease (e.g., as childrens proficiency at reading
increases, the amount of reading they do also increases). Or the opposite may be
observed i.e. the two variables change in opposite directions (as the amount of
exercise they engage in increases, peoples weight decreases). Correlations dont
mean that one variable causes another, or that they both have the same cause, but they
can provide valuable information about associations to expect in an evaluation.
They can help shed light on the reasons that your work was effective or, perhaps,
less effective than youd hoped. By combining quantitative and qualitative analysis,
you can often determine not only what worked or didnt, but why. The effect of
cultural issues, how well methods are used, the appropriateness of your approach for
the population these as well as other factors that influence success can be
highlighted by careful data collection and analysis. This knowledge gives you a basis
for adapting and changing what you do to make it more likely youll achieve the
desired outcomes in the future.
They can provide you with credible evidence to show stakeholders that your
program is successful, or that youve uncovered, and are addressing limitations.
Stakeholders, such as funders and community boards, want to know their investments
are well spent. Showing evidence of intermediate outcomes (e.g. new programs and
policies) and longer-term outcomes (e.g., improvements in education or health
indicators) is becoming increasingly important to receiving and retaining funding.
Their use shows that youre serious about evaluation and about improving your
work. Being a good trustee or steward of community investment includes regular
review of data regarding progress and improvement.
They can show the field what youre learning, and thus pave the way for others
to implement successful methods and approaches. In that way, youll be helping to
improve community efforts and, ultimately, quality of life for people who benefit.

When and by whom should data be collected and

analyzed?
As far as data collection goes, the when part of this question is relatively simple: data
collection should start no later than when you begin your work or before you begin in order
to establish a baseline or starting point and continue throughout. Ideally, you should collect
data for a period of time before you start your program or intervention in order to determine
if there are any trends in the data before the onset of the intervention. Additionally, in order
to gauge your programs longer-term effects, you should collect follow-up data for a period
of time following the conclusion of the program.
The timing of analysis can be looked at in at least two ways: One is that its best to analyze
your information when youve collected all of it, so you can look at it as a whole. The other is
that if you analyze it as you go along, youll be able to adjust your thinking about what
information you actually need, and to adjust your program to respond to the information
youre getting. Which of these approaches you take depends on your research purposes. If
youre more concerned with a summative evaluation finding out whether your approach
was effective, you might be more inclined toward the first. If youre oriented toward
improvement a formative evaluation we recommend gathering information along the way.

Both approaches are legitimate, but ongoing data collection and review can particularly lead
to improvements in your work.
The who question can be more complex. If youre reasonably familiar with statistics and
statistical procedures, and you have the resources in time, money, and personnel, its likely
that youll do a somewhat formal study, using standard statistical tests. (Theres a great deal
of software both for sale and free or open-source available to help you.)
If thats not the case, you have some choices:

You can hire or find a volunteer outside evaluator, such as from a nearby college or
university, to take care of data collection and/or analysis for you.
You can conduct a less formal evaluation. Your results may not be as sophisticated as
if you subjected them to rigorous statistical procedures, but they can still tell you a lot
about your program. Just the numbers the number of dropouts (and when most
dropped out), for instance, or the characteristics of the people you serve can give
you important and usable information.
You can try to learn enough about statistics and statistical software to conduct a
formal evaluation yourself. (Take a course, for example.)
You can collect the data and then send it off to someone a university program, a
friendly statistician or researcher, or someone you hire to process it for you.
You can collect and rely largely on qualitative data. Whether this is an option
depends to a large extent on what your program is about. You wouldnt want to
conduct a formal evaluation of effectiveness of a new medication using only
qualitative data, but you might be able to draw some reasonable conclusions about use
or compliance patterns from qualitative information.
If possible, use a randomized or closely matched control group for comparison. If
your control is properly structured, you can draw some fairly reliable conclusions
simply by comparing its results to those of your intervention group. Again, these
results wont be as reliable as if the comparison were made using statistical
procedures, but they can point you in the right direction. Its fairly easy to tell
whether or not theres a major difference between the numbers for the two or more
groups. If 95% of the students in your class passed the test, and only 60% of those in
a similar but uninstructed control group did, you can be pretty sure that your class
made a difference in some way, although you may not be able to tell exactly what it
was that mattered. By the same token, if 72% of your students passed and 70% of the
control group did as well, it seems pretty clear that your instruction had essentially no
effect, if the groups were starting from approximately the same place.

Who should actually collect and analyze data also depends on the form of your evaluation. If
youre doing a participatory evaluation, much of the data collection - and analyzing - will be
done by community members or program participants themselves. If youre conducting an
evaluation in which the observation is specialized, the data collectors may be staff members,
professionals, highly trained volunteers, or others with specific skills or training (graduate
students, for example). Analysis also could be accomplished by a participatory process. Even
where complicated statistical procedures are necessary, participants and/or community
members might be involved in sorting out what those results actually mean once the math is
done and the results are in. Another way analysis can be accomplished is by professionals or
other trained individuals, depending upon the nature of the data to be analyzed, the methods
of analysis, and the level of sophistication aimed at in the conclusions.

How do you collect and analyze data?

Whether your evaluation includes formal or informal research procedures, youll still have to
collect and analyze data, and there are some basic steps you can take to do so.

Implement your measurement system

We've previously discussed designing an observational system to gather information.
Now its time to put that system in place.

Clearly define and describe what measurements or observations are needed. The
definition and description should be clear enough to enable observers to agree on
what theyre observing and reliably record data in the same way.
Select and train observers. Particularly if this is part of a participatory process,
observers need training to know what to record; to recognize key behaviors, events,
and conditions; and to reach an acceptable level of inter-rater reliability (agreement
among observers).
Conduct observations at the appropriate times for the appropriate period of
time. This may include reviewing archival material; conducting interviews, surveys,
or focus groups; engaging in direct observation; etc.
Record data in the agreed-upon ways. These may include pencil and paper, computer
(using a laptop or handheld device in the field, entering numbers into a program, etc.),
audio or video, journals, etc.

Organize the data youve collected

How you do this depends on what youre planning to do with it, and on what youre
interested in.

Enter any necessary data into the computer. This may mean simply typing comments,
descriptions, etc., into a word processing program, or entering various kinds of
information (possibly including audio and video) into a database, spreadsheet, a GIS
(Geographic Information Systems) program, or some other type of software or file.
Transcribe any audio- or videotapes. This makes them easier to work with and copy,
and allows the opportunity to clarify any hard-to-understand passages of speech.
Score any tests and record the scores appropriately.
Sort your information in ways appropriate to your interest. This may include sorting
by category of observation, by event, by place, by individual, by group, by the time of
observation, or by a combination or some other standard.
When possible, necessary, and appropriate, transform qualitative into quantitative
data. This might involve, for example, counting the number of times specific issues
were mentioned in interviews, or how often certain behaviors were observed.

Conduct data graphing, visual inspection, statistical analysis, or other

operations on the data as appropriate
Weve referred several times to statistical procedures that you can apply to quantitative data.
If you have the right numbers, you can find out a great deal about whether your program is
causing or contributing to change and improvement, what that change is, whether there are

any expected or unexpected connections among variables, how your group compares to
another youre measuring, etc.
There are other excellent possibilities for analysis besides statistical procedures,
however. A few include:

Simple counting, graphing and visual inspection of frequency or rates of behavior,

events, etc., over time.
Using visual inspection of patterns over time to identify discontinuities (marked
increases, decreases) in the measures over time (sessions, weeks, months).
Calculating the mean (average), median (midpoint), and/or mode (most frequent) of a
series of measurements or observations. What was the average blood pressure, for
instance, of people who exercised 30 minutes a day at least five days a week, as
opposed to that of people who exercised two days a week or less?
Using qualitative interviews, conversations, and participant observation to observe
(and track changes in) the people or situation. Journals can be particularly revealing
in this area because they record peoples experiences and reflections over time.
Finding patterns in qualitative data. If many people refer to similar problems or
barriers, these may be important in understanding the issue, determining what works
or doesnt work and why, or more.
Comparing actual results to previously determined goals or benchmarks. One
measure of success might be meeting a goal for planning or program implementation,
for example.

Take note of any significant or interesting results

Depending on the nature of your research, results may be statistically significant (the 95% or
better certainty that we discussed earlier), or simply important or unusual. They may or may
not be socially significant (i.e., large enough to solve the problem).
There are a number of different kinds of results you might be looking for.

Differences within people or groups. If you have repeated measurements for

individuals/groups over time, we can see if there are marked increases/decreases in
the (frequency, rate) of behavior (events, etc.) following introduction of the program
or intervention. When the effects are seen when and only when the intervention is
introduced and if the intervention is staggered (delayed) across people or groups
this increases our confidence that the intervention, and not something else, is
producing the observed effects.
Differences between or among two or more groups. If you have one or more
randomized control groups in a formal study (groups that are drawn at random from
the same population as the group in your program, but are not getting the same
program or intervention, or are getting none at all), then the statistical significance of
differences between or among the groups should tell you whether your program has
any more influence on the dependent variable(s) than whats experienced by the other
groups.
Results that show statistically significant changes. With or without a control or
comparison group, many statistical procedures can tell you whether changes in
dependent variables are truly significant (or not likely due to chance). These results
may say nothing about the causes of the change (or they may, depending on how

youve structured your evaluation), but they do tell you whats happening, and give
you a place to start.
Correlations. Correlation means that there are connections between or among two or
more variables. Correlations can sometimes point to important relationships you
might not have predicted. Sometimes they can shed light on the issue itself, and
sometimes on the effects of a groups cultural practices. In some cases, they can
highlight potential causes of an issue or condition, and thus pave the way for future
interventions.

Correlation between variables doesnt tell you that one necessarily causes the other, but
simply that changes in one have a relationship to changes in the other. Among American
teenagers, for instance, there is probably a fairly high correlation between an increase in body
size and an understanding of algebra. This is not because one causes the other, but rather the
result of the fact that American schools tend to begin teaching algebra in the seventh, eighth,
or ninth grades, a time when many 12-, 13-, and 14-year-olds are naturally experiencing a
growth spurt.
On the other hand, correlations can reveal important connections. A very high correlation
between, for instance, the use of a particular medication and the onset of depression might
lead to the withdrawal of that medication, or at least a study of its side effects, and increased
awareness and caution among doctors who prescribe it. A very high correlation between
gang membership and having a parent with a substance abuse problem may not reveal a
direct cause-and-effect relationship, but may tell you something important about who is more
at risk for substance abuse.

Patterns. In both quantitative and qualitative information, patterns often emerge:

certain health conditions seem to cluster in particular geographical areas; people from
a particular group behave in similar ways; etc. These patterns may not be specifically
what you were looking for or expected to find, but they may either be important in
themselves or shed light on the areas youre interested in. In some cases, you may
need to subject them to statistical procedures (regression analysis, for example) to see
if, in fact, theyre random, or if they constitute actual patterns.
Obvious important findings. Whether as a result of statistical analysis, or of
examination of your data and application of logic, some findings may stand out. If
70% of a group of overweight participants in a healthy eating and physical activity
program lowered their weight and blood pressure significantly, compared to only 20%
of a similar group not in the program, you can probably assume that program may
have been effective. If theres no change whatsoever in education outcomes after two
years of your education program, then youre either running an ineffective program,
or youre simply not reaching those who are most likely to have poorer outcomes
(which can also be interpreted to mean youre running an ineffective program.)

Not all important findings will necessarily tell you whether your program worked, or what is
the most effective method. It might be obvious from your data collection, for instance, that,
while violence or roadway injuries may not be seen as a problem citywide, they are much
higher in one or more particular areas, or that the rates of diabetes are markedly higher for
particular groups or those living in areas with greater disparities of income. If you have the
resources, its wise to look at the results of your research in a number of different ways, both
to find out how to improve your program, and to learn what else you might do to affect the
issue.

Interpret the results

Once youve organized your results and run them through whatever statistical or other
analysis youve planned for, its time to figure out what they mean for your
evaluation. Probably the most common question that evaluation research is directed toward is
whether the program being evaluated works or makes a difference. In research terms, that
often translates to What were the effects of the independent variable (the program,
intervention, etc.) on the dependent variable(s) (the behavior, conditions, or other factors it
was meant to change)? There are a number of possible answers to this question:

Your program had exactly the effects on the dependent variable(s) you expected and
hoped it would. Statistics or other analysis showed clear positive effects at a high
level of significance for the people in your program and if you used a multiplegroup design none, or far fewer, of the same effects for a similar control group
and/or for a group that received a different intervention with the same purpose. Your
early childhood education program, for instance, greatly increased development
outcomes for children in the community, and also contributed to an increase in the
percentage of children succeeding in school.
Your program had no effect. Your program produced no significant results on the
dependent variable, whether alone or compared to other groups. This would mean no
change as a result of your program or intervention.
Your program had a negative effect. For instance, intimate partner violence increased
(or at least appeared to) as a result of your intervention. (It is relatively common for
reported events, such as violence or injury, to increase when the intervention results in
improved surveillance and ease of reporting).
Your program had the effects you hoped for and other effects as well.
o These effects might be positive. Your youth violence prevention program, for
instance, might have resulted in greatly reduced violence among teens, and
might also have resulted in significantly improved academic performance for
the kids involved.
o These effects might be neutral. The same youth violence prevention program
might somehow result in youth watching TV more often after school.
o These effects might be negative. (These effects are usually called unintended
consequences.) Youth violence might decrease significantly, but the incidence
of teen pregnancies or alcohol consumption among youth in the program
might increase significantly at the same time.
o These effects might be multiple, or mixed.For instance, a program to reduce
HIV/AIDS might lower rates of unprotected sex but might also increase
conflict and instances of partner violence. Your program had no effect or a
negative effect and other effects as well. As with programs with positive
effects, these might be positive, neutral, or negative; single or multiple; or
consistent or mixed.

If your analysis gives you a clear indication that what youre doing is accomplishing your
purposes, interpretation is relatively simple: You should keep doing it, while trying out ways
to make it even more effective, or while aiming at other related issues as well.
As we discuss elsewhere in the Community Tool Box, good programs are dynamic -constantly striving to improve, rather than assuming that what theyre doing is as good as it
can be.

If your analysis shows that your program is ineffective or negative, however or, for that
matter, if a positive analysis leaves you wondering how to make your successful efforts still
more successful interpretation becomes more complex. Are you using an absolutely wrong
approach? Are you using an approach that could be effective, but is poorly implement? Is
there a particular contributing factor youre failing to take into account? Are there barriers to
success of culture, experience, personal characteristics, systematic discrimination present
in the population from which participants are drawn? Are there particular components or
elements you can change to make your program more effective, or should you start again
from scratch? What should you address to make a good program better?
Careful and insightful interpretation of your data may allow you to answer questions like
these. You may be able to use correlations, for instance, to generate hypotheses about your
results. If positive or negative changes in particular variables are consistently associated with
positive or negative changes in other variables, the two may be connected. (The word may
is important here. The two may be connected, but they may not, or both may be related to a
third variable that youre not aware of or that you consider trivial.) Such a connection can
point the way toward a factor (e.g., access to support) that is causing the changes in both
variables, and that must be addressed to make your program successful. Correlations may
also indicate patterns in your data, or may lead to an unexpected way of looking at the issue
youre addressing.
You can often use qualitative data to understand the meaning of an intervention, and peoples
reactions to the results.The observation that participants are continually suffering from a
variety of health problems may be traced, through qualitative data, to nutrition problems (due
either to poverty or ignorance) or to lack of access to health services, or to cultural
restrictions (some Muslim women may be unwilling or unable because of family
prohibition to accept care and treatment from male doctors, for example).
Once you have organized your data, both statistical results and anything that cant be
analyzed statistically need to be analyzed logically. This may not give you convincing
information but it will almost undoubtedly give you some ideas to follow up on, and some
indications of connections and avenues you might not yet have considered. It will also show
you some additional results people reacting differently than before to the program, for
example. The numbers can tell you whether there is change, but they cant always tell you
what causes it or why (although they sometimes can), or why some people benefit while
others dont. Those are often matters for logical analysis, or critical thinking.
Analyzing and interpreting the data youve collected brings you, in a sense, back to the
beginning. You can use the information youve gained to adjust and improve your program or
intervention, evaluate it again, and use that information to adjust and improve it further, for as
long as it runs. You have to keep up the process to ensure that youre doing the best work you
can and encouraging changes in individuals, systems, and policies that make for a better and
healthier community.
You have to become a cultural detective to understand your initiative, and, in some ways,
every evaluation is an anthropological study.

In Summary

The heart of evaluation research is gathering information about the program or intervention
youre evaluating and analyzing it to determine what it tells you about the effectiveness of
what youre doing, as well as about how you can maintain and improve that effectiveness.
Collecting quantitative data information expressed in numbers and subjecting it to a
visual inspection or formal statistical analysis can tell you whether your work is having the
desired effect, and may be able to tell you why or why not as well. It can also highlight
connections (correlations) among variables, and call attention to factors you may not have
considered.
Collecting and analyzing qualitative data interviews, descriptions of environmental factors,
or events, and circumstances can provide insight into how participants experience the issue
youre addressing, what barriers and advantages they experience, and what you might change
or add to improve what you do.
Once youve gained the knowledge that your information provides, its time to start the
process again. Use what youve learned to continue to evaluate what you do by collecting and
analyzing data, and continually improve your program.