Você está na página 1de 4

How to measure

training effectiveness
By Roger Grice CEO Newsletter Date: September 2010
Special points of
interest:
Goal-based evaluation
Goal-based evaluation

Kirkpatrick’s four steps Each year globally many mil- aim was to determine the ef- conducted by the National
to measuring training lions of dollars are spent on fectiveness of the training by Research Council showed that
effectiveness training. It would seem reason- scientific means on the level of human behavior is indeed diffi-
able to establish just how performance of the individual. cult to quantify exactly. Never-
Input process and out- effective this training has been Although there have been theless the greater attention
put models if only to help justify the some critics of this methodol- that was given to training
budget. Evaluation techniques ogy it has proved to be valu- methodology and in particular
Flowchart of training are not new, indeed the most able in that organizations have the effectiveness of manage-
evaluation influential early work on been able to quantify the rela- ment development during the
evaluation was performed in tionship between their corpo- 1960s and 1970s has led to
the USA during the 1940s by rate objectives (often ex- much better understanding and
Tyler. Probably the most im- pressed as key performance the emergence of systems
portant aspect of Ralph W indicators (KPIs) today) and evaluation methodology. As we
Tyler’s work was the realiza- the training aims and objec- will see this approach can be
tion of the importance of ob- tives. The main criticism of the divided into two main schools;
jectives in designing an effec- scientific/experimental ap- the narrow focused approach
tive school curriculum. This proach is that the methodology being attributed to Donald L
Inside this issue: early methodology can be de- is unable to take into account Kirkpatrick in the early 1960s
fined as the scientific/ the unexpected or uninten- and the somewhat broader
experimental approach. The tional learning that can take model advocated by P.Warr et
model emphasized the impor- place within a dynamic and al (The Evaluation of Manage-
tance of knowing the educa- changing learning environment. ment Training) known as the
Cost Effective 2 tional standards of the individu- Those readers who can re- CIRO model. The CIRO model
Training als in the control group before member reading about the looked at context evaluation,
the new education initiative original, and now famous Haw- input evaluation, reaction
took place and then determin- thorne experiments to deter- evaluation and outcome evalua-
Evidence or 3 ing the change that had taken mine the effects of illumination tion.
proof place by measuring the differ- levels on production held at
ence in attainment after the the Western Electric plant in
educational intervention was Cicero, Illinois in the 1920’s
The IPO Model 3 completed. In other words the and the later experiments also

IPO Flowchart 3
Donald L Kirkpatrick’s “Four steps to measuring training effectiveness”
This model uses four separate reaction of the trainee to the trainee has actually learnt any-
stages for the evaluation of the training. This sort of measure- thing. What really seems to be
About Sentrico 4 effectiveness of a training pro- ment is concerned with how being asked of the trainee was
and Roger Grice gram. The four stages are the trainees “feel” about the how “happy” were you with
course. The usual course feed- the course; hence the some-
Reaction back sheets are an example of what derogatory description
Conclusion 4 the Kirkpatrick level 1 that is often applied to the
Learning
evaluation. Most organizations “happy sheets”!
Behavior do not do any more than this
type of measurement and We will now look at the next
analysis. The drawback is that level of evaluation concerned
Results
we do not really know if the with Learning.
The first stage is about the
Page 2 How to measure training effectiveness

“Four steps to measuring training effectiveness” continued


Level 2. we have to make the meas- Level 3
urement and consider the
Things can be improved by learning at a certain time? This is concerned with
using a pre-test and post-test Well because we do not “behavior”. By that we mean
and comparing the results. know if the learning has had the measurable change in an
The model sug- The questions need to be time to be internalized and individual as a result of their
objective and closely related attendance on the training
gested by to the course objectives
become “concrete”. All too
course. This is, in my opinion,
Kirkpatrick is often trainees (and delegates
(more about that later). In this at conferences for example) the least we should be expect-
one of the most way we can determine if the
will have difficulty remember-
ing from any training program.
popular and can training actually delivered ing what was in the course or After all what is the point of
knowledge and this was un- spending money and using
be a useful ap- derstood by the trainees at
seminar they attended a few
resources if the training does
proach to evalu- days later let alone months
the time. An organization that not effect some measurable
later. There are various tech-
ating the effec- does this can be confident that niques that we can employ to change in the behavior of the
tiveness of train- the trainee has actually learnt
improve the level of recall but trainee?
ing. something at that time. basically “if we don’t use it we
Why do I make the point that will lose it”!

Level 4 Evaluation
“These are Kirkpatrick is now concerned ture in the companies balance The reason is that most do
metrics which with the training to determine sheets and KPIs. not have in place any system
really have an if it has actually been trans- for measuring the improve-
lated into tangible benefits to ment in competence of the
impact on the the organization. Quite simply individual; let alone a system-
We have to ask ourselves is
“bottom line” has productivity and or quality this not the real reason for atic approach to identifying
and for that been improved? Have the training? Training has to make the most effective means of
number of accidents or inci- a real difference in perform- assisting the individual to be-
reason feature dents been reduced? Has plant ance and effectiveness; this is coming competent.
in the availability and or plant utiliza- tied closely to competence.
companies tion been improved? Has the Training has be proven to
morale of the workforce deliver results and be cost
balance sheets changed for the better? These effective.
and KPIs”. are metrics which really have
an impact on the “bottom It might sound simple but it is
line” and for that reason fea- not for most organizations.

Cost effective training


If we all now accept that the has been methodical we will to the individual at that time in
have at our disposal a data- their development and truly
majority of organizations carry
base of learning objectives. I meets both their needs and
out training to change the
behavior of the workforce and call these TCOs or terminal those of the organization.
competence objectives. These
improve their performance in If we use these TCOs to de-
the workplace we also must TCOs must relate to the
needs of the organization and velop the training program
accept that this cannot be (whether it is a classroom
achieved without a clear plan will contain the
“underpinning” knowledge based course or a workshop
Eliminating waste by and strategy. This rational activity) we will have a course
approach to training and de- required to understand the
ensuring the training tasks or duties the employee “tailored” to the needs of the
velopment is best demon- organization and furthermore
carried out delivers must perform as part of his/
strated by the Competence we will be able to measure the
what is required by her duties as well as the prac-
Profile (s) or Competence improvement in performance
the organization is tical skills necessary. In this
essential for a sound Map (s) for a particular job (s) and contribution to the team’s
in the organization (s). If the way we are able to ensure
business case. that the training is appropriate KPIs.
approach in constructing these
By Roger Grice CEO Page 3

Evidence or Proof ?
A goal can be considered to forming work related duties Lower insurance premiums
be something that the organi- (like driving a truck for exam-
zation strives to achieve ple). If a company with over a Improved morale
through the meeting of spe- 1,000 employees for example
cific objectives. Achieving the achieves a figure of say Enhanced reputation
individual objectives in a me- 6,000,000 man hours without
Reduced stress
thodical and logical way effec- a LTI then clearly their HSE
tively maps out the process systems must be working. This No legal costs
towards the eventual achieve- can be translated into real
company benefits and major No compensation cases The proof of the effective-
ment of the goal. ness of a training interven-
savings to the organization.
If we look at an important KPI Lower staff turnover tion can be overwhelm-
There are many tangible bene-
for any organization it might fits and other benefits which ingly shown by hard facts
be expressed as “zero lost Employer of choice and figures, as well as
are very important but more
time injuries”; in other words other sources of evidence
no employee being away from difficult to quantify. obtained by staff inter-
work for more than 3 days Benefits can include:- views and focus groups.
because of an injury sustained
while in the company or per- Better productivity

Input, Process and Output


I have presented an argument to adopt new work systems (benefits which are longer
for mapping out the compe- and processes. term but actually determine “ The ultimate
tences required to perform a
Some leading companies rec-
the availability of future train- payoff or
job and to linking these com- ing resources).
petences to the delivery of ognized this change model and added value of
one (IBM) introduced the IPO The following quote is particu- an employee’s
training interventions. In effect
paradigm. The concern being larly relevant here
this means that the strategic
is training delivering the de- learning
goals of the organization must “ The ultimate payoff or added
be cascaded down to team sired results within the organi- experience is
zation? The IPO model was value of an employee’s learn-
level within the organizations. ing experience is how well he how well he or
In time new competences will designed to confront and
solve problem.
or she performs on the job.” she performs
be required for team mem-
bers as the industry “drivers” In this model a distinction is
David S Bushnell writing about on the job.”
the “Input, Process, Output: A
like new technology and com- made between output (short model for Evaluating Training”
petition encourages or per- term benefits) and outcomes 1990 Training and Development
haps forces the organization Journal

Flow chart of input-output approach to training evaluation

Trainee reac- Profitability


Trainee quali- Value adding activities tions improved
fications/ Customer
experience/ Knowledge and satisfaction
current com- Plan Design Develop Deliver skills gained increased
petence Productivity
Instructor Improved job enhanced There are several factors
abilities performance Reduced acci-
Training ma- dents and
we have to consider when
terials
Subject matter experts Coaching /mentoring
incidents planning a training inter-
Resources vention, but the most
Competence Management System Inputs important is how it will
make appositive impact on
business performance.
Expenditure on training
Roger Grice Associates Ltd
Windsor House
107 Talbot Road,
Rhondda Cynon Taff,
CF72AE
United Kingdom.

AboutRoger Grice Associates Ltd and


Sentrico ™ Competency Management System.

Phone: 44(0)1443 228612 Roger Grice is the CEO of a Consultancy that spe-
E-mail: cialises in business solutions for medium and large
post@rogergrice.co.uk enterprises, particularly in the energy, oil and gas
sector. With our team of highly qualified and experi-
enced consultants we are able to assist our clients in
organizational development and business improve-
ment through the development of their own people.
We believe passionately in what we do and strive to
provide a service which is truly “world class”. Major
projects that we have been involved in have received
international recognition for their novel approach
and the sustainable development of the local work-
Please visit our website force to competence standards recognised through-
www.rogergrice.co.uk out the world.

Roger has over twenty years experience in the oil and gas
business and is a “world class” training expert with an MSc Eur Ing Roger Grice MSc BA
in Training and Performance Management . (Hons) C.Eng. MIEE. FCIPD.

Conclusion
We have seen that evaluation part of the company database needs of the organization by
of training interventions such and accessible through its satisfying the competence
as conventional courses are Competence Management development requirements of
generally a neglected area as System. the individual. By achieving this
regards evaluation of their goal we have a rational and
effectiveness. In general the The training intervention has justifiable case for training and
“happy sheet” type of evalua- to be designed to meet the we are able to prove that
tion is ineffective as it is in- training really does deliver
clined to be subjective, and tangible benefits, not least to
influenced by considerations the “bottom line”.
which are actually outside the
In this global marketplace it is
course itself. That is the
course content could be poor essential to maintain a com-
petitive advantage through the
or inappropriate but was de-
livered by a good instructor workforce; this is best
achieved by an effective Per-
who managed the group well
and they actually “enjoyed” We should avoid training formance Management system
the experience but learnt evaluation which is only based underpinned by a robust
on subjective and often flawed Competence Management
little!
data. Instead we should be using System such as Sentrico™.
Instead I have suggested that metrics which enable us to
we owe it to the organization measure improvements in per-
to identify the competences formance that improve the
that are really required by the organizations competitiveness
individual to do his or her job and profitability.
well. These competences are
the skills and underpinning
knowledge which should be

Você também pode gostar