This document describes Kirkpatrick's four levels of evaluating training effectiveness: reaction, learning, behavior, and results. It provides examples of evaluation tools and methods for each level, and discusses the relevance and practicality of evaluating at each level. While all levels can be measured, levels 3 and 4 relating to behavior and results are more challenging but provide more meaningful data on training impact. The document recommends starting by measuring easier levels before progressing to more complex behavioral assessments.
This document describes Kirkpatrick's four levels of evaluating training effectiveness: reaction, learning, behavior, and results. It provides examples of evaluation tools and methods for each level, and discusses the relevance and practicality of evaluating at each level. While all levels can be measured, levels 3 and 4 relating to behavior and results are more challenging but provide more meaningful data on training impact. The document recommends starting by measuring easier levels before progressing to more complex behavioral assessments.
This document describes Kirkpatrick's four levels of evaluating training effectiveness: reaction, learning, behavior, and results. It provides examples of evaluation tools and methods for each level, and discusses the relevance and practicality of evaluating at each level. While all levels can be measured, levels 3 and 4 relating to behavior and results are more challenging but provide more meaningful data on training impact. The document recommends starting by measuring easier levels before progressing to more complex behavioral assessments.
10 KIRKPATRICKS FOUR LEVELS OF EVALUATING TRAINING
EFFECTIVENESS 1. Reaction 2. Learning 3. Behavior 4. Results This grid illustrates the Kirkpatrick's structure detail, and particularly the modern-day interpretation of the Kirkpatrick Learning Evaluation Model, usage, implications, and examples of tools and methods. Kirkpatrick Learning Evaluation Model L E V E L EVALUA -TION TYPE What is measured? EVALUATION DESCRIPTION & CHARACTERI- STICS EXAMPLES of Evaluation Tools & Methods RELEVANCE AND PRACTICABILITY 1 R E A C T I O N
Reaction evaluation is how the delegates felt, and their personal reactions to the training or learning experience. 'Happy sheets' Feedback forms based on subjective personal reaction to the training experience Verbal reaction which can be noted and analyzed Post-training surveys or questionnaires Verbal or written reports given by delegates to managers back at their jobs It throws light on the level of learner satisfaction. The analysis at this level serves as inputs to the facilitator and training administrator. It enables them to make decisions on continuing the programmed, making changes to the content, methodology, etc. Very easy to obtain reaction feedback. Feedback is not expensive to gather or to analyze for groups 2 L E A R N I N G
Learning evaluation is the measurement of changes pertaining to knowledge, skill & attitude from before & after the learning experience. Facilitators utilize Pre & post-test scores On-the-job assessment Supervisors reports to measure & to check on the learning that has occurred.
Methods of assessment need to be closely related to the aims of the learning. Measurement need to be established, so as to limit the risk of inconsistent assessment. Relatively simple to set up, but more investment and thought required than reaction evaluation. Highly relevant and clear- cut for certain training such as quantifiable or technical skills. Measuring the effectiveness of training at this level is important as it gives an indication about the quantum of change vis--vis the learning objectives that were set. It provides critical inputs to fine-tune the design of the programme.
3 B E H A V I O U R
Behaviour evaluation measures the application of the learning in the work context. It measures the extent to which the trainees applied the learning and changed their behaviour. Observation and interview are required to assess change, relevance of change, and sustainability of change. Self-assessment can be useful, using carefully designed criteria and measurements. 360-degree feedback is useful method. Assessments need to be designed to reduce subjective judgment of the observer or interviewer. Assessments can be designed around relevant performance scenarios, and specific key performance indicators or criteria. Assessments tend to be more successful when integrated within existing management and coaching protocols. Measurement of behavior change is less easy to quantify and interpret than reaction and learning evaluation. Inputs at this level should be taken from participants and their supervisors. Their inputs can indicate the cause of success or failure. Simple quick response systems unlikely to be adequate. Cooperation and skill of observers, typically line- managers, are important factors, and difficult to control. Management and analysis of ongoing assessments are difficult, and virtually impossible without a well- designed system from the beginning. Behavior change evaluation is possible given good support and involvement from line managers or trainees, so it is helpful to involve them from the start, and to identify benefits for them, which links to the level-4 evaluation. 4 R E S U L T S
Results evaluation measures effectiveness of the programme in terms of business objectives resulting from the improved performance of the trainee.
At this level one look at key performance indicators such as increase in productivity, decrease in defects, cycle time reduction, return on investment, staff turnover etc. Quality inspection. Financial report. Interview with sales manager. For senior people, annual appraisals, ongoing agreement of key business objectives are integral to measure business results derived from training. It is possible that many of the measures are already in place via normal management systems and reporting. The challenge is to identify which and how to relate the trainee's input and influence. Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training, so they Individually, results evaluation is not particularly difficult. Across an entire organization it becomes very much more challenging, not least because of the reliance on line-management, and the frequency and scale of changing structures, responsibilities and roles, which complicates the process of attributing clear accountability Also, external factors greatly affect organizational and business performance, which cloud the true cause of good or poor results. understand what is to be measured.
Since Kirkpatrick established his original model, other theorists (for example Jack Phillips), and indeed Kirkpatrick himself, have referred to a possible fifth level, namely ROI (RETURN ON INVESTMENT).
ROI can easily be included in Kirkpatrick's original fourth level 'Results'. The inclusion and relevance of a fifth level is therefore arguably only relevant if the assessment of Return on Investment might otherwise be ignored or forgotten when referring simply to the 'Results' level.
Finally, it is possible for organizations to measure effectiveness for all programmes at level-1 and level-2. This can be built into the design of the training programme. Also it is easy to measure training programmes related to technical and functional areas at level-3 and level-4. But it is not easy to do this with behavioral skills programmes. Organizations that choose to measure training effectiveness can start with the former category before moving to measuring behavioral skills at level-3 and level-4.
There are mechanisms to measure effectiveness of behavioral skills at level-3. These are cumbersome to implement. It needs a fair amount of investment by the organisation in terms of time and money. While measuring effectiveness of training measure all programmes at level-1 and level-2. The measures at level-3 and level-4 can start with the functional skills, before moving on to the behavioral skills programmes.