Training Evaluation

UN Training of Trainers (ToT) CourseBangladesh, June 2012

UN PEACEKEEPING TRAINING SYSTEM


Why is this Important for me?


Why is this Important for me?

As peacekeeping trainer, you must know how training can be professionally evaluated with a view to : • Determining the impact of the training on the target audience

• Incorporating modifications as deemed needful for making it best fitted and most effective to mitigate the competency gap


Explain why training should be evaluated

Describe different levels of outcomes integral for training evaluation

Use evaluation model and appropriate criteria to evaluate training

Learning Outcomes


UN Training Cycle: A Six Step Process

3. Design & Plan

Training

1. Analyze Performance

Gaps

6. Monitor And Improve

5. Evaluate Training

Objectives

2. Define Training Needs

How Does UN Training System Look Like?

4. Deliver Training


STEP FIVE

Training Evaluation

Fill up

Competency Gap


• Evaluation is the systematic and continuous process of clarifying the effectiveness of activities

• Evaluation is traditionally represented as the final stage in a systematic approach with the purpose to maximize the result of your effort and to determine future strategy

Understanding Evaluation

STEP FIVE

Training Evaluation


• Feedback to improve future training efforts

• Foundation for positive reception of future training proposals

• Validates the training program• Justifies resource expenditure• Accountability

Why to Evaluate?

STEP FIVE

Training Evaluation


For effective training evaluation the principal questions should be: • To what extent were the identified training needs/ objectives achieved by the program?

• What specifically did the learners learn? Was it relevant?

• What commitment have the learners made about the learning they are going to implement back on the work?

• How successful were the learners in implementing their action plans? Was the training appropriate?

What to Evaluate?

STEP FIVE

Training Evaluation


STEP FIVE Measuring Stick

Training Objectives

• Achieved

• Partially Achieved

• Not achieved

Competency Gap

• Adressed

• Partially Adressed

• Not Adressed

ReactionLearningBehaviour & Result

• As Expected

• Less than Expected

• Not at all Expected

Training Evaluation


Six general approaches to training evaluation can be identified as follows:• Goal-based evaluation• Goal-free evaluation• Responsive evaluation• Systems evaluation• Professional review• Quasi-legal validation

Approaches

STEP FIVE

Training Evaluation


Levels of Outcomes

STEP FIVE

Outcomes are the means by which you know how well the process is working

Behavior

Training Evaluation


• Reaction directs how learning can be integrated/ internalized

• Level of learning determines how much behaviour can be changed back on the job

• Change in behaviour diectly proportions organizational impact (result)

Linkage between Levels

STEP FIVE

Behaviour

Learning

Reaction

Result =

Training Evaluation


Kirkpatrick’s Mirror

STEP FIVE

Behaviour

What behaviours are necessary to further this objectives?

Result

Training Evaluation


• Reaction - what the trainees thought and felt about the training - could it motivate them - how much?

• Learning - the resulting increase in thinking or capability - change in knowledge, attitude and skill

• Behaviour - extent of behaviour and capability improvement - transformation of attitude-knowledge-skill height

• Results - the effects on the work or environment resultant from the trainee's performance

Evaluation of Levels

STEP FIVE

Training Evaluation


• Evaluating reaction is the same thing as measuring customer satisfaction. If training is going to be effective, it is important that students react favorably to it

• The instruments for Reaction evaluations are:

Happy sheetsFeedback formVerbal reactionPost-training surveysQuestionnaires

Overview

TRAINING

EVALUATION

Reaction


• Did the trainees like and enjoy the training?

• Did they consider the training relevant?

• Was it a good use of their time?• Did they like the venue, style, timing, domestics, etc?

• What was their level of participation?

• How easy and comfortable is the new experience?

Evaluation CriteriaReaction

TRAINING

EVALUATION


• Learning evaluation is the measurement of the change in knowledge, attitude and skill – from before to after the training

• The instruments for conducting Learning valuations are:Typical assessments or tests before and after the training

Interview or observation

Overview

TRAINING

EVALUATION

Learning


• Did the trainees learn what was intended in the learning outcomes?

• Did the trainees experience what was intended for them to experience?

• What is the extent of advancement or change in the trainees after the training, in the direction or area that was intended?

Evaluation Criteria

TRAINING

EVALUATION

Learning


• Behaviour evaluation is measuring what change in job behavior occurred because people attended a training program?

• The instruments for conducting Behaviour valuations are:Observation and interview over time to assess change, relevance of change, and sustainability of change

Overview

TRAINING

EVALUATION

Behaviour


• Did the trainees apply their learning when back on the job?

• Were there noticeable and measurable changes in the activities and performances of the trainees when back on their roles?

• Did the changes in behaviour and new level of knowledge sustain?

• Are the trainees aware of their changes in behaviour, knowledge and skill level?

Evaluation Criteria

TRAINING

EVALUATION

Behaviour


• Result evaluation is the effect on the work or environment consequential from the improved performance of the trainees

• The instruments for conducting Results valuations are:Measures indicating organizational key performances

Overview

TRAINING

EVALUATION

Result


Evaluation Criteria

TRAINING

EVALUATION

Result

Organisational key performance indicators include:

• Output• Cost • Quality• Time • Service feedback• Return on investment (ROI)ROI


• ROI is a monetary measurement that is used to evaluate the efficiency and effectiveness of an investment made by an organization

• Investments take many forms – financial, human capital, equipment, and training programs

• The ROI Methodology was developed by Jack Phillips in the 1970’s and implemented globally during the 1990’s

Overview

TRAINING

EVALUATION

ROI


The core steps follow:• Determine the direct costs of the training

• Measure the productivity or performance increase

• Translate the increase into a dollar value benefit

• Subtract the dollar value benefit from the cost of training

• Calculate the ROI

Steps

TRAINING

EVALUATION

ROI


• A comprehensive approach to training measurement

• Begins with planning the project (impact study)

• Moves into the tools and techniques to collect data

• Analyze the data• Finally report the data• The end result is not only a Level 5 ROI but also a measurements on the Kirkpatrick 4 Levels

Parameter

TRAINING

EVALUATION

ROI


Flow Chart

TRAINING

EVALUATION

ROI


Finally It Is Kirkpatrick-Philips ModelResults

Behavior/Impact

Learning

Reaction

STEP FIVE

Training Evaluation


STEP FIVE

Training Evaluation


Evaluation Target

STEP FIVE

Training Evaluation


Evaluation Flow Chart

STEP FIVE

UN Training of Trainers (ToT) CourseBuenos Aires, November 2011

Training Evaluation


Remember

Teaching

• Giving Knowledge• Theoretical• Passive Participation

• Watch Me • Try it Yourself

Training

• Sharing Knowledge

• Real-life or Practical

• Active Participation

• Lets do it together

• Try and improve it further

STEP FIVE

Training Evaluation


QuestionsSTEP FIVE

?

? ?Results

Behavior/Impact

Learning

Reaction

Behaviour

Learning

Reaction

Result


Group Activity on Reaction Evaluation

Learning AssessmentSTEP FIVE


SummarySTEP FIVE


Thank You

Training Evaluation

Documents

Transcript of Training Evaluation