You’ve been in your learning and development role for a little while now. You always receive positive feedback from colleagues that the training you create, either face to face or digitally is great. Your companies’ senior management team however, expects more than just positive feedback. They’re expecting you to report to them on a quarterly basis the impact your training is having on the key business objectives or metrics. Where do you begin?
You’re first step might be to create a happy sheet, followed by an email out to managers to ask what’s changed. You wouldn’t be alone in this approach, it’s typical. If you want to demonstrate real measurable difference to our senior management team however, you need to think bigger than this.
One particular methodology that can help you think bigger is Kirkpatrick Training Evaluation.
What is it?
Kirkpatrick Training Evaluation is a model created by Don Kirkpatrick. Which today is regarded as one of the main models of evaluating training.
As learning and development professionals we pride ourselves on our ability to create effective training. By not measuring the impact of what we create and how much is applied on the job then we are firstly, doing ourselves a disservice. Secondly, leaving ourselves wide open to being seen to not deliver on business objectives.
Overview of the model and the levels.
The Kirkpatrick Training Evaluation model is split into four levels, Results, Behaviours, Learning and Reaction. You may be inclined to start at reaction (level 1) and work your way up to results (level 4), as this is the chronological order. The model actually requires you to start at level 4 and work backwards.
The Kirkpatrick principle is to “begin with the end in mind”. Therefore if you’ve waited until after the training has taken place to think about evaluation, you’ve left it too late. Evaluation should be considered right at the earliest point, usually at the scoping stage.
Whilst scoping out any learning intervention with your stakeholder discuss the purpose of the training and what will be different for the individuals in terms of their behaviour (L3), and company as a result of the learning (L4). By doing this you are already starting to gather your success measures.
When to use the Kirkpatrick model
It’s worth noting that you don’t need to apply this model to every learning intervention your business takes. Kirkpatrick Training Evaluation is best placed for particular types of learning projects (outlined below). As to work through every level in detail can be time consuming, and unnecessary for a one off session for a team.
I implemented an evaluation procedure at a previous company, defining when full evaluation would be appropriate. This categorised four key areas:
- High profile
- High cost or time commitment
- Large groups
- New projects
Kirkpatrick Training Evaluation levels
- Results – to what extent has the learning impacted on key measurable business objectives
- Behaviour – to what extent has the learning been applied back on the job
- Learning – to what extent has the knowledge been retained
- Reaction – to what extent had the learner seen the initial learning as valuable
Level 4 – Results
Results is always the starting point for evaluation. You need to understand what impact you want to have, to know if you are creating the right content to meet the need.
When scoping out any learning intervention work with your stakeholder to truly understand what wider business impact there will be as a result of the training. There should only be one overarching aim. Once you have this clear then move onto the other levels.
Examples of the result might be: To achieve profitable growth through high quality sales or to provide premium customer service and a value added experience to every customer.
Some questions that can help with scoping for L4.
- What does the organisation exist to do? – Consider the biggest possible goal or impact that could change on an organisation perspective, and how the learning intervention fits into this
- What is the wider purpose of this learning happening?
- What would be the impact if the learning didn’t take place and we did nothing?
Level 3 – Behaviour
Behaviour is the stage when speaking to stakeholders whilst scoping they usually haven’t considered. Behaviour means, what should the learners be doing differently as a result of attending the learning back at work? It could be following a process correctly, improved accuracy, increased sales, improved engagement etc.
Behaviour is the area that is most challenging, yes learners have attended but that doesn’t mean they are going to apply it back in the workplace. Your job is to understand what barriers might exist that would prevent them implementing the learning, and remove them as far as possible with the stakeholder before the learning takes place.
If you’ve not anticipated and removed the barriers in advance then this is an immediate block to learning – it could be as simple as having job aids or process maps ready for everyone upon completion.
To measure impact at this level there are lots of options. Such as job based observations, assessment of metrics such as data accuracy or improved customer service metrics, interviews with learners or their managers.
Examples of key behaviours could be: Knowledge and/or confidence using a particular model/process/procedure, 80% accuracy in completing the tasks, commitment to change.
Questions to help with the scoping for L3:
- What (up to) 5 key behaviours does the individual need to display back in the workplace to reach the goal set out at L4 results stage?
- What barriers might get in the way of individuals being able to implement this learning back at work?
- What job aids could we create to help individuals apply the learning back on the job?
Level 2 – Learning
Learning, the extent to which the knowledge, skills or attitude is retained. The classic assessments of this is through testing. Some organisations use testing in induction to ensure learners meet a certain standard prior to being allowed to complete particular tasks, which is a good example of level 2. Level 2 can also be assessed during the learning intervention through the use of role plays or simulation exercises.
Some examples of Level 2 checks:
- Knowledge tests/check
- Observation of work
- Role play
Questions to ask at level 2 with the learners either during the learning or after:
- Do you have the knowledge/skills required to complete this particular task back in the workplace?
- Do you believe it would be worthwhile to use this learning in the workplace?
- How confident are you with this learning?
- How likely are you to implement this learning back at work?
Level 1 – Reaction
Reaction is the stage that we are most familiar with, those trusty happy sheets (immediate reaction evaluation questionnaires) we all use. Level 1 covers to what extent the learner had seen the initial learning as valuable.
The happy sheet is a good initial measure but it’s usually the last thing we ask learners to complete and they usually quickly fill it in and dart out the room. On the other hand, if you ask for responses after the training the completion rate drops dramatically. My advice is don’t worry too much, as this is the area that requires the least amount of focus.
The less focus on this stage the better because it doesn’t tell the business as much compared to focussing your efforts on level 3 and 4.
If you do wish to complete a happy sheet, Kirkpatrick recommends doing a pre/post evaluation, but asking learners to complete both elements POST learning. The reason for this is learners are likely to be honest about what they did/didn’t know post learning, rather than pre.
For the level 1 forms you do ask learners to complete, move away from a generic form for every session. Personalise each level 1 evaluation to the specific behaviours outlined in Level 3. By using those specific behaviours as measures it immediately has more value than asking if you had nice refreshments and had fun.
Final tip for level 1: Don’t underestimate your knowledge as the facilitator of the session, upon reflection you are likely to know which bits worked well and the elements that didn’t.
Questions for level 1 in the evaluation form:
- Please rate (insert behaviour here) before and after the training – do this for each of the up to 5 behaviours specified in level 3.
- How can this programme be improved?
Kirkpatrick Training Evaluation provides a framework for you, as a learning and development professional to clearly demonstrate your value and impact to the business.
This guide hopefully has provided you with the tools and techniques to get started evaluating all the way to level four. Below is a summary timeline of how to embed the Kirkpatrick Training Evaluation model into each step of your learning cycle.
Want to read more on training evaluation, try my blog post on training measures outputs vs outcomes.