How to measure the success of learning and development

Companies are rapidly expanding learning and development initiatives with the hope of increasing retention, belonging and revenue. This article shows you how to craft a parallel framework to measures the effectiveness of training programs.

Ineffective training is a waste of time and money. Research shows that United States companies alone spent $92 billion this year on learning and development. Yet, without measuring the effectiveness of training, how do you know if your development dollars are increasing employee performance and enhancing your business overall? (Spoiler alert: you don’t!)

Unlike marketing or sales, there are fewer metrics to measure the success of your programming. This is why it is so critical to deploy learning and development analytics in parallel with your training programs. Data can uncover valuable insights into participation rates, progress trends, the number of completed certifications, drop-off areas, and missing links in your learning framework.

What Is Training Effectiveness?

Training effectiveness is the degree to which learning improves employee performance. For example, how well your team members enhanced their sales and communication skills, increased their productivity, or met a business goal. To measure training effectiveness, you use key performance indicators (KPIs), or more specifically, training metrics.?

What are Training Metrics?

Training metrics are qualitative and quantitative measures used to track and assess the results of the learning and development process. They often include:

• Pass or fail rate: the successful completion of tests and quizzes.
• Scoring: estimate learners’ performance, allowing instructional designers to adjust the difficulty level of a quiz or test if needed.
• Training experience satisfaction: identifies how learners perceive training.?
• Training completion and dropout rate: number of learners who finished a course.
• Post-training performance: how employee performance has improved.
• Return on investment (ROI): compares the financial value of the learning outcomes against the investment you made to achieve those results.?

How to Identify the Right Metrics for Your Training

To identify the right metrics that will be relevant for your company’s training, consider the following:?

Ask what stakeholders need

Before laying out an evaluation strategy, understand key stakeholders’ expected metrics. What do they consider the most valuable indicators? What are their priorities for understanding? Communicate your vision of training results clearly and keep stakeholders informed along the way.?

Track department-specific data?

If you’re a large company, tracking how well different departments perform and what factors impact that will be telling. A Learning Management System (LMS) can track and organize the data on assigned, attempted and completed courses.?

Consider your learning model?

Metrics will differ depending on the type of training: synchronous (same time), asynchronous (not the same time), or blended. In asynchronous training, for example, you can track how learners progress through modules. Different employees will likely take differing time and effort to grasp and retain new knowledge, which means the metrics can include the number of modules completed, the time spent on each, and scores attained in tests after each module.?

How to measure training effectiveness?

Once the training metrics are narrowed down, measuring training effectiveness can be conducted through 1:1 discussions, surveys and questionnaires, pre/post-training quizzes and assessments, and examinations.?

Five proven evaluation models most often trusted by companies today include:

1. Kirkpatrick’s Four-level Training Evaluation Model
2. The Phillips ROI Model
3. Kaufman’s Five Levels of Evaluation
4. Anderson’s Model of Learning Evaluation
5. Summative vs. Formative Evaluation

In this article, we’ll be focusing on the Four-Level Kirkpatrick model developed in the 1950s. Variations of the Kirkpatrick model can be used to build training metrics or KPI (key performance indicator) framework for your training.

Level 1 - Reaction

The first step is to evaluate the learners’ reactions and responses to the training. Use surveys, questionnaires or talk to learners before and after the course to collect their feedback on the learning experience. At the end of Level 1, you should know how well the training was received and determine any gaps in the training content.?

Sample survey questions:

1. What did you enjoy most about the training?
2. How much new information did you receive?
3. Was the information actionable? How relevant is it to your job?
4. Were you engaged with the content?
5. Was the training well-organized and well-run?
6. Did you face any difficulties when taking the course?
7. What could have been improved?

Level 2 - Learning

The second step is to measure the knowledge and skills learned during the training. At this stage, you can determine if the training is meeting its set objectives, what specific skills that can be developed, and the scope for improvements in content and method of delivery. A combination of metrics can be used such as:

• Test scores during and after the training
• Evaluation of applied learning projects
• Influence on performance KPIs
• Course completion and certification
• Supervisor report and feedback

Level 3 - Behavior

Step three assesses the behavioral change (if any and to what extent) due to the training. How has learning been implemented at work? Are learners confident to share their new skills and knowledge with their peers? Evaluate by using a combination of these methods:

• Self-assessment questionnaires
• Informal feedback from peers and managers
• Focus groups
• On-the-job observation
• Actual job performance key performance indicators (KPIs)
• Customer surveys, comments, or complaints

Level 4 - Results

The final step is to measure the training’s impact on business goals and results, such as reduced cost, improved quality, faster project completion, increased productivity, employee retention, better marketing leads, increased sales, and higher morale. Key metrics to measure are:

• Improved business results
• Increased productivity and quality of work
• Employee retention
• Higher morale
• Customer satisfaction index

Measuring the ROI?

Quantifying the results is necessary to calculate the return on investment (ROI). Did the outcomes after training justify its cost? The formula for calculating ROI is:?

ROI (%) = [(Total Program Benefits – Total Program Costs) / Total Program Costs] *100

Total Program Benefits means the profit produced for the company by the employee over a year as a result of, for example:

• increased sales (versus pre-training performance);
• fewer errors;
• increased customer satisfaction;
• revenue gains.

The most challenging part of an ROI study is getting reliable estimates for the financial benefits. Ask the accounting department to help. Total Program Costs involve all the training expenses for the same period, including

• initial needs assessment;
• software or equipment expenses;
• learning content;
• travel and infrastructure costs;
• instructor costs;
• participant salaries.

Ultimately, understanding training evaluation metrics and how to measure the effectiveness of online learning will prevent you from wasting time and money while making your training programs (and workforce) more effective.

Start identifying training metrics, reconciling those with stakeholders expectations, departments and training models, and then move through the four levels of the Kirkpatrick model to evaluate your corporate training’s effectiveness.