The Art and Science of Analyzing Learning Effectiveness

Most learning and development professionals are familiar with the four levels of Kirkpatrick's Learning Evaluation Model as shown in Figure 1.

Figure 1 : Kirkpatrick's Four Levels Evaluation Model

In theory, the above model is very simple. Achieving Levels 1 &2 is very much within the control of Learning & Development (L&D) professionals. The real challenge is to monitor and facilitate Level 3 & 4. Without the desired behavioral change ( Level 3 ), the training will not achieve the result ( Level 4 ). Experience L&D Professionals will appreciate the influence of non-training intervention ( e.g. company policies, resources, leadership support, etc.) to achieve training results. Without levels 3 & 4, the training investment is merely a WASTE. Association of Talent Development revealed about 85% of training investment is a waste in a report titled 'The Missing Link In Learning : Transfer' in their July 2018 publication shown in Figure 2. Globally, billions of Ringgit are wasted in the name of training.?

Figure 2 : Missing Link in Learning : Transfer

Incorporating the strategy to achieve Level 4 is a key element of an effective L&D. The theoretical foundation and application of Kirkpatrick's Model is a valuable skill set for any L&D professional with the passion to create an impactful training. L&D professionals are key players to enable an organization to optimize their most valuable asset: TALENT.

I was first introduced to Kirkpatrick's Learning Evaluation Model in 1985 while pursuing my BSc. ( Education ) degree at USM, Penang. The solid theoretical foundation from our amazing professors of education is extremely valuable in my L&D professional life.?

In 1994, I was blessed with the opportunity to attend 2 weeks Certification Program on Royal Malaysian Navy Training System (RMNTS). Kirkpatrick's Model is a key element of RMNTS. The foundation of Kirkpatrick's Model was valuable to perform my role as Head of Training Evaluation Unit of KD PELANDOK, the Training Center of Royal Malaysian Navy. Competency to perform the assigned role of every ship crew is critical for the effective operation of a Navy ship. Every staff has a pre-defined role. Failure to perform their role according to the expected competency standard may impact the ability to defend the maritime sovereignty of our nation. The peace and stability that we enjoy are the results of our security services including Royal Malaysian Navy. The unstable country will have a significant negative impact on the foreign direct investment and socio-economic development of our beloved nation. Thus, competency is a matter of life and death.

In the year 2000, I had another amazing valuable opportunity to apply Kirkpatrick Model in my professional life. I was the Head of the Marine Technology Academy, Boustead Naval Shipyard. My team was responsible for the Analysis, Design, Development, Implementation, and Evaluation of training requirements for competency building of the newly built New Generation Patrol Vessel (NGPV). My team of training specialists conducted training and validated the crew competency while sailing the ship from our base in Lumut, Perak to the famous Langkawi Island for 10 days. Validating the competency of the crew was the key requirement to meet the acceptance criteria of the six NGPV of this RM 6 Billion Mega Project. 14 years of hands-on experience as a Project Manager (Training) for this project was super valuable for my professional life in L&D.

I am currently a member of a pilot project in Cambodia to certify L&D Professionals. Level 4 Evaluation of the training impact is a key element of this certification program. Being a member of this project is a great learning experience for me. Kirkpatrick's model requires adaptation to be aligned with organizational cultures. The combination of our training and coaching program is critical for the success of this pilot project. I look forward to the valuable lesson from this pilot project.

Implementing Level 3 & 4 are time-consuming, resource-intensive, and expensive. Many variables influence change of behavior that contributes to an organization's performance improvement and business results. There is no one-size-fits-all formula to achieve business results from training investment. Based on my experience as a learner and practitioner of Kirkpatrick's model, it is a valuable guide to strive towards Level 4.

Moving towards Level 4 could be a CHANGE for many L&D professionals. Change begin with ABC, Awareness Before Change. I hope more L&D Professionals will be aware of the power of Kirkpatrick's Model to create an IMPACTFUL L&D program.