Data-Driven Methods to Pass Exam P

Data is a powerful tool. It gives us the ability to extract quantifiable metrics from qualitative observations. It also lets us draw qualitative conclusions from quantified indicators. When we have access to large amounts of data, we begin to notice patterns and can predict the seemingly unpredictable.

When I started studying for actuarial exams a decade ago, I had no idea how many hours other students were studying, how they were performing on mock exams, or how quickly they were solving questions. Particularly for the earlier exams, where passing is more or less based on relative performance, this information would have been extremely helpful for me to gauge my expected performance relative to my peers.

Today, with the rapid evolution of technology and the decreasing cost of data storage, all this is now possible. The digitalization of education allows eLearning platforms to store data from large cohorts of students. The dissemination of this data allows students to draw patterns from it to inform their study behaviors.

Furthermore, for exams that have had consistent pass rates and relatively unchanged syllabus objectives, I posit that it is entirely plausible to use data-driven methods to predict a candidate’s pass rate with a high degree of accuracy. This information can further be used to inform the candidate’s decision-making process around studying and their level of confidence going into an exam.

All this sounds great in theory, but where can you view this data?

Built in the era of social media and digital connectivity, The Actuarial Nexus is predicated on the idea that studying for exams does not have to be an isolated experience. Each user on the platform has a profile. Within each user’s profile, rather than being bombarded with pictures of kittens, food, and unrealistic expectations, you are presented with your record of practice problems, mock exams, and other study patterns. If you worked 1,000 practice problems, detailed statistics for all 1,000 problems are recorded, including the solve time, accuracy, category, and every other metric imaginable. From there, benchmarking tools are available to gauge your performance relative to other users.?

Even if you had data, what metrics could you use to predict your chance of passing?

The historic pass percentage for Exam P has consistently been between 40% and 50%. This implies that to pass, you must perform in the top 40-50% of candidates.

The tricky part is defining what it means to be considered a “top” candidate. Currently, we rank your percentile performance across six metrics: time per question, total attempts, total correct attempts, 1st attempt accuracy, accuracy and level. The level is a separate ELO-based calculation that incorporates a variety of other factors, including the difficulty the questions attempted.

A weighted average of these percentiles is then used to determine your overall relative percentile. The percentile is then benchmarked against the pass rate to give you a sense of how close you are to passing. At the extremes, this methodology works fairly well. The top performers that were expected to pass actually did pass. Naturally, the candidates who are around 40%-50% are most at risk. However, because this number is readily available for users, it could (and I would argue should) be used as motivation to increase areas you’re weak in so you perform better relatively. If a candidate can increase their percentile from 40% to 90%, then I would feel much more confident about that candidate’s chance to pass.

The metrics not captured in the percentile above include diversity of syllabus topics and mock exam performance. These are measured by other tools available on the platform that may be covered in another post.

The Actuarial Nexus is still in its infancy so many of these predictive tools will continue to mature in the coming months. I imagine the predictive power will naturally increase as more problems are solved. My hope is that in time, the platform can utilize data-driven methods to confidently predict a student’s chance to pass an exam.

In 2023, technology is at a point where real time data is fast, cheap to host and not terribly difficult to implement. All the tools are available. It is just a matter of knowing which tools to use and building something worthwhile with them.

Follow us on LinkedIn to stay updated: https://www.dhirubhai.net/company/the-actuarial-nexus/