Score Normalisation in Multiple Shift Exams - Part 1 : Introduction
One of the issues that many candidates try to understand is how scores are normalised when competitive exams are held in Multiple Shifts with varying difficulty and finally how they are ranked when the results are announced. Are there any models that the institutions use in these situations to help resolve this issue? Are there models where scores can be Normalised so that all candidates have a level playing field?
The purpose of this document was to analyse the approaches used by various Institutions when competitive exams are conducted in shifts (where each shift has a large group of candidates taking the exam of varied difficulty), leading to a final ranking system of the candidates. Since this is long documents explaining different approaches, I will publish my theory / approach in 4 different blogs.
There have been objections raised by candidates due to non-transparency of the methodology used for ranking the candidates. This research is based on simulated data, models and using some of the recent results available on the Internet. The problem got me interested into identifying the best possible approach when a recent competitive exam was conducted in Multiple Shifts. The study example in this research considers the recent NEET PG Exam and the questions raised by the candidates regarding the published results.
NEET PG 2024 is held by National Board of Examinations (NBE) in Medical Sciences, for MBBS graduates for entry into PG specialised courses across India. For the people who are uninformed, the NEET PG entrance exam was conducted in the second week of Aug 2024 and results announced around the last week of Aug 2024. The exam was conducted in 2 shifts (with more than 100,000+ candidates in each shift), varied difficulty (as per the candidate’s feedback) and the results were announced based on Percentile score. Candidates were ranked in descending order based on their Percentile score. NBE had released a notice stating how it will do the ranking based on Normalisation and Percentile scores. Many students contested the Percentile scores that they got and have now filed a case in Supreme Court of India asking NBE to be transparent related to Percentile calculation and ranking system.
This got me interested and I started my research and created 3 Approach’s with different models, for generating the ranking of a candidate.
Definition of Percentile
Let’s start with the basic definition of Percentile which I state below:
“Percentile is a number indicating the percent of cases falling at or below that score. There is no 100th percentile score.”
领英推荐
It is not possible that a candidate X scores 100th percentile in a competitive exam. A value - 100th percentile implies that 100% of the candidates who appeared in the exam are behind X, in terms of ranking, which is not possible as X, him/herself, is one of the candidates who appeared in the exam. As per the above definition, how come NBE and some other institutions in India give 100 Percentile to toppers in their competitive exams. Here we will not debate on that but try to find a best fit model for Multiple shift exams with varied difficulty and different number of candidates in each shift.
Before we look at the various approaches, let’s try out an example on Percentile calculation, based on above Percentile definition and that no Candidate can get 100 Percentile. Let’s consider:
Suppose, X is in the highest in the range, and there are 120,000 candidates and if X marks are same as 4 other candidates, then:
That means that under no circumstances, X and the 4 others can have a perfect 100 percentile score, unlike what NBE and other institutions claim that a person can achieve.\
Based on the above theory, we now look into the various modeling approaches that I have created and finally concluded which approach that I think (others might have different methods or opinions) is the best fit for competitive exams being conducted in different with varying difficulty.