Score Normalisation in Multiple Shift Exams - Part 4: Normalisation of Raw Scores

In the previous 3 blogs on this topic, we saw 2 approaches which I feels will not lead to a level playing field of the candidates. Let's look at the 3rd Model, which might solve the problem,

In this model, let’s apply Normalisation to the candidates score even if Multiple shift exams are conducted with different types of difficulty. This model normalises the score and creates a level playing field for the candidates in each of the shifts. This involves quite a few calculations based on the below considerations:

• Both the Shifts exams were not of same or equal difficulty
• Total Number of candidates in each shift is not the same
• The advantage in applying Normalisation is that when we equate the highest score of each shift to 100 and then normalise the other scores w.r.t this normalised 100, we are now creating an even and level playground for all.
• Any advantage / disadvantage that either of the Shift has in terms of difficulty of the exam is nullified here.
• Percentile calculation based on collating the raw scores of each shift is eliminated as the final Percentage calculation uses the Normalised value after which the Percentile scores are calculated after merging this Normalised Percentage score.

Normalisation is done as per the below process:

1. First calculate the score obtained by each candidate in each shift
2. Tabulate the same in 2 separate tables
3. The Topper’s score in each Shift is equated to 100%, and all other candidates score is normalised against this score as given below:

• If the Shift 1 highest score is 717. This is equated to 100%, which becomes the highest percentage in this Shift. This is the Percentage and not the Percentile.
• If the Shift 2 highest score is 707. This is equated to 100%, which becomes the highest percentage in this Shift. This is the Percentage and not the Percentile.

4. Now calculate the score of other candidates in each shift as below:

a.??Shift 1

Consider a Candidate who got a final score of 697.

Then his percentage is = 697 *100 / 717 = 97.2105997%

b.?Shift 2

Consider a Candidate who also got a final score of 697.

Then his percentage is = 697 *100 / 707 = 98.5855728%

5.?Collate / merge both the Shifts Percentages into one Table and arrange all these Percentages in Descending Order (Highest to Lowest).

6.?Now that we have the Normalised scores for all candidates, calculate the Percentile score for all the candidates accordingly.

The Rank for the candidates is given the below “Table 5” for the simulated data of Table 1 & Table 2 (without the Percentile Column) given in Approach 1 above.

Assuming, Total Candidates = 216136

Now let’s do a comparison of each of the Models that we have created and simulated the ranking of the candidates. In the below Table, one can see the ranking difference between:

a.?????? Percentages & Percentile calculated based on Raw Score (Table 4 in Part 3 of this Normlisation Blog)

b.????? Percentages & Percentile calculated after Normalisation (Table 5 above)

c.?????? Collating Percentile calculated for each Shift (Table 3 in Part 2 of this Normlisation Blog)

Summary

From all the above Models and Simulations and the results that we have obtained, the Normalisation Model (of this Part 4 blog) is the best approach for resolving ranking issues related to Multiple Shifts exams with different difficulty in each exam. The number of shift can also be more than 2 and still the above approach and Model will work, as this is setting a Level Playing field of the scores before calculating the Percentiles and the ranks for each candidate.

There might be others who might have a better model and solution than what I have put forward, but using Raw Score’s & Percentages based on the Raw Scores without Normalisation will never give the right ranking of a candidate for Multiple Shifts exam with varying difficulty in each Shift (Approach mentioned in earlier blogs).

Since the number of candidates in each Shift are also not equal, Approach 1 is also ruled out.

Conclusion

Lack of transparency, using the right approach / models to rank the candidates in competitive exams conducted in Multiple shifts is harming our system and putting a lot of stress on our young generation of upcoming Engineers / Doctors and other graduates, who want to specialise in a field of their interest. Has anyone evaluated the questions set for each shift to check if all the Shifts paper were of equal difficulty? These questions and many more such doubts have crept into the minds of candidates writing these exams.

I am sure that there are many more Mathematicians and Statisticians, who can come up with a better process and methodology in similar exam situations. If we can have Duckworth Lewis model for Cricket, can we not have a method created for Multiple Shift exams, taking into consideration the difficulty of the exam in each shift the questions asked in each subject?

Coming up with right Normalisation process / solution is what I have tried to achieve here. Mathematicians & Statisticians should delve into this problem and try to identify the right models for resolving examinations being held in Multiple Shifts. Of course, there might not be a right & best approach now and but an approach with minimal bias will be the best model that can be used in the future.