Exposing the Flaws: Why NIRF Rankings Are Misleading Students and Urgently Need a Revamp?
Dr. Deepessh Divaakaran
Built & Sold 2 EdTech Start-ups | Author | Mentor MeitY Startup Hub
The results of NIRF 2024 will soon be released by the Ministry of Education, and I would like to discuss some flaws in the model we've been using for NIRF rankings. I'm sure the government has received feedback on this flawed model and will hopefully review and improve the framework.
Many may not agree with my views and I expect some backlash, but it's essential to address this issue. As stakeholders in education, we need to take a serious look at the current framework. Having been associated with many universities as an industry expert and involved in the NIRF project for the past 7-8 years, I have observed that there have been minimal changes in the National Institutional Ranking Framework (NIRF).
Another issue, which I have discussed with many academicians, is how flawed the NIRF ranking system is. NIRF is a tool that is available to the general public and students seeking admission to universities and colleges. The introduction of NIRF into the Indian education system in 2015 was a laudable step, as we did not have such a system before. The intention was to create a healthy competition among peer universities and colleges in India.
This framework also paved the way for many institutes to participate in global rankings like Times Higher Education and QS World Rankings.
However, I fear this framework has led many universities and colleges to engage in malpractice, report incorrect numbers, and focus on quantity over quality in research papers. Several of its important parameters are poorly designed, leading to absurd ratings.
Given that most colleges aim for good performance in these rankings, it is crucial that NIRF ranking parameters are reviewed and defined more intelligently.
NIRF 2023: Misaligned Weightage on University Exams Highlights Need for Ranking Reform
NIRF overall rankings in 2023 used 16 parameters to evaluate universities. The figure below shows the importance (weight) allocated to each parameter, arranged in descending order. Theoretically, these parameters represent a good spread of criteria. However, even the most important parameters are ill-designed.
What we found is that University Exams have been given a higher weightage than other parameters. While we talk about implementing NEP 2020, where the focus should be more on research and other activities rather than academic marks and traditional exams, NIRF still gives higher weightage to exams. This discrepancy shows a mismatch between our educational goals and the current ranking system.
This highlights the need for a serious review of the ranking parameters to align them better with our educational objectives and the evolving needs of students and institutions.
Re-evaluating the University Exams Parameter in NIRF Rankings
NIRF defines this parameter as the fraction of students that cleared all the exams within the stipulated time. If 80 percent or more students clear all the exams, the university gets full marks for this parameter. In the table below, we see that 71 percent of the top 100 overall colleges got full marks for this parameter.
The most significant parameter of NIRF is thus also the most useless, as almost everyone scores full marks in it. This parameter should be redefined to measure something more meaningful, such as the difficulty level of university exams using the average grade point in an institute.
This change would provide a better indication of the academic rigor and quality of education, rather than just the number of students passing their exams.
Addressing Research Parameter Flaws in NIRF Rankings
Research is one of the most important parameters in university rankings. We observed that two of the most important parameters, quality of publications and number of publications, are very strongly correlated. This is highly counter-intuitive and means that the number of publications is effectively being counted twice.
An explanation for this paradox could be that universities are publishing more and self-citing. Science magazine has flagged a case of an Indian university using self-citations to raise its rankings. Giving so much importance to the number of papers has also led to an increase in research misconduct. Retractions of articles from India have increased 2.5-fold from 2020 to 2022 compared to 2017 to 2019.
A suggestion to improve this parameter would be to include a penalty for retractions due to research misconduct. This will help reduce incidents of misconduct, as universities will have an incentive to ensure malpractices don’t take place.
Questioning Faculty Quality Rankings in NIRF
From students’ perspective, faculty quality is one of the most important parameters. However, according to the NIRF definition, Saveetha Institute has the best faculty quality in India (Refer Table below). Premier institutes like the Indian Institute of Science (ranked 11th) and IIT Madras (ranked 55th) ranking so low on this list raises serious questions about whether NIRF has defined this parameter properly.
One of the primary issues with the current NIRF definition of faculty quality is that it may not adequately capture the true expertise and impact of the faculty. Factors such as faculty qualifications, teaching effectiveness, industry experience, and international collaborations are often overlooked. Additionally, the current framework might give undue weight to factors that can be easily manipulated or do not necessarily reflect the actual quality of education provided.
In contrast, global standard rankings like the Times Higher Education (THE) and QS World University Rankings use a more comprehensive approach to evaluate faculty quality. They consider a range of factors, including faculty-student ratio, research impact (citations per faculty), academic reputation, and employer reputation. These metrics provide a more holistic view of faculty quality and their contribution to both teaching and research.
To align NIRF with global standards and ensure a more accurate assessment of faculty quality, it is essential to redefine this parameter. The new definition should include metrics such as:
By incorporating these elements, NIRF can provide a more meaningful and reliable measure of faculty quality, helping students make better-informed decisions and encouraging institutions to focus on genuine improvement.
Revaluating the Impact of University Size on NIRF Rankings
Size (number of students) has become an important factor for universities to move up the rankings. In the table below, we see that many private universities top this parameter. This is not surprising, as private universities are expanding at a much faster rate than public universities.
However, we need to be cautious about such uninhibited expansion, as it might not always be accompanied by a corresponding increase in the necessary infrastructure and human resources. This rapid growth can strain resources and negatively impact the quality of education. Additionally, it puts public universities, which often have more stringent expansion policies, at a disadvantage.
One significant issue with using size as a major ranking factor is that it does not necessarily reflect the quality of education provided. A larger student body does not guarantee better educational outcomes, and may, in fact, lead to overcrowded classrooms and overworked faculty if not managed properly.
In contrast, global standard rankings like the Times Higher Education (THE) and QS World University Rankings place more emphasis on factors that reflect educational quality and institutional impact rather than just size.
Instead of merely considering the size of student intake, it would be more effective to measure the quality of student intake based on scores in standardized entrance tests like JEE, NEET, or related exams. This approach would provide a more accurate reflection of the academic capabilities of the student body and ensure that universities are maintaining high standards in their admissions process.
By focusing on the quality rather than the quantity of students, we can encourage universities to prioritize excellence in education and better prepare their students for future success.
Questioning the Transparency of Peer Reputation in NIRF Ranking
This is one of the most opaque parameters in NIRF. It is assessed through a ‘survey’, the details of which are not disclosed. In below Table, we see the rankings according to peer reputation for some colleges. It is surprising to see that some newly established private institutions have quickly gathered more reputation than older, reputed institutes like BITS Pilani, the Indian Institute of Technology (Indian School of Mines), Dhanbad, and the National Institute of Technology, Warangal.
领英推荐
The lack of transparency in this parameter raises several concerns. Firstly, without knowing the methodology and criteria used in the survey, it is difficult to assess the validity and reliability of the results. Secondly, the rapid rise in reputation for new institutions suggests potential biases or flaws in the survey process.
In global standard rankings like the Times Higher Education (THE) and QS World University Rankings, peer reputation is also considered but with more transparency. These rankings provide details on how surveys are conducted, the sample size, and the demographics of the respondents. They also balance peer reputation with other objective metrics to ensure a fair and comprehensive evaluation.
To improve the peer reputation parameter in NIRF, it is crucial to enhance transparency and provide detailed information about the survey process. This includes:
By adopting these practices, NIRF can ensure that the peer reputation parameter is both transparent and credible, helping to build trust and confidence in the ranking system.
Addressing Data Accuracy and Integrity Issues in NIRF Rankings
Apart from the parameters being badly defined, there are also questions about the correctness of the data submitted to NIRF by the universities. In a survey carried out by India Research Watch with 410 respondents, most people (39 percent) thought that the data submitted to NIRF could be wrong. Another 36 percent said that paper quantity has become too important. We also showed earlier the reason why it is so, as the number of publications effectively counts for 21 percent of NIRF.
In my experience, although I am not naming universities or colleges here, I have seen top institutions hire private consultants to help them fake data just to get better rankings in NIRF. This malpractice has led to industry-wide false data being captured and significant discrepancies in the data submitted by universities to NIRF, NAAC, and NBA. If you compare any university featured in NIRF and look at their respective data submitted to NIRF, NAAC, and NBA, you will find huge discrepancies. It is puzzling why the government is not able to catch these wrongdoers and take strong action against them.
I am aware that a well-reputed agency in global rankings, QS World Ranking, has set up shop in India under the name QS I-GAUGE. QS I-GAUGE is a brand incorporated in India as an independent private-sector initiative specialized in rating colleges, universities, and schools. It combines the global expertise, experience, and reputation of UK-based (QS) Quacquarelli Symonds with deep local knowledge from Indian education luminaries. Many universities join this organization and pay hefty amounts hoping that QS will one day consider them for inclusion in their rankings. Otherwise, there is not much benefit to the universities being part of this organization when you already have National and International Accreditations like NBA, NAAC, ABET, AMBA etc.
To address these issues, the following steps can be taken:
By taking these measures, NIRF can enhance the reliability of its rankings and ensure that they truly reflect the quality and performance of higher education institutions in India.
Recommendations and Reforms for Improving NIRF Rankings
The National Institutional Ranking Framework (NIRF) plays a crucial role in shaping the perception and performance of higher education institutions in India. However, as highlighted in this article, several flaws and issues undermine the effectiveness and credibility of the current ranking system. To address these challenges and ensure that NIRF rankings accurately reflect the true quality of institutions, it is essential to implement comprehensive reforms. The following recommendations aim to enhance the robustness, transparency, and fairness of the NIRF ranking process, ultimately benefiting students, educators, and academic institutions across the country.
Complete Review of Parameters
As shown in this article, many of the parameters are ill-defined. All of them need to be reviewed to ensure that they are not being gamed or generating an unhealthy race among institutions.
Data Verification Teams
Since the data submitted forms the basis for the rankings, it is crucial that it is accurate. All anomalies must be audited by a competent team to ensure the data submitted is correct. This will help maintain the integrity of the rankings and ensure that they truly reflect the performance of the institutions.
Greater Transparency
All the raw data for all the colleges (even those not in the top 100) should be released in a tabular format (CSV). Additionally, the details of the methodology used to calculate peer reputation must be made public. Finally, there are some functions used to calculate composite scores that NIRF has kept secret. In the interest of transparency, they should divulge these functions.
Formation of a Competent Review Committee
Like in the case of the National Testing Agency, we hope the government forms a competent committee and orders a complete review of NIRF rankings. Given that the future of so many students and academics depends on it, it is important to get it right. We can do much better than the current state of affairs.
Incorporation of Qualitative Metrics
In addition to quantitative data, incorporating qualitative metrics such as student satisfaction surveys, employer feedback, and alumni outcomes can provide a more holistic view of an institution's performance.
Regular Updates and Revisions
The parameters and methodologies used in NIRF rankings should be regularly updated and revised to reflect the changing landscape of higher education. This will ensure that the rankings remain relevant and accurate over time.
Penalties for Malpractice
Introduce strict penalties for institutions found to be submitting false data. This can include disqualification from rankings, financial penalties, and public disclosure of the malpractice to deter others from engaging in similar behaviour.
Focus on Outcomes
Shift the focus from input metrics (such as the number of publications) to outcome metrics (such as the impact of research, employability of graduates, and innovation). This will encourage institutions to prioritize quality over quantity.
By implementing these recommendations and reforms, NIRF can improve the accuracy, transparency, and credibility of its rankings, ensuring they truly reflect the quality and performance of higher education institutions in India.
A Call for Urgent Reform: Ensuring the Future of Higher Education in India
In conclusion, while the National Institutional Ranking Framework (NIRF) was a commendable initiative to bring transparency and accountability to higher education in India, its current flaws and shortcomings undermine its purpose. The reliance on poorly defined parameters, the lack of data verification, and the opaque processes involved in the rankings have led to a system that is often gamed and misrepresents the true quality of institutions.
To restore trust and credibility in NIRF, it is imperative that the government undertakes a comprehensive review and implements the recommended reforms. By focusing on accuracy, transparency, and meaningful metrics, we can ensure that NIRF truly reflects the excellence and potential of Indian higher education. The future of countless students and the reputation of our academic institutions depend on it. Let us work together to create a ranking system that drives genuine improvement and fosters a culture of quality education.
The time for change is now. We owe it to our students, educators, and the future of our nation to get this right.
Statutory Warning
The views and opinions expressed in this article are those of the author and do not necessarily reflect the official policy or position of any university, institution, or organization mentioned. The author has made every effort to ensure the accuracy of the information provided, but neither the author nor the publisher assumes any responsibility for errors, omissions, or contrary interpretations of the subject matter herein. The content is provided for informational purposes only and should not be considered as professional advice. Readers are encouraged to conduct their own research and seek professional guidance before making any decisions based on the information provided in this article.
For More Information or Discussion please contact: [email protected] | +91 8086 01 5111.
Business Transformation Consultant: Ex Program Director, TCS
4 个月This is a pretty comprehensive analysis on the subject. The ACTORS should take it in positive spirit to plan changes, keeping Students interest in focus. I agree with the views expressed as I have been part of the PEER review members. Great post Dr. Deepessh Divaakaran
Mentor & Co-Founder @ CiteWorthy Lifesciences
4 个月Thank you for talking about this issue sir. This comprehensive and well-presented analysis highlights the important challenges facing higher education today. I hope this conversation raises awareness and prompts the government into action to address these problems.