The difficulty in understanding Matiangi’s 2016 KCSE exam results

The name Matiangi, is now synonymous with fidelity, achievement and a no-nonsense approach to things in Kenya. For those that need an introduction, Dr. Fred Matiangi is the Cabinet Secretary for the Ministry of Education and in the year 2016, he was constantly in the news for his far-reaching reforms in the education sector. His determination to rid the sector of cartels and mediocrity was relentless and it culminated in the release of the 2016 national exam results in record time, with few (if any) irregularities and student performance that shocked the nation. 

Results from the secondary school exam especially shocked the nation. Only 141 students countrywide scored the top grade ‘A’, unlike in 2015 and in 2014, when 2,636 and 3,073 respectively, attained this top grade. The entire set of results point towards a collective failure by the class of 2016. 

This immediately prompted a major debate in the country. Some say that the exams in previous years were not credible and thousands of underserving students earned top grades, while the other side of the debate is of the opinion that the 2016 results are artificial because exam marking was rushed and they point to some glaring inconsistencies in subject results, especially the language subjects. 

The exam council is yet to give a detailed report of the exam so I can’t really discuss the 2016 KCSE results forensically. I however would like to lead you into discovering a few things that can be observed in the backdrop of these results. I promise to spare you the intricacies of average, standard deviation, variance and other statistical witchcraft, so bear with me.

Standardized Exams

Kenya Certificate of Primary Examinations (KCPE) and Kenya Certificate of Secondary Examinations (KCSE) are national standardized exams designed to give everyone a level playing ground in the access of further education in Kenya. The idea is that to properly assess the entire country, standardized exams give the best opportunity for fairness because results are expected to be in a standardized format. 

Standardized results take the format of few low achievers, an average majority and few high achievers. Represented on a graph, the distribution of results should be in a curve that looks like a bell, it’s what statisticians call a normal, standardized or bell curve. 

The 2016 KCSE Results

The 2016 results however do not look anything like a bell curve. The bulk of the students are on the wrong end of the results with the grade ‘D-’, a grade representing near total failure, carrying 149,929 of the 571,161 candidates that sat the exam (26%). 

 Compare this with the 2014 and 2015 results and you will notice that as much as it still isn’t a bell curve in any of those years, the distribution of results is more even (you can tell by the smoother curves) and the grades carrying the highest number of students are ‘D+’ and ‘D’ for 2014 and 2015 respectively. The overall results are still foreboding, but not as badly as the 2016 results. 

 Like I said, I will not speculate as to why the 2016 results appear to be remarkably different and instead highlight something that can be derived from observing these graphs. 

Intelligence 

Intelligence is a person’s ability to acquire and apply knowledge. Intelligence is mostly natural ability, even though it can be enhanced a little, and the Intelligence Quotient (IQ) Test, is a standardized way of measuring it. 

The chart above is from a medical school in Canada. 55 represents very low intelligence while 145 represents exceptional and rare intelligence. 100 represents average intelligence and this is where the bulk of the population lies. 

This Canadian IQ test, and virtually all other IQ tests globally, yield such a bell curve because it is normal that across any population of people, that the majority will have average IQ while an exceptional few will have wither very low of very high IQ. 

So, if intelligence, which is the measure of natural ability, yields a bell curve, it should follow that KCSE results should yield a similar bell curve because it is a measure of hundreds of thousands of students across the country. 

Observing the 2014, 2015 and especially the 2016 results, could it be that the KCSE exam is not a standardized exam? Could it be that despite being a standardized exam it’s simply impossible to standardize the results because the majority perform very poorly? Or is it that KCSE measures something else that is not consistent with intelligence and hence impossible to get a normal result from more than 500,000 candidates? 

What exactly does KCSE Measure? Intelligence vs Aptitude vs Achievement

Intelligence, like previously mentioned, is general cognitive capability, while aptitude is ability to learn or develop certain specific skills like working with numbers, writing, engineering or computer programming. Think of aptitude as talent. One can’t quite prepare for an aptitude or an intelligence test; it’s a test of that which is inherent in someone.

Achievement tests on the other hand are about gauging how much one has learnt and to pass an achievement test, one must prepare by reading and mastering content beforehand. To that end, KCSE is most certainly an achievement test. 

This distinction is the main reason I think it is impossible to achieve a normal (bell curve) in KCSE results. Being an achievement test, results are 100% dependent on preparation. Looking closely at our schools, there is so much inequality in the distribution of educational resources and quality of teachers and so preparation levels vary significantly. 

This means that every year, hundreds of thousands of students are chronically under prepared and consequently judged to be poor students when their only fault is being in the wrong school at the wrong time. 

And how do they end up in wrong schools? The primary school exam, KCPE, is similarly structured and the best prepared and performing candidates always end up in secondary schools with the best resources and teachers, giving them a higher chance of passing KCSE. 

The inequality in the system of education is so severe that progress is almost certainly determined by how rich one is, because being rich affords one the chance to be in a private primary school, and private primary schools tend to prepare students better the KCPE exam. 

The way forward? 

It is impossible to improve all the schools and teachers overnight to the level that we can offer primary or secondary school students standardized exams with a clear conscience. 

A new curriculum in 2018 is being mooted as the answer to the challenges we have, but I fear that the bodies charged with managing education – the Ministry of Education, the Kenya Institute for Curriculum Development (KICD) and the Kenya National Exam Council (KNEC), are very poor creators and users of organizational knowledge. These organisations, despite their decades long work and experience, have learnt little and they depend on individuals such as Dr. Matiangi (who come rarely) to improve them and demand quality. 

This means that a new curriculum might not resolve the problems as anticipated because the current thinking and way of doing things by the country’s education managers will stymie the new system. 

So,

Under our current circumstances, budgets and pace of growth, I propose that we make some changes to the way KCPE and KCSE is structured. A simple restructuring that does not require major institutional reforms but one that will make the exams truly standardized and fair. 

I propose that we transform KCPE and KCSE from exams that exclusively measure Achievement to exams that also measure Aptitude and Intelligence. By doing this, natural ability will be measured alongside achievement and good students, irrespective of their school level disadvantages, will shine through. 

For KCPE, we should go as far as making it up to 50% a measure of intelligence and the other 50% a measure of achievement, because by the end of primary school, students are still discovering the world broadly and are not yet specialized.

This does not mean that we no longer teach the vital literacy, numeracy, science and social subjects that we currently do. It simply means that while competing for places in secondary school, school level disadvantages should not prevent a student with high potential from progressing. 

For KCSE, I propose that we make it up to 50% a measure of aptitude because at this level, students are preparing to specialize in university and diploma college and we need to discover the best careers for them. 

We currently use subjects to determine aptitude, but this method is wildly inaccurate because a student that is good at Mathematics is considered good enough for an entire range of careers from Actuarial Science to Wireless Networking. The aptitude testing in KCSE should give more accurate indications as to whether a students is inclined towards being an accountant or a telecommunications engineer. 

But let’s not be naive

I am aware that there are other factors that influence the KCSE results. One such factor, I speculate, is the country’s higher education and vocational education capacity. Our country’s universities have an annual intake capacity of roughly 90,000 and in the 2016 KCSE results, 89,000 students attained the required C+ grade to join university. Coincidence? 

--

About the Author

Bill Mike is a co-founder of Elewa, a company committed to improving the quality of education in Kenya by using training and technology to make education meaningful and enjoyable. 

Besides that, he offers consultancy on ICT in Education matters with a focus on helping organisations in Kenya (and Africa) design learning systems and solutions that take advantage of the power of ICT.