Plan with realism, act with hope

I recently conducted an online survey on LinkedIn, trying to understand people's instinctive guesses for the probability of success of various candidates on a tough competitive exam like the IIT selection exams (those involve a pair, actually).

I wrote this:

A popular competitive exam is taken by 1 million (10 lakh) candidates every year. 10000 people are successful each year (that is, the population success rate is 1%). People who take coaching, who have studied in good schools, who are from cities, and who are from somewhat wealthy families have some advantage, but people without these advantages also can (and sometimes do) succeed.

I attach a figure below. There are 5 subplots indicating 102 respondents' probability estimates for 5 situations. Since the probability estimates themselves (expressed as a percentage) vary a lot, I have plotted their empirical cumulative distribution function (CDF) and marked the median as well.

(1) The first question was about the chances of a candidate who is in the top 10% of his/her village school. As you can see, the median estimated probability is 5%. This seems a bit high to me. About 2/3 of India's population lives in villages. India has above 650 million people below age 25, and the population density at younger ages is approximately uniform. Thus, there are about 650/25 = 26 million 18 year olds, so say about 12 million 18 year olds in villages. (A rough estimate.) Say only 10% of them go to high school (a guess; very many girls do not study until class 12, and many boys do not do so either ...). That's 1.2 million. The top 10% of those are 120,000. But many candidates appear twice, so maybe two years are represented in the exam-taking population. Say 2 lakhs. 5% of that is 10000, the entire number of selected people. But actually, city boys are proportionately over-represented in the IIT system. I think, for this problem, the respondents are a little overoptimistic. Many respondents are wildly optimistic.

I think a reasonable estimate may be 2-3%. Let me know in the comments if you agree.

(2) The second question was about the chances of an average city student making a second attempt. For rough analysis, the number of people who get in with the first attempt, and those who get in with the second attempt, are comparable (see the fifth question). Each year, a million students make the attempt. Only 1% succeed. If the exam is any good then it should be highly unlikely for someone who is NOT in the top 5% to get in. Maybe, just maybe, someone in the top 10% may put in a superhuman effort, get very lucky, and get in. But an average student? What are the chances? Negligible.

There is a report on the JEE Advanced of 2018 here:

https://www.iitk.ac.in/new/data/jee-report/1_JEE_2018_Vol_1.1_1.2-28-2-19.pdf

The data there shows the following. A relatively small percentage of all JEE Mains candidates were allowed to appear for the JEE Advanced (under 20%). Of those, in each category (Gen/OBC/SC/ST ... does not matter), the number of people offered seats was less than 10%. Thus, overall, less than 2%, regardless of category. Someone "average", i.e., in the 40-60 percent bracket, does not have a realistic chance even on a second attempt, in my opinion. Even with an advantage coming from being a city person, the median guess of 20% (see figure) is highly optimistic in my opinion. Many respondents are wildly optimistic.

(3) The third question was about a hard working average student at a coaching center. Coaching centers have many lakhs of students. Most of them are hard working. I think respondents got emotionally affected by "hardworking" and "coaching center", but the key word is "average". The average student of a coaching center is hard working and does not get in. Just think of how many lakhs of students there are in coaching centers. The respondents' median estimate of about 37% (or about 1/e) seems wildly optimistic to me.

Just think about it like this. The coaching business trains several lakhs of students each year, and those students pay some lakhs of rupees each year. At a rough estimate, 2 lakh students paying 2 lakh rupees per year, 2 batches at any time, means Rs 8000 crores per year. More than a billion US dollars a year, the national price of two years of hope leading in most cases to disappointment if not heartbreak. But let us not dwell on that.

If the coaching system trains even 2 lakh students, then 1 lakh of them are presumably above average, and there are only 10000 seats anyway. The chances couldn't be better than 10% (10000 out of 1 lakh) overall. But of course, the chances are better for the brighter students and far poorer for the near-average students. I do not see how the chances could be even as high as 5%. Let me know in the comments if you disagree.

(4) The fourth question was about the chances of an intelligent student with average effort at a coaching center. The respondents' median estimate is almost 50%. This is amazingly high.

JEE success requires talent plus labor plus training plus luck. In my time (1985), more than 20% of class 12 at St Xavier's College in Kolkata got into IITs. Today's numbers are not nearly that high, I think. Talent remains the same, luck remains the same. Labor and training (including coaching class strategies) must have made a difference. Now, labor is what converts strategy into performance. Now see the question again.

There is a very large number of students who are intelligent and hardworking and well-trained by a good coaching class. The lucky ones among them do well.

Look at it another way. From personal experience, I do not believe intelligence can overcome lack of extreme effort. Why?

Because I teach the ones who get in. I meet them regularly. They are fairly intelligent, approximately as intelligent as my Kharagpur classmates from my time. Today, as a teacher, I go to class with a piece of chalk, no stick and no shield, and I come out routinely with my shirt still on my back, because I am old and have had time to think about the subject. So how intelligent can they be?

If they were in fact extremely intelligent, I could not face 200 of them and teach a fundamental subject, and set standard exams they sometimes have trouble passing. Occasionally, of course, a few extremely intelligent ones do go through.

I am talking about the middle of the distribution in the IIT Kanpur classroom.

(5) The last question had to with guessing what percentage of successful students actually got in on the second attempt. The respondents' median estimate is 30 percent.

If the entrance exam actually and perfectly measured a static quantity (like inherent talent rather than ability), then you would get detected the first time, the second time would not give you any advantage, and the percentage of repeaters would be zero. If the added year gave you a huge measurable advantage, then many would repeat, and the proportion of repeaters among successful students would be high, maybe well above 50%. If there was both some random error in the exam's assessment as well as some improvement in your ability, then the solution would be more complicated.

If you, a reader, can formulate that problem under interesting assumptions, please post that separately and put a link to it in the comments. For now, I accept the respondents' median estimate of 30%.

Conclusions

I offer some final thoughts. I think our assessment of probabilities, if we are not careful, are driven by emotion when we do not have data.

With data, things are fine. On a sunny morning in May, we do not expect rain in the evening, and the probability is in our favor.

Without data, we face things that are actually unknowable. Suppose you have a nephew who gets 85% marks, seems serious, is willing to go to a coaching class, took the JEE Advanced once and did not clear it, and wants to repeat. What are his chances of success on the second attempt? We can talk of a probability, but it is not a repeatable experiment. The frequentist approach to probability does not work here. Yet, consequences can be serious.

The main point is that JEE is not even that important. It is just one thing.

After studying in some college, should you give some years to UPSC preparations? How many years? When and why and how do you decide to quit? (Face it, most do not clear it; so most must quit.)

Suppose you get a job offer from some small company, your family needs money, your cousin says you can join another big company as a low-paid trainee with a "decent" chance of getting a proper job the next year, you also secure admission to the MTech program in an IIT, and three of your college friends are starting a business and ask you to join. How should you decide which option to select?

Suppose you are working in some company and you have a chance to rise rapidly to a managerial position reporting to s specific person, or a chance to grow more slowly with a larger group of employees on a less exciting career track, what criteria should you use to evaluate your options?

These are difficult choices. Due to personal tastes, I prefer a somewhat more conservative approach. But if you are not being so conservative, exactly how conservative are you being?

How does one plan for an unknown future, without data, in a time when there will surely be more workers than work?

I do not know a definitive answer. I think a general principle is to develop competence and a reputation for being reliable, and earn the trust of one's superiors.

Thank you for reading so far. If you would like to see more of my general musings about building and sustaining a career in engineering, please take a look at my book, whose message is encapsulated in the title of this article: Plan with realism, act with hope.

The book is here, if you want to take a look.

https://www.amazon.in/Sustain-Career-Engineering-Anindya-Chatterjee/dp/1637816235/ref=sr_1_1?dchild=1&qid=1617911153&refinements=p_27%3AAnindya+Chatterjee&s=books&sr=1-1