Winning Strategies for ML Competitions from Past Winners

Knocktober starts in 6 hours. We thought of providing you the winning strategies from our past competitions. Read on, to know the hackathon approach of three top data scientists. They have also shared useful tips & tricks, that will definitely help you to improve your leaderboard position.

Let's dig in and find out what are the ways that can help you win Knocktober.

1.Sudalai Rajkumar (SRK) , Senior Data Scientist, AV Rank 1 (Read detailed article here)

His approach in past competitions:

1. Understanding the problem and dataset
2. Pre-processing the data: Data cleansing, Outlier removal, Normalization / Standardization, Dummy variable creation
3. Feature engineering : Feature selection, Feature transformation, Variable interaction and Feature creation
4. Selecting the modeling algorithm
5. Parameter tuning through cross validation
6. Building the model
7. Checking the results by making a submission

Once you’ve executed these 7 steps, a basic framework will be ready to do more experimentation. Further, you can concentrate more on:

1. Feature engineering – This is where bigger improvements come from most of the times
2. Building varied kind of models and ensembling them – This will help go that extra mile towards the end

Last but not the least,  we must perform a solid local validation. Else, we might end up over fitting on the public leader board.

Tips from SRK:

1. Understanding the problem – It is really important to have a thorough understanding of the problem that we are trying to solve. Only after we’ve understood the problem clearly, we can derive suitable insights from data to tackle the problem and obtain good results.

2. Structured Thinking – It’s a unique way of thinking through the problems. Being a data scientist, one needs to be more structured in his/her thinking in order to obtain good results.

3. Effective communication of results – Effective communication of derived results is as important as performing the data analysis.

2. Rohan Rao, Lead Data Scientist, AV Rank 4 (Read detailed article here)

His approach in past competitions:

1. Understand the problem / objective you are trying to solve.
2. Understand and summarize what data you have / need.
3. Carefully read about the evaluation metric.
4. Explore and visualize the data, build simple, base models for benchmark.
5. Setup a robust / thorough validation framework consistent with the evaluation conditions.
6. Work on feature engineering and optimizing algorithms.
7. Try out as many different models / ideas as you can.
8. Ensemble / Blend / Stack multiple models.
9. Never hesitate in asking questions, taking help or even teaming up with others.

Tips from Rohan:

1. Gauge the complexity of the problem: Explore the data as much as possible. Plot features, summarize columns, build benchmark models, and during the process, get a sense of the problem, data, time, complexity, etc. And then slowly build a good solid concrete solution by working on one idea after another.
2. Algorithm: I use XGBoost and feature engineering for building ML solutions and it’s been a part of my winning solution for most of the contests I’ve done well in, so a big thanks to the community who are actively developing and improving it each day. I also like Collaborative Filtering techniques, which I’ve implemented very often in my work.
3. Feature Selection Ways: My thumb rule of feature selection is based on CV or Val scores. If selecting a feature improves CV score, I use it, else discard. For large number of features, I usually build small quick models and check variable importance or information gain, and select the top-x from them.

3. Steve Donoho, Top Data Scientist (Read detailed article here)

His approach in past competitions:

• Well, I start by simply familiarizing myself with the data. I plot histograms and scatter plots of the various variables and see how they are correlated with the dependent variable. I sometimes run an algorithm like GBM or Random Forest on all the variables simply to get a ranking of variable importance. 
• I usually start very simple and work my way toward more complex if necessary. My first few submissions are usually just “baseline” submissions of extremely simple models – like “guess the average” or “guess the average segmented by variable X.” These are simply to establish what is possible with very simple models. You’d be surprised that you can sometimes come very close to the score of someone doing something very complex by just using a simple model.

Tips from Steve:

1. Making Predictions: This is an important step that is often missed by many – they just throw the raw dependent variable into their favorite algorithm and hope for the best. But sometimes you want to create a derived dependent variable. 
2. I probably spend 50% of my time on data exploration and cleansing depending on the problem.

Go on & use these tips from the winners and takeaway your first win in Knocktober.

If you still haven't registered, don't waste anymore time.

Register Now

Winning prize amount:

• INR 50K (~$750) - 1st Place
• INR 25K (~$350) - 2nd Place
• INR 15K (~$225) - 3rd Place

All the Best !!