2.5 Years Notes on My Applied AI Research Process

Something suddenly happened

As I was reviewing the ICASSP 2025 papers, something suddenly struck me based on my past paper writing, and presentation creation for my lab research group meetings. I create a proper scoring rubric for grading the papers for ICASSP. It worked wonderfully. All my labmates including my professor appreciated it, although it was hard work. More than that I learned a hell lot. I took 2-3 hours to read and review a paper step by step as evidenced by the following review snippet, I shared with you.

This can help aspiring researchers

I have always loved research. Research gives me that space, where I can understand something deeply, and then go “Aha!”. Research is very special to me. In my school days and college days, no one could guide me step by step. Professors told me that the students should be learning the process themselves, and I agree with that now for I learned it, now in my Applied AI research area. However, in this age of Large Language Models, the education system needs to get perturbed, where every enthusiastic and determined student should be taught how to do basic research and its processes so that they can think beyond and do something even better. It takes commitment, and patience to build one’s foundations step by step properly and do research with an unwavering spirit of passion. I share my 2.5 years of learning how to do research in Applied AI.

In the last week, I have intensely written about the Applied AI research process, and today I decided to pin down my thoughts properly step by step. I have made four images to explain the process and help you understand colorfully. :D The above picture summarizes them all together. I gave all my notes so that you decrypt through it, and do some mental exercises. Come on!

It includes two aspects:

• The Full Research Process Step by Step
• Organizing Code for Deep Learning Experiments

I have written about these in detail in my LinkedIn posts:

• ??.?? ?????????? ?????????? ???? ???? ?????????????? ???? ???????????????? ??????????????
• ?????? ?????????????? ???? ???????????????? ??????????????, ?? ??????
• ???????????????? ???? ?????????? ???????? ?????????????? ???? ?????????????????
• Motivation to Innovate & Research
• ?????? ???? ?????? ???????? ?????? ?????? ???????????????????????? ???? ???????? ???????????????? ?????????????
• ?????????????? ???? ???????????????? ????????????????
• My First Post on my Research Process (which people liked)
• ????????++ ???????? ???????????????? ?????? Core ?????????????????????? ???? ?????????????? ????????????????
• My ?????????? ??????????/?????????????????????? ???? ???????? ????????????????
• ???????????? ?????????????????????? ?????? ?????????? ???????? ?????????????? ???????????????? ???? ??????????????
• ???? ???????? ???????????????? ???????????????????????? ??????????????
• Building Domain-Enriched Deep Learning Algorithms
• My First Post on Deep Learning Code / Experimentation
• ??-???????? ???????????????? ???? ?????????? ?? ?????????????? ???????? ??????????????
• Coding GPT2 Architecture from scratch (inspired by Andrej Karpathy)
• ???? ?????????? ???????????????? ???? ???????????????????? ?? ???????????? ?????????????? ???????? ?? ???????????????? ?????????? ???? ?????????????? (I did U-Net)

The whole reason for giving quite a few links is to make you understand that I went through this process, and documented the journey to build my foundations in Machine Learning. Yes, my background in Mathematics, Probability, and Statistics was strong. However, I have to learn Machine Learning, Deep Learning, Programming, Data Structures, and Algorithms while doing my research. This process is not for you if you are looking for a short-term investment. If you are planning to invest yourself in research for the rest of your career, this post is for you. Read along.

My Research Process: A High-Level Overview

1. Foundations First

Before diving into applied AI research, building a strong foundation is essential. Here’s what I focus on:

• Core Skills: Mathematics, Probability, Statistics, Data Structures, Algorithms, and Machine Learning Fundamentals.
• Programming Tools: Python and PyTorch—these are indispensable in my workflow.

2. Implement Foundational Models

Learn by doing. Code the foundational models in your problem domain (discriminative or generative) from scratch. I recommend starting with the Hugging Face course, which offers excellent resources for implementing state-of-the-art models.

3. Structured Research Approach

Each research endeavor follows a clear yet flexible structure:

1. Understand the Data and Problem: What are we solving? What is the data like?
2. Analyze Foundational Models: What models suit this problem best?
3. Adaptation: Tailor the models to fit your data and problem.
4. Experiments and Results: Iterate, test hypotheses, and refine.

At the end of each week, I consolidate my findings and send them as part of a newsletter. This discipline ensures consistent learning and sharing.

A Day in the Life of My Research Journey

The essence of my research day includes the following steps:

1. Read Papers: Start with new publications in top journals like MICCAI, CVPR, and NeurIPS.
2. Explore Code: Check GitHub repositories for promising papers and experiment with them.
3. Modify Code: Adapt these implementations to my research problem.
4. Run Experiments: Write scripts and let them compute results overnight.
5. Analyze Results: Evaluate the outputs and integrate them into my research story.
6. Creative Thinking: Spend significant time thinking—connecting ideas, troubleshooting, and questioning.

When stuck, I return to steps 1 and 2 for inspiration. Weekends are for sharing knowledge and resetting for the week ahead.

For Beginner Researchers

Getting Started with Papers

• Start Small: Explore papers in top journals within your domain.
• Understand Structure: Analyze sections like Introduction, Past Works, Methodology, and Experiments.
• Learn Fundamentals: Dive into foundational architectures or methods you don’t yet understand.
• Recreate or Reproduce: Code the methods from scratch to internalize the details.

Training Your Research Mindset

• Identify data representations and estimators for your problem.
• Understand the properties of estimators, including robustness, low-data performance, and relevant metrics.

Advanced Research: Niche Down

1. Domain Focus

Select a domain—be it medical imaging, computer vision, or NLP—and explore journals and conferences in that area.

2. Trend Analysis

Identify patterns in methodologies and experiments across a set of 10-15 impactful papers.

3. Mentorship and Collaboration

Engage in discussions and debates on ideas, much like the ancient Gurukul learning system in India.

Tools and Resources

• Books: Understanding Deep Learning by Simon Prince, Introduction to Deep Learning by Sebastian Raschka.
• Online Courses: Hugging Face Course link.
• Frameworks: Build foundational models like UNet, ResNet, Vision Transformers, and GPT from scratch.

Final Thoughts

Research is a journey of both structure and serendipity. By following a disciplined process, you create space for creativity to flourish. The joy of discovery—connecting disparate ideas, solving problems, and pushing boundaries—is unmatched. Remember, progress takes time, but the reward is a deeper understanding of your field and the satisfaction of contributing meaningfully to it. Let me know your thoughts, and feel free to share your ideas and challenges.

Happy researching! ??