You're working with machine learning. Are you making the most of your time?

It's not just about spending hours coding and experimenting with algorithms. To truly excel in this domain, it's essential to adopt a strategic approach that maximizes productivity and ensures effective problem-solving. like 1. Define Clear Objectives 2. Data Understanding and Quality Assessment 3. Selecting the Right Algorithm 4. Evaluation Metrics 5. Continuous Learning and Improvement 6. Background Studies and Research 7. Experimentation and Iteration 8. Collaboration and Knowledge Sharing

ML Practitioner's major time is devoted to data pre-processing- sampling, profiling, cleansing, Algorithm selection, and Model Building. The next important component is parameter tuning techniques which are reflected in the model deployment into the cloud, the optimized model will reduce the cost implications.

As a PhD and ML researcher, I devoted my major time on data collection, data pre-processing, data transforming, data cleansing and data sampling. The next important phase is selection of algorithms based on use case or objectives. Once identified set of ML algorithms, need to train and test on sample data and compare the results. Based on observations, best algorithms can be selected. This process is iterative and major time taking.

Defining the problem statement properly on paper is half the problem solved. I've seen so many projects fail or suffer from Scope Creep just because the stakeholders never specified the problem on paper.

1. Define the specific problem you aim to solve with ML. 2. Identify available data and additional data requirements. 3. Choose metrics for evaluating project success. 4. Focus efforts on goal-oriented strategies. 5. Select appropriate ML tools and techniques. 6. Plan data collection and preprocessing methods. 7. Establish clear project milestones and timelines. 8. Involve stakeholders in goal-setting for alignment. 9. Anticipate challenges related to data or implementation. 10. Continuously refine goals based on project insights.

Create automation scripts or better with AutoML tools. There are a whole sets of MLOps to help to make best use of your time. Enjoy.

1. Automate data cleaning and preprocessing tasks. 2. Utilize scikit-learn for efficient data manipulation and modeling. 3. Leverage keras for deep learning projects. 4. Employ mlflow for tracking experiments. 5. Use streamlit for creating interactive visualizations. 6. Streamline feature engineering to save time. 7. Opt for pre-built models for initial tests. 8. Integrate tools for automatic hyperparameter tuning. 9. Explore libraries for various ML tasks to avoid redundancy. 10. Focus on innovation rather than manual, repetitive work.

Automate but verify. While you should automate as much as possible in order to save time during development, it is crucial to take a closer look at the inputs and outputs of your data pipeline. Make sure to really understand what you are trying to do, and consider if it makes sense. This is especially important when you have a limited amount of data available during development, as specific patterns in your data may not be contained or be overrepresented in a small dataset.

Utilizing libraries such as scikit-learn and Keras is beneficial, but in my opinion, incorporating functional programming principles is essential for ensuring reliable repeatability and automation.

Automate and streamlining seems like a great idea. However, what I have found useful is to first set an expectation on yourself. I would look at the data, listen to audio and look at the picture to see if I can train my eyes and find the correct answer. If I can tell - I will make a machine to tell. If not - the machine might work might not work. It is very important to know for yourself to know the data quality. Sometime, people get ahead of themselves and automate the process based upon some library that does it for them. I love pytorch - everything is out there - laid out in open for me to inspect when things don't work. Remember - anyone can write and train a model. To figure out scenarios in which things go south is the key.

1. Stay updated with latest ML research and trends. 2. Read blogs, listen to podcasts, and subscribe to newsletters. 3. Engage with ML books and academic papers for deep insights. 4. Join online ML communities and forums. 5. Use Stack Overflow for resolving technical issues. 6. Participate in Kaggle competitions to hone skills. 7. Explore datasets on Kaggle for practical experience. 8. Follow ML content on Medium for tutorials and articles. 9. Share ideas and seek feedback from peers online. 10. Continuously learn from others' experiences and insights.

Data cleaning in ML is time-consuming but important for accurate models. It should be automated using libraries or functions whenever possible. Popular libraries for this task include Pandas, NumPy, and SciPy, which offer functions for handling missing values, removing duplicates, and transforming data types. Also, Scikit-learn can be used for encoding categorical variables, and feature selection. TensorFlow and PyTorch offer preprocessing modules for image augmentation and sequence processing.

ML intersects with various disciplines, including computer science, statistics, mathematics, and domain-specific areas like healthcare and finance. Learning from experts in these diverse fields offers valuable perspectives, enriching problem-solving approaches and fostering innovation. Given the complexity of ML projects and the volume of data involved, collaboration enables individuals to pool resources, share expertise, and tackle challenges more effectively. Also this helps individuals gain insights into potential pitfalls and best practices in ML. By understanding common challenges like overfitting, data leakage, or misinterpretation of results, individuals can enhance their decision-making and avoid costly mistakes in ML projects.

Chatgpt est un excellent moyen d'aller vite aussi. Ce qui est intéressant, c'est qu'il suffit juste de demander du code et il vous le sort. Après, il faut conna?tre ce qui se cache derrière, c'est aussi ?a l'IA responsable. Et si il y a des erreurs, il faudra le corriger.

Despite learning from other external sources, from an organisational perspecitve it is important share your experience, knowledge and learnings also internal. Using ML within organisations is not an agenda which you can easily implement and then use it, but instead it rather compares to a mindset, which the whole organisation has to adopt and which also has to be integrated in organisational processes and structures.

I'm constantly experimenting with new methods and iterating on my approaches to ensure I maximize productivity and make the most of my time.

See if you can build a model for the problem that has any predictive power that is routed in statistics such a simple regression or decision trees. You might find in most cases you can go surprisingly far without involving deep learning and the statistical checks can be helpful in informing your inputs for a deep learning algorithm, This experimentation will help you narrow down the exact outcomes for which you want to employ deep learning which is the costliest and hardest to explain form of machine learning. Your statistical model outputs could also serve as input for a neural net, shortcutting some of the inference requirements of a such a network.

On top of experimenting and iterating, do not be shy about failing quickly. If a particular model or workflow is not showing results do not be afraid to shut it down and learn lessons. Ask the question, "is it worth the investment" and "Is AI the correct approach".

Working with product/business stakeholders on something that has vague definitions of done can be tricky. Less mature organizations won't understand that having a model does not indicate that you've solved the problem, only that you've made an attempt to. These orgs will often be very good at understanding breadth metrics (recall), but quite bad at understanding quality reqs (precision). In such cases, evangelizing DS specific process will be important. Google "CRISP-DM" and "Microsoft teams DS process" and become well versed in how they differ from trad SDLCs. And be prepared to spend some effort in A/B testing, as it will be rare that product will be able to tell you a priori where to be on your ROC curve, or even if it's good enough.

The trade-off between exploitation and exploration is at its peak when working on ML projects. With experience an ML engineer starts to better understand when to come up with new innovative ideas and when to use existing solutions. The definition of a clear workflow is very important to validate the approach or improve upon an existing one. A good tracking of the different experiments is also very important.

Achieving a balance between model quality and computational efficiency is essential. Aim for both accuracy and speed in machine learning pipelines. Optimise model complexity and resource allocation to strike the right balance for specific requirements.

Balancing quality and speed in machine learning hinges on setting clear goals with stakeholders and maintaining flexibility in your approach. Overemphasis on either aspect can lead to missed targets or wasted resources. Adopt agile methodologies like CRISP-DM, tailor your development lifecycle to suit data science projects, and familiarize yourself with the nuances of machine learning processes. Emphasize the importance of precision alongside recall to stakeholders, and prepare for extensive A/B testing. This helps in determining the most effective position on your ROC curve and ensures the model meets the desired thresholds for both performance and speed.

1. Prioritize tasks and features to optimize both quality and speed. 2. Set realistic expectations to achieve a balance between accuracy and time. 3. Monitor progress and performance regularly to make adjustments as needed. 4. Avoid over-engineering or over-optimizing to maintain efficiency. 5. Use appropriate resources and tools to streamline the process without compromising quality.

The start and end should be well curated. For example, First try to fix the dataset by performing Data Mining and EDA. This step takes the maximum time but it is worth it. Second, work on the model. Try different models to experiment, then try to improve the accuracy and quality. Third, once you reach the place where you are not aware of any more models, try focusing on changing the features and KPIs instead of building fancy models. Finally if you have more time, work on building more models with your experimental knowledge from the past. Do not forget to document and measure your progress on a regular basis, analyze the performance and adjust or change the strategies accordingly.

In the fast-paced world of machine learning, balancing speed and quality is essential. While I strive for quick results through rapid prototyping and iteration, I never compromise on quality. Thorough testing, validation, and model tuning are non-negotiables to ensure that my solutions are both efficient and effective.

Think about whether you should be using machine learning at all. I was a freelance ML consultant for 16 years, and at least half the time the right answer for the client didn't use ML.

Keep in mind that machine learning includes more topics than neural networks and AI. A big, fancy AI model is not always the right approach. There are other simpler models that have the benefit of being much more explainable, such as a Random Forest model or Naive Bayes. The first thing to think about when approaching a problem that could be solved with machine learning is what model would be the most beneficial. That includes interpretability, not just the overall performance of the model.

Augment the user on his/her job to be done. ML is a great technology but : adds value? Start with the person or user that you want to “augment”, this means ML must help in the use case, in the situation where this technology excels. Example: use ML with cloud but costs are still high vs use ML with tinyML at a lower cost but still bringing a ton of benefits for user. Another example: Does ML simpler algorithms can augment the user instead of forcing the use of new deep learning Transformers? Recommending or Knowing this will be named “ML Strategic Thinking”

In machine learning, time optimization goes beyond computational efficiency. It's about driving innovation and addressing complex challenges. Balancing algorithmic depth with practical utility is key. Techniques like transfer learning, ensemble methods, and meta-learning boost model training and accuracy. Agile and DevOps methodologies enable smooth ML model integration, promoting continuous enhancement. Thus, maximizing time in ML isn't just about speed but cultivating innovation, collaboration, and adaptability in AI's dynamic landscape.

Don't stop and don't give up. The objective is not to just run ML models but to understand the business problems and then lay out the roadmap best suited for you. It necessarily need not be Data Science always but can also be solved using heurestics or both. And if you did perform a regression or classification model, keep iterating and hypertuning to reach the zenith of your hypothesis else no.