Navigating Success in AI Projects: Key Considerations for Entrepreneurs and Developers

More than 70% of AI projects fail. There are several reasons behind this. I myself have participated in some of these failed projects, and in this article, I want to talk about the reasons behind this failure. This article is suitable for entrepreneurs who want to invest in the AI field, junior machine learning engineers working in AI industries, and software developers involved in AI projects.

Is your desire clear?

The first and most important question that every entrepreneur should ask is: Do I know exactly what I want? It may be a ridiculous question for a lot of people who want to invest in the AI field, but trust me, most entrepreneurs do not know exactly what they want. Before any investment in AI, write your desires, expectations, input, and output on paper. Read it aloud and even ask your friends to read the paper and check if it is understandable. Sometimes, investors unfamiliar with the AI world want a software product, not an AI one, and mostly, they have a foggy idea about their product. So, if you can't write your idea clearly on paper, please don't waste your money. Give time to yourself and ask for help to make it clearer.

How many resources do you have?

After your idea is clear enough, ask yourself how much money you want to spend. Does the project need to be deployed in a real-time environment or not? Can you provide a huge amount of data or not? How much money should you spend on hardware? Is there any copyright problem for gathering data or not?

Maybe you can't answer these questions, but you should ask your tech team to answer them exactly.

A senior software engineer is an essential member of every AI project!

The role of a senior software engineer who knows the back-end, design patterns, and databases is essential. Unfortunately, most ML engineers underestimate these skills, and consequently, the code becomes messy, and for simple tasks, they need tons of energy. However, a senior software engineer knows how to design a system where ML engineers don't need to spend lots of energy on simple tasks.

Agile method is your friend.

I know it is too obvious that you need to build a system with minimum accuracy and then start to reform part of the system, but I met some people who believe that they needed to do one step completely and then go to step two and so on. When you design a pipeline, you really don't need to do one step perfectly and then go to the next step because in some steps, you realize that the previous step needs fundamental changes which were impossible to realize in the previous step.

Non-deep learning models are too essential.

These days I see a lot of people who believe that machine learning is equal to deep learning. It is the deadliest mistake that every ML engineer can make. In most cases, you don't have a huge amount of data and processing resources. Sometimes, simple SVM or even KNN can do more than transformers. Most of the time, employers can't provide good data for you, and sometimes the size of input changes based on user desires. If you only know deep learning, you face a huge amount of obstacles. Remember, in the industry, models have different prices. Deep models are very expensive because they need expensive devices. So learn classic models, know when you should use them, what their limits are, and in which distribution of data these models work better. What are the computation cost and memory cost they have?

Don't use a very recent model for your project!

I know in university, your professor may be mad if you want to propose a paper that belongs to 2019, but in the industry, your employer may be mad if you want to use a deployment from 2022 or 2023. Most very recent models (except for some very few exceptions) are buggy and not tested. You have to use a stable and trustable repository that belongs to 2020 or 2021, not more recent!

Meeting, meeting, meeting.

As an ML engineer, you have to have a meeting every day with your team and every week (at least) with employers. Talk to them and ask them to talk more about the project. Be clear, and if you see their desires are impossible, tell them. Always remember, if no one in the world has done the project (or at least similar projects to the project you want to do), you may not be able to solve the problem.

Conclusion

In conclusion, the success of AI projects hinges on addressing key considerations throughout the development process. Entrepreneurs venturing into the AI field must first articulate their desires clearly, ensuring a well-defined vision before investment. The importance of resources, both financial and technical, cannot be overstated, prompting a thorough assessment of project requirements.

The pivotal role of a senior software engineer in navigating complexities and optimizing efficiency underscores the need for a skilled and experienced team. Embracing the Agile methodology proves invaluable, allowing for iterative improvements and adaptability in the face of evolving project needs.

Recognizing the essential nature of non-deep learning models is crucial, emphasizing the diversity of approaches beyond the allure of deep learning. Practical considerations, such as computation and memory costs, must guide the choice of models in accordance with the project's unique demands.

Caution is advised against embracing the very latest models, as stability and reliability often take precedence over the allure of cutting-edge technologies. Leveraging established repositories from 2020 or 2021 ensures a more tested and dependable foundation.

Lastly, the emphasis on regular communication through meetings fosters a collaborative environment. Transparency regarding project feasibility and managing expectations becomes paramount, paving the way for successful outcomes. In the dynamic landscape of AI, strategic planning, a skilled team, and adaptability emerge as the cornerstones of project triumph.