What are the most common failures in Artificial Intelligence and how can you avoid them?

Data is one of the most critical yet frequently overlooked components in AI. AI researchers, product managers, and engineers should dedicate a considerable amount of time to understanding their data.

A lot of organizations have tried to treat data as a project. It can’t be treated as a project; it has to be treated as a function. Until organizations start to treat data as an imperative operational unit, they’re going to continue to struggle to get any kind of consistency and quality in data. Such strategic missteps may signal an ongoing issue at the C-level, with company leaders recognizing the importance of data and analytics but falling short of making the strategic changes and investments necessary for success.

One of the common problems that data scientist find themselves in is to pursue complex models before identifying if the data contains the 'promised' characteristics and also if the data and output have some corrolations. One easy way to check this is to use a simple xgboost model just to have the feel of the data. Even though it seems redundant work as you will implement eventually more suitable model it is always helpful to begin simple. There are so many things that can get wrong in dl models (normalization, architecture..) so validating data before going there save you a lot of time

This is a major problem with large enterprises to keep the data clean so the AI models bring correct responses. Building the right ETL processes can help clearing the garbage data and keep the curated data separate from raw data which further can be fed to machine learning models. This whole process can be super exciting but at the same time you need to keep a close watch on your data pipeline management to keep your cost under control.

The most underestimated part is the data. It is important to understand the data and its nature before preprocessing to decide what the input to and output from the model should be. However good a model might be, the data quality highly determines the model quality. Understanding the meaning of each parameter in the data and the factors affecting it seemed unnecessary when I started my first project but as the project progressed, I understood how that was probably one of the most important sections. Significant time is to be allotted for properly studying and understanding the data before deciding the preprocessing techniques.

When looking at predictive AI models, overfitting can be a consequence of immediately starting with the latest, shiniest, or hyped model with too little knowledge of the data and problem at hand. I always suggest to start with the basic models like linear regression, naive Bayes, decision trees. This will give you a first insight on how the data behaves, how input variables should be treated and where it needs improvement most. After this first set of models you can start applying other methods and extend from there. These basic models have a lower chance of overtraining and also give you a good reference when applying all trained models on a final independent validation set.

Testing on unseen data is a must. AI tends to memorise and cheat (i.e. overfit) even more then humans. Understand how to do train-test splits well, keep your holdout set clean and regularise whatever you can. Using large pretrained models with minimal tuning counts a regularization. Leverage pretrained models whenever possible.

In my experience: Overfitting is a common yet avoidable pitfall in AI, demanding strategic data handling. Properly splitting data into training, validation, and test sets has been fundamental. Regularization techniques, such as dropout, weight decay, or early stopping, are crucial. Employing cross-validation, ensemble methods, or transfer learning can further safeguard against overfitting, ensuring models are robust and adaptable. These steps have been invaluable in enhancing the performance and reliability of AI models in various projects, ensuring they generalize well to new data

Imagina que estás utilizando imágenes médicas para diagnosticar enfermedades como el cáncer a través de un modelo de aprendizaje profundo. Si el modelo se sobreajusta, podría aprender a identificar características irrelevantes en las imágenes de entrenamiento que no son indicativas de la enfermedad. Esto podría resultar en diagnósticos incorrectos cuando se presenten nuevas imágenes de pacientes.

Overfitting occurs when the results obtained are simply too good to be true. This can happen if you have too many descriptors as you may end up just recognizing faul patterns from the noise in your data. To avoid an overfitted model I would suggest the following validation setup: 1. Split your dataset (90/10) or (80/20) or others, depending on the case. The bigger part of the split will be used for training and internal validation, the smaller part of the split will be for external validation. Sometimes it could be recommended to have several split configurations and try them all out. 2. Use Cross-Validation! If possible, Leave One Out Cross Validation will help you avoid overfitting the most. 3. Dont forget to use the external test!

The ability to explain the rational behind the outputs a model generate is one important aspect to consider AI models as reliable ones. The models have to be "glassboxes" than "blackboxes"

Lack of knowledge of the data in the minds of model creators, especially that is accumulated over time is the biggest challenge. Pairing business owners with tech is a way to overcome this

From my personal journey: The enigma of AI's "black box" often leads to skepticism, especially in crucial sectors like healthcare or finance. My projects have shown that demystifying AI through XAI techniques, such as feature importance and saliency maps, is key. Integrating feedback loops and collaborative oversight has proven to strengthen AI's reliability and societal acceptance. Ensuring models are interpretable not only builds trust but also clarifies their decision-making process. This blend of transparency and collaboration underscores the importance of human oversight in AI development.

Desafios: Desconfian?a: A falta de explicabilidade pode gerar desconfian?a e hesitancia em rela??o à IA. Problemas de responsabilidade: Se as decis?es da IA n?o forem compreensíveis, torna-se difícil determinar responsabilidades em caso de erros ou falhas. Solu??es: IA Explicável (XAI): Técnicas como importancia de features, mapas de saliência e contrafactuais podem ser usadas para explicar as decis?es da IA. Abordagens Human-in-the-loop: Envolver humanos no processo de IA, seja através de feedback, colabora??o ou supervis?o, pode ajudar a garantir que os modelos sejam alinhados com os valores humanos e que as decis?es sejam compreensíveis e responsáveis.

Explainability approaches can be both helpful and misleading. It always depends on who needs an explanation and for what purpose. For example, if developers want to identify weaknesses and errors in the system, or if users want to investigate how changes to their inputs affect their outputs, the right explainability concepts can be a great support. However, scientific findings are increasingly showing that the same approaches are unsuitable as analysis tools for detecting discriminatory system behavior. As a provider of an "explanation," it is therefore always important to know what insights a specific recipient needs. Conversely, recipients should critically assess the explanation's relevance and avoid misleading interpretations.

Artificial intelligence is based on knowledge composed of both objective facts as well as opinions, subjective perception and individual experience. One trap is false expectations. Models are trained by diverse sources. Thus, they help to find and offer options in an interactive and culture agnostic way but not to deliver truth. Expect AI to assist you, expect proposals, not directives how to behave correctly. Another trap is the temptation of cross culture standards. People are individuals. Thus, use AI to harmonize knowledge and establish compatibility over conformity, to gain better communication and collaboration. A central almighty AI is a risk for our democracy. Let’s support private models by fostering sovereignty over our profiles.

In addition to, AI has imperfections in social and environmental risks — a. Algorithmic bias b. GHG Emissions/Carbon Management c. privacy/cyber risks d. Cyber hygiene/literacy of B2B, B2C, or B2G e. rural connectivity/inclusion — access to infra. in rural geographic terrains f. Holistic Risk audit/mapping at firms g. Access

While AI systems aim to streamline processes and enhance decision-making, they can sometimes produce unintended consequences. These can range from biased outcomes to ethical dilemmas and even safety concerns. For instance, biased algorithms may perpetuate discrimination or unfair treatment based on gender, race, or socioeconomic status. To avoid such pitfalls, it's crucial to prioritize ethical considerations throughout the AI development lifecycle. This includes conducting thorough impact assessments, implementing safeguards against bias, and ensuring transparency and accountability in AI systems. Additionally, ongoing monitoring and feedback mechanisms can help detect and address unintended consequences.

Unintended consequences in AI can manifest as algorithmic bias, perpetuating discrimination based on race, gender, or socioeconomic status. They can also lead to privacy violations, like an AI crime reduction system that indiscriminately monitors individuals. To avoid these challenges, we must conduct thorough impact assessments, implement fairness constraints, and ensure transparency and accountability. Adversarial testing techniques can help identify potential vulnerabilities. When we apply ethical foresight and diverse perspectives, we harness AI while minimizing the risk of harmful outcomes. It's not just about what AI can do, but what it should do. Just scan recent news headlines to understand the unintended consequences of AI.

- Humans can understand and respond to emotions, while machines lack this capability, making them less effective in emotionally charged situations. - Humans grasp complex social and cultural contexts, vital for nuanced decisions, whereas machines rely on algorithms and data, lacking human depth. - Machines follow rules but lack human moral reasoning and ethical judgment.

Resistance to change is the most common failure, not only for AI, but for all transformations at any organization. Successful change requires successful individual changes. The unit of change becomes the individual. Individual change is always challenging. It is hard to break with what we have been doing until today and that has worked for us, to abandon the familiar area in which we have lived, to venture into a future that at best is unclear and uncertain. One of the biggest risks is that employees do not adapt to the new technologies and processes that the company introduces. This can make even the most detailed transformation plans ineffective. Therefore, the need to emphasize Change Management is crucial at any stage.

More often, the resistance to the change and in particular about the introduction of AI comes from the fear factor. It is important to understand and make people aware that AI is going to augment what people do and not replace them,

To address resistance to AI solutions within an organization, the approach to be followed revolves around the following pillars: 1) Educating stakeholders about AI benefits 2) Addressing job security concerns 3) Ensuring transparency and ethical AI practices 4) Providing training and support 5) Starting small and scaling progressively 6) Building a culture of collaboration 7) Investing in knowledge and experience with data & prioritizing data privacy

Data quality isssue is the most commom isssue which may lead towards the failure the neural network in terms of performance. One must pay focus on the data quality.

Focus on adding AI to processes that are time-consuming to or frustrating on the one hand, but easy to check and improve from an initial solution. That way you can transform resistance into a super-power for whoever is currently working on this task.

Continuous improvement: AI isn’t a set-it-and-forget-it deal. It needs upkeep, retraining, and tweaking as data and environments change. Stay on top of it, and keep improving.

Artificial Intelligence (AI) faces challenges and occasional failures, particularly in specialized industries. The key issues I have faced: Lack of Relevant and Sufficient Data: AI models rely on high-quality data but often face limitations due to insufficient or irrelevant data for specific scenarios, leading to biased models. Cultivating diverse datasets, implementing data governance practices, and generating your own data when necessary can counteract this. Skepticism Toward AI: In areas such as public safety and emergency response, AI is often met with skepticism due to the need for fast, reliable decision-making. Position AI as a tool for faster, informed decisions while maintaining human oversight.

I started off in AI when knowledge-based systems and expert systems were still (part of) the state of the art. I taught logic and reasoning, as well as search algorithms. Todays systems work differently. They are learning systems and they learn from human data. When ChatGPT came out, I saw a huge gap in expectation and what people got. People are expecting "logical" formal reasoning machines when they think of AI. However, the new AI is far from that. It's actually more human than people expect it to be. It's been shown that without a proper design of the prompt, GPT-3 and GPT-4 have human flaws in reasoning and arithmetic isn't their strong suite. It's not a bug, it's a feature, though. Systems that learn from humans reason like humans.

From my perspective - having Human-in-the-Loop for feedback Loops drastically enhances the model performance. These humans could be a subject matter expert or could simply be live end users. And these recommendations or corrections should be fed back to the model for training and continuous improvement.

As a digital transformation consultant, I often encounter individuals who initially approach AI with excitement and curiosity, but quickly become disillusioned when they realize that AI cannot deliver perfect results in every situation. It's important to understand that AI is still in its early stages, and while it has already made significant progress, it will continue to evolve and improve over time. Therefore, it's crucial to maintain a long-term perspective and keep an eye on AI developments, as it's poised to play a critical role in shaping the future of technology and business.

Adopting AI technology is a big game changer. Well into our path of adoption, we realized we would fail from the start if we did not re-tool our HR team to operate the AI. This means that, unlike other technology applications where IT manages the technical side, we needed domain knowledge on the "back end" to ensure the outputs and experience of conversational AI to make it work to our standard. We needed an HR Automation team who understands the underlying process/experience, the design, and can also control the "dials" on the back end. This is a prerequisite, not something you learn as you go. Otherwise end user adoption will be lost on first impression.

In my view the biggest issues in AI are the anthropocentric terminology, missing public awareness and the resulting exaggerated expectations. With AI becoming integrated into most productivity tools, it is high time that the general public is better informed about what AI actually is (a large number of loosely related algorithms rather than one super-intelligence) and how it works (by extracting statistical relations from data (unsupervised learning), from human feedback (supervised learning), or a combination of these.

In my industry (media and entertainment), common AI failures include biased content recommendations, over-personalization leading to echo chambers, inaccurate content tagging and classification, and failure to capture nuanced audience preferences. To avoid these, diversify training data, implement mechanisms to avoid feedback loops, employ multi-dimensional tagging and classification approaches, and incorporate user feedback for continuous improvement.

AI holds transformative potential for businesses, but its success depends on navigating common pitfalls. By clearly defining objectives, prioritizing ROI-driven projects, ensuring data quality for a robust AI platform, and scaling progressively, we can greatly enhance the likelihood of AI project's success.

It’s imperative processes that still require human interaction are not rushed off to the first AI iterations possible. It’s not difficult to imagine a chain reaction stemming from an unsupervised AI decision that could result in firms realizing catastrophic losses. Automating processes that are easily rolled back is one thing, but roles that require precision and constant judgement should be left alone until we have a more defined understanding of AI’s current capabilities,