登录查看更多内容

What are the best ways to ensure the reliability of AI and machine learning models?

由人工智能和领英社区提供技术支持

Artificial intelligence (AI) and machine learning (ML) models are powerful tools for solving complex problems and generating insights from data. However, they also pose challenges for ensuring their reliability, especially in critical domains such as health, finance, or security. Reliability refers to the ability of a model to perform consistently and accurately under various conditions and scenarios. In this article, you will learn about some of the best ways to ensure the reliability of AI and ML models, from design to deployment.

此文章中的业界达人

由社区从 4 条内容中精选。了解更多

Ruben Vereecken

Fractional CTO in Data & AI | 3x Founder | Ex-Google | Ex-PhD in AI/AGI

1 Define clear objectives

The first step to ensure the reliability of AI and ML models is to define clear and measurable objectives for what you want to achieve with them. Objectives should be aligned with the problem statement, the data sources, the stakeholders, and the ethical principles. Objectives should also be specific, realistic, and quantifiable, so that you can evaluate the performance and impact of your models. For example, if you want to build a model that predicts customer churn, you should define what churn means, how to measure it, and what are the acceptable levels of accuracy and error.

添加您的观点

2 Choose appropriate methods

The second step to ensure the reliability of AI and ML models is to choose appropriate methods for building and testing them. Methods include the algorithms, frameworks, tools, and techniques that you use to design, train, validate, and optimize your models. Methods should be suitable for the type, size, and quality of your data, as well as the complexity and uncertainty of your problem. Methods should also be transparent, explainable, and reproducible, so that you can understand how your models work and how to improve them. For example, if you want to build a model that classifies images, you should choose a method that can handle high-dimensional and noisy data, such as deep neural networks.

添加您的观点

Ruben Vereecken

Fractional CTO in Data & AI | 3x Founder | Ex-Google | Ex-PhD in AI/AGI
举报内容
I find it useful to first think of the type of prediction you want to do. Generally, this will fall under two categories: 1. Regression – when you're estimating a number, like the customer churn example 2. Classification – you're labelling something, like "what does this picture show" This distinction will guide the rest of your methodology, so it's good to make early on. Bonus: interestingly, LLMs and other GenAI methods follow the same principles as classification methods. Classification methods build a distribution of likely labels, then draw the most likely one. All LLMs do is continuously draw the most likely next word, given a history of all the previous words.

已翻译

赞

3 Implement best practices

The third step to ensure the reliability of AI and ML models is to implement best practices for developing and deploying them. Best practices include the standards, guidelines, and procedures that you follow to ensure the quality, security, and efficiency of your models. Best practices cover aspects such as data collection, preprocessing, cleaning, analysis, feature engineering, model selection, tuning, evaluation, documentation, version control, monitoring, and maintenance. Best practices also involve following ethical and legal norms, such as respecting privacy, fairness, accountability, and transparency. For example, if you want to build a model that recommends products, you should implement best practices that ensure the diversity, relevance, and safety of your recommendations.

添加您的观点

Ruben Vereecken

Fractional CTO in Data & AI | 3x Founder | Ex-Google | Ex-PhD in AI/AGI
举报内容
Most non-regression methods estimate a probability distribution to draw from. Classification methods simply draw "the most likely label" from a distribution, LLMs draw the most likely next word, etc. This is important to know because you can't really extrapolate beyond known data with probability distributions, only interpolate. This is why training data biases are so dangerous: any pattern in the training data will be reflected in the estimated probability distribution, which in turn will reflect in future predictions. So if your training data for your salary prediction model includes a pattern of women getting paid less than men... then that's exactly what it will keep predicting.

已翻译

赞

4 Perform rigorous testing

The fourth step to ensure the reliability of AI and ML models is to perform rigorous testing before and after deploying them. Testing involves checking the functionality, performance, robustness, and usability of your models under different scenarios and conditions. Testing should be done on various datasets, platforms, environments, and users, to identify and fix any errors, bugs, biases, or vulnerabilities. Testing should also be done periodically and continuously, to monitor and update your models as new data, feedback, or requirements emerge. For example, if you want to build a model that detects fraud, you should perform testing that covers normal and abnormal cases, as well as potential attacks or anomalies.

添加您的观点

Ruben Vereecken

Fractional CTO in Data & AI | 3x Founder | Ex-Google | Ex-PhD in AI/AGI
举报内容
One basic principle of testing that is often violated or forgotten: always keep separate data for training, validation, and testing. A common data split is 50/30/20 - Training: actual training of the model, this should be the biggest chunk of data - Validation: model selection, eg how many layers in your neural network, how big are feature vectors, etc - Testing: how well does the trained model perform on completely unseen data? One of the mistakes made by engineers that I see most often is changing the model based on performance on the testing data. That's basically using testing data for training. Finally, it's key to ensure that all data partitions follow the same distribution.

已翻译

赞

5 Seek feedback and collaboration

The fifth step to ensure the reliability of AI and ML models is to seek feedback and collaboration from various sources and perspectives. Feedback and collaboration can help you improve the quality, accuracy, and usefulness of your models, as well as avoid or mitigate any risks, harms, or unintended consequences. Feedback and collaboration can come from your peers, mentors, experts, clients, users, regulators, or the public, depending on the context and purpose of your models. Feedback and collaboration can also involve sharing, reviewing, or auditing your models, data, code, or results, to ensure their validity, reliability, and reproducibility. For example, if you want to build a model that diagnoses diseases, you should seek feedback and collaboration from medical professionals, patients, researchers, and ethicists.

添加您的观点

6 Learn and improve

The sixth and final step to ensure the reliability of AI and ML models is to learn and improve from your experiences and outcomes. Learning and improving involve analyzing, reflecting, and acting on the results, feedback, and lessons that you obtain from your models. Learning and improving can help you enhance your skills, knowledge, and confidence, as well as discover new opportunities, challenges, or solutions. Learning and improving can also involve updating, revising, or retiring your models, depending on their performance, impact, and relevance. For example, if you want to build a model that predicts stock prices, you should learn and improve from the market trends, customer behavior, and economic factors that affect your predictions.

添加您的观点

7 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?

添加您的观点

Ruben Vereecken

Fractional CTO in Data & AI | 3x Founder | Ex-Google | Ex-PhD in AI/AGI
举报内容
Too often, scientists get excited by the latest shiny thing, with everyone piling on to try the slightest variation of the latest model with the latest performance. That's incredibly incremental work, and very little of it sees the light of day. Instead, focus on the underlying principles. What are the benefits of one model over another, really? What is it doing under the hood? Is it estimating a probability distribution? Are the results it arrives that significantly different from a simpler approach?

已翻译

赞

Computer Engineering

+ 关注

给文章评分

我们借助人工智能创建了此文章。您认为这篇文章怎么样？

很棒不太好

举报此文章

查看全部

What are the best ways to ensure the reliability of AI and machine learning models?

1

2

3

4

5

6

7

1 Define clear objectives

2 Choose appropriate methods

3 Implement best practices

4 Perform rigorous testing

5 Seek feedback and collaboration

6 Learn and improve

7 Here’s what else to consider

Computer Engineering

给文章评分

感谢您的反馈

更多Computer Engineering相关文章

更多相关阅读内容

What are the best ways to ensure the reliability of AI and machine learning models?

1

2

3

4

5

6

7

1 Define clear objectives

2 Choose appropriate methods

3 Implement best practices

4 Perform rigorous testing

5 Seek feedback and collaboration

6 Learn and improve

7 Here’s what else to consider

Computer Engineering

给文章评分

感谢您的反馈

查看其他技能