Finding the balance between reducing false positives and increasing true positives
Prospero AG
20 years' experience developing and implementing business solutions based on Predictive Analytics.
The most important feature of a predictive analytics platform is the generation of high-quality prediction models that have the best ratios of true and false positives. No model is perfect, but many organisations compromise by implementing models that lean towards over-detecting (disproportionate false positives) or under-detecting (few false positives but many missed true detections).
Finding the Balance
The most important feature of a predictive analytics platform is the generation of high-quality prediction models that have the best ratios of true and false positives. No model is perfect, but many organisations compromise by implementing models that lean towards over-detecting (disproportionate false positives) or under-detecting (few false positives but many missed true detections). This is key for optimising business processes as finding a good balance through high quality prediction models will lead to improved efficiencies in business processes, improved data-driven decision making, increased accuracy in forecasts and predictions, better understanding of market requirements leading to innovation and a clear competitive advantage.
Trade-Offs and Thresholds
The better the quality of a predictive model, the better the detection rate and the ratio of true to false positives. Adjusting the model to improve the detection rate without answering the business question precisely, is a common mistake, and therefore it is important that the integrity of the business question is maintained. Deciding on a final detection rate also requires understanding of the cost and benefits of each type of prediction in your specific context and making a decision that aligns with your requirements, goals and priorities.
In general, the trade-off between true positives and false positives can be managed by adjusting the threshold for making a positive prediction. A higher threshold will result in fewer positive predictions (both true and false), while a lower threshold will result in more positive predictions (both true and false). Some who implement models manually, use a confusion matrix that can provide visualisation of the model’s performance related to where the model is making errors or where it is being accurate.
Managing False Positives
If your priority is to minimise false positives, you can choose a higher threshold, which will result in fewer false positives at the cost of potentially missing some true positives. On the other hand, if your priority is to maximise true positives, you can choose a lower threshold, which will result in more true positives but also more false positives. These techniques, without a platform created to automate these steps, will be a manually intensive task, which can cause long delays or costs to find the optimal detection rate.
Using Evaluation Metrics
Another approach is to use evaluation metrics that take both true positives and false positives into account, such as precision and recall, and optimise for a trade-off between the two. Precision is the ratio of true positives to the total number of positive predictions, while recall is the ratio of true positives to the total number of actual positive instances in the data. Depending on your specific needs, you may want to prioritise precision or recall, or find a balance between the two.
High quality prediction models
The optimal detection rate for predictive analytics models depends on the clear understanding of the problem domain, the nature of the data, and the level of accuracy required for the intended use case. In general, a higher detection rate indicates that the model is better at identifying relevant patterns or anomalies in the data. However, it's important to keep in mind that there is always a trade-off between detection rate and other performance metrics, such as precision, recall, and false positive rate, as this will ultimately reflect in cost and benefit considerations. Ultimately, trying to create a high quality prediction model is not a simple task, and organisations should turn to platforms with a strong track record of implementing models with a high detection quality.
About Prospero AG
Prospero is a leading company in the development and implementation of Business Solutions based on Predictive Analytics and Machine Learning. Our Award Winning AI Platform DetectX?, and it’s business solutions, are used by clients in the banking, insurance, life sciences, and manufacturing industries across 12 different countries.
Read more articles from Prospero here
Head Strategy & Business Development
1 年Being a passionate fellow user of Prospero’s solutions, what I find especially interesting and useful is the fact that the system can support you in the process of deciding and determining the most optimal tradeoff / cut-off ratio. Knowing your data, precisely understanding your customers, their profiles / specifics, knowing the monetary values of false positive and true positives, and being able to compare them, the Prospero system will guide you through this process and recommend you the best possible ratio for a specific case, specific situation, or a specific customer segment. This can support you in your operational and strategic decision making process, adjusting to your risk appetite, governance rules and standards, or management style. EXAMPLE: Focused on increasing market shares in new territories, a manager might decide to be more aggressive and for a certain period of time accept unusually higher number of possible defaults, but trying to provide almost every new student with a credit card, knowing how much an average default would cost him and how much is the potential gain from every new customer.?So this is just a small example how your operational or strategic decisions can be supported by advanced analytics.