Using Bayesian Stats to Evaluate Your Model During a Pilot

Using Bayesian Stats to Evaluate Your Model During a Pilot

??Hakim Elakhrass ??Hakim Elakhrass

??Hakim Elakhrass

post-deployment data science | OSS | co-founder @ nannyML

发布日期: 2024年6月21日
+ 关注

In this week’s newsletter, we highlight three recent papers that have caught the attention of our data scientists. We also discuss upcoming features for NannyML Cloud, which have been in high demand. Additionally, we share two recent additions to our Post Deployment Data Science Blog. Finally, we provide details about our upcoming webinar. Let’s kick things off!

Papers that caught our eyes ??

A Short Survey on Importance Weighting for Machine Learning

Paper: https://arxiv.org/abs/2403.10175v2

Concept Drift Detection using Ensemble of Integrally Private Models

Paper: https://arxiv.org/abs/2406.04903

Improving the Validity and Practical Usefulness of AI/ML Evaluations Using an Estimands Framework

Paper: https://arxiv.org/abs/2406.10366

NannyML News

New feature coming soon

One of the most requested features is coming to NannyML Cloud: Segmentation! ?? This feature lets you split your data into groups to analyze them separately.

Now you can know if your model’s issues are coming from a single segment or all!

A Comprehensive Guide to Univariate Drift Detection Methods

Article: https://www.nannyml.com/blog/comprehensive-guide-univariate-methods

Feeling overwhelmed by all the univariate drift methods? Kavita Rana wrote an experimental comparison between univariate drift methods to understand when to use one or the other.

领英推荐

Don’t Drift Away with Your Data

Article: https://www.nannyml.com/blog/monitoring-data-drift

Data is going to drift, but it is up to you to prevent it from affecting your model’s performance. Check out Taliya Weinstein 's blog if you are looking for a gentle introduction to data drift.


Shoutout to the community ??

Thanks to Eric Holland and Ilia Ekhlakov for the kind words; it truly helps us keep up with the hard work! ??

Want to learn how to apply Bayesian statistics to evaluate your ML models before they go into production?

The science behind this talk will blow your mind ??

Join Wojtek Kuberski on 27th June to learn more about it.

Topic:?Why you Need to Pilot your ML Systems

Date:?27th June 2024, 1:30 pm pm CET

Register for the event:





Post Deployment Data Science Post Deployment Data Science

Post Deployment Data Science

2,954 位关注者

订阅

要查看或添加评论，请登录

??Hakim Elakhrass的更多文章

  • All about ML Monitoring in Finance
    2024年9月27日

    All about ML Monitoring in Finance

    Today’s newsletter is all about Finance and ML. We've been working with data science teams in the financial sector for…

    4 条评论
  • Retraining is not all you need
    2024年7月19日

    Retraining is not all you need

    This week’s newsletter highlights two papers that have caught our attention. We discuss what we did to make NannyML…

    1 条评论
  • PSI, or Not PSI, That Is the Question
    2024年6月7日

    PSI, or Not PSI, That Is the Question

    At NannyML, we’re often asked to add the Population Stability Index (PSI) to our suite of data drift detection methods…

    2 条评论
  • People are catching up with post-deployment data science
    2024年6月3日

    People are catching up with post-deployment data science

    The world of post-deployment data science is more active than ever. In the last couple of weeks, there has been plenty…

    4 条评论
  • Industrial Machine learning Use Cases
    2024年5月22日

    Industrial Machine learning Use Cases

    In the current geopolitical climate, manufacturing is increasingly returning to the US and Europe. These modern…

  • Can we detect LLM hallucinations?
    2024年5月8日

    Can we detect LLM hallucinations?

    A quick review of our experiments Since the beginning of NannyML, our approach to tackling new problems has always…

    6 条评论
  • Exploring LLM hallucination detection methods
    2024年2月9日

    Exploring LLM hallucination detection methods

    Over the past few weeks, we've been experimenting with several methods for LLM hallucination detection. Here's a…

    5 条评论
  • Post-deployment hype
    2023年11月3日

    Post-deployment hype

    In this week's issue, we share educational material to get into post-deployment data science, new findings on…

    1 条评论
  • Data science is changing
    2023年9月22日

    Data science is changing

    That isn’t a surprise. It is one of the fastest evolving disciplines that there is.

    4 条评论
  • KISS at OpenAI, #batchforlife, and data science conspiracies
    2023年9月1日

    KISS at OpenAI, #batchforlife, and data science conspiracies

    Keep it simple stupid. From openAI, simple batch processing, and the simplicity of AWS Sagemaker Algorithms.

    8 条评论

社区洞察

其他会员也浏览了