Issue #317 - The ML Engineer ??

How time flies!! This week we still continue to celebrate 6 years since starting this weekly MLE newsletter ?????? What started with just one commit, today now has almost 70,000 subscribers ?? And not a single Sunday missed ?? Thank you to everyone for your continued support - in today's newsletter we share a special edition celebrating our achievements throughout 2023!

This week in Machine Learning:

• Chip Huyen on Agentic AI
• 10 Year Review of Anomaly Detection
• Modular SVGs with GenAI Models
• Efficient Coding with LLMs
• Extracting AI models from Apps
• Open Source ML Frameworks
• Awesome AI Guidelines to check out this week
• + more ??

If you're looking for an interesting career opportunity, I'm ?hiring for a few roles including Data Science Manager (Forecasting), as well as Principal Product Manager (Forecasting) - check them out and please do share with your network!

Chip Huyen on Agentic AI

Chip Huyen shares an exerpt from her latest book on AI Engineering, diving specifically into the concept of AI agents: Agentic systems extend the capabilities of an ML model by providing structured tools such as a code executor or search engine, and a planning mechanism to decide which tools to invoke and in what order. The mechanisms to design the system, including the "tools" accessible is quite important; too few tools limit capabilities; too many complicate usage. AI Agents are still emerging in the field, which means that best practice and guidance is being developed - this is a great resource to get the latest literature digested into a single place.

10y Review of Anomaly Detection

Time-series anomaly detection has advanced significantly over the last decade, and these are some key methods, benchmarks and concepts: When looking towards anomaly detection techniques no single method excels universally; current benchmarks and labeling often lack realism or consistency, and evaluation metrics vary widely - there has been some really interesting efforts recently which aim to unify datasets to tackle this such as TSB-UAD and TSB-AD. There has also been a known gap to assess the performance of different methods, for which various measures have been presented such as AUC-based measures, Range-AUC, VUS, etc. There has been continued progress on more robust benchmarks, methods to handle noise and lag, and new techniques that address high-dimensional or irregular time-series data through ensembling, AutoML, and beyond. Check out this survey from various universities which provides a survey of the past decade of time-series anomaly detection.

Modular SVGs with GenAI Models

Creating SVGs is a pain, but as expected a new AI model is making this process extremely simple: NeuralSVG is a text-to-SVG framework that generates clean, layered, and easily editable SVGs from text using a compact MLP. The architecture is quite simple and builds on top of Score Distillation Sampling (SDS) from modern diffusion models, but instead of producing dense, pixel-like vector shapes, it enforces a layered design with a small number of semantically meaningful elements. It is quite interesting to see GenAI reaching further applications and tackling more and more opportunities in the creative space to support domain experts with productivity tooling.

Efficient Coding with LLMs

One of the main opportunities in AI is to support developers as a co-pilot - Tailscale co-founder (+ former Google StaffEng) walks through his learnings adopting LLMs in day-do-day coding: It is interesting to see practitioners adopting LLMs for development in real-world workflows - in this case the primary usecases are 1) autocomplete, 2) search-like tasks, and 3) interactive chat-based code generation. This is still an area which is quite nascent and hence there's still a lot of new opportunities that will emerge changing the way that we think about it; certainly a space to keep a close eye for developers.

Extracting AI models from Apps

Did you know it's possible to extract AI models shipped with Android Apps? This is a really interesting article that goes through extracting a TFLite model from an APK: As broader adoption towards on-device AI models increase, security considerations will need to evolve in order to protect these resources, particularly given that often on-device processing is done to protect privacy and security of users. It's quite surprising how easily it is possible to extracted ML Models from Android apps even when encrypted - here it basically just requires hooking into the model-loading process with dynamic instrumentation tools like "Frida". It is interesting to see how fast the field of ML Security is evolving, this is clearly becoming one of the most important areas of attention as AI hits production in larger and broader magnitudes.

Upcoming MLOps Events

The MLOps ecosystem continues to grow at break-neck speeds, making it ever harder for us as practitioners to stay up to date with relevant developments. A fantsatic way to keep on-top of relevant resources is through the great community and events that the MLOps and Production ML ecosystem offers. This is the reason why we have started curating a list of upcoming events in the space, which are outlined below.

• The State of AI in 2024 - WeAreDevelopers 2024
• Responsible AI Workshop Keynote - NeurIPS 2021
• Practical Guide to ML Explainability - PyCon London
• ML Monitoring: Outliers, Drift, XAI - PyCon Keynote
• Metadata for E2E MLOps - Kubecon NA 2022
• ML Performance Evaluation at Scale - KubeCon Eur 2021
• Industry Strength LLMs - PyData Global 2022
• ML Security Workshop Keynote - NeurIPS 2022

Open Source MLOps Tools

Check out the fast-growing ecosystem of production ML tools & frameworks at the github repository which has reached over 10,000 ? github stars. We are currently looking for more libraries to add - if you know of any that are not listed, please let us know or feel free to add a PR. Four featured libraries in the GPU acceleration space are outlined below.

• Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced? data processing usecases.
• CuPy - An implementation of NumPy-compatible multi-dimensional array on CUDA. CuPy consists of the core multi-dimensional array class, cupy.ndarray, and many functions on it.
• Jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
• CuDF - Built based on the Apache Arrow columnar memory format, cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

? If you know of any open source and open community events that are not listed do give us a heads up so we can add them!

OSS: Policy & Guidelines

As AI systems become more prevalent in society, we face bigger and tougher societal challenges. We have seen a large number of resources that aim to takle these challenges in the form of AI Guidelines, Principles, Ethics Frameworks, etc, however there are so many resources it is hard to navigate. Because of this we started an Open Source initiative that aims to map the ecosystem to make it simpler to navigate. You can find multiple principles in the repo - some examples include the following:

• MLSecOps Top 10 Vulnerabilities - This is an initiative that aims to further the field of machine learning security by identifying the top 10 most common vulnerabiliites in the machine learning lifecycle as well as best practices.
• AI & Machine Learning 8 principles for Responsible ML - The Institute for Ethical AI & Machine Learning has put together 8 principles for responsible machine learning that are to be adopted by individuals and delivery teams designing, building and operating machine learning systems.
• An Evaluation of Guidelines - The Ethics of Ethics; A research paper that analyses multiple Ethics principles.
• ACM's Code of Ethics and Professional Conduct - This is the code of ethics that has been put together in 1992 by the Association for Computer Machinery and updated in 2018.

If you know of any guidelines that are not in the "Awesome AI Guidelines" list, please do give us a heads up or feel free to add a pull request!

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website