Data + LLM News - July 2024

Introduction

Hi there, I hope you've been enjoying the beautiful month of July. Whether this newsletter finds you before, during, or after a well-deserved holiday, we've got some interesting data news to share. Even as the world slows down for summer, the data stream keeps flowing.

As always, you can expect to find:

• One?tool?that we believe is worth digging into as a?data?person.
• A curated list of the best articles we've read this month, along with brief teasers.
• Show and Tell: Some exciting updates from the CastorDoc team.
• A data meme, to brighten your day.

Let’s dive in.

Data Tool

There is some movement in the data observability world. Soda, the data quality tool raised $14M to fuel its US expansion. Soda's platform uses AI to help businesses test, debug, and prevent data issues, making high-quality data accessible to everyone in an organization. The timing is just right, as AI's growth drives demand for high-quality data. Soda's client list includes Disney and Bloomberg, and they have a growing global user base. The company now plans to strengthen its North American presence. We’ve made a landscape of the data observability ecosystem - and can see below where Soda fits in compared to other solutions. You will find a more detailed analysis of the data observability landscape here, and a benchmark of different tools here.

Data News

• A Brief History of Modern Data Stack. Remember the "modern data stack" that took the industry by storm? Well, it's evolving into something new. in this piece, Ananth Packkildurai retraces the journey from the early days of big data to the rise and fall of the modern data stack, and introduces what's coming next. The "Next-Gen Data Stack" aims to fix the problems of its predecessor by focusing on open interoperability, developer-friendly tools, and cost efficiency. It's an exciting time as the industry learns from past challenges and works towards more sustainable, user-friendly data solutions. If you're curious about where data engineering is headed, you should find this piece interesting.
• The Analytics Development Lifecycle. Ever wonder what occupies Tristan Handy's poolside thoughts? You’ll find out here. In this piece, he introduces his work-in-progress on the Analytics Development Lifecycle (ADLC), an update to his 2016 post on mature analytics workflows. The ADLC aims to provide a comprehensive model for building mature analytics capabilities in organizations of any size. He shares some ideas about what users should expect from good analytics systems. This piece is a good reminder that while we've come a long way, there's still tons of room for improvement in how we do data. Handy's full of energy about the work ahead - and honestly, it's contagious.
• How Top Data Teams are Structured. Mikkel Dengs?e dug into 40 leading data teams across the US and Europe, and the findings are interesting. On average, data teams are split pretty evenly between insights roles (think analysts and data scientists) at 46% and data engineering roles at 43%. Machine learning folks make up about 11%. What's interesting is how team structures change as companies grow. Smaller teams tend to have more data engineers, while the big players invest heavily in machine learning. And once companies hit scale, they start taking data governance seriously. Whether you're just starting to build your data team or wondering how you stack up against others, you’ll want to read this piece.

Show & Tell

• Feature of the month: Real-Time Response Streaming. This feature lets you see results as they're generated. No more waiting for complete answers—dive into insights immediately as you type. This experience is more dynamic, and keeps you engaged and informed throughout your data analysis journey. Available now on both the CastorDoc app and Chrome extension. Want to try it for yourself? Get in touch with the team. You can see how it works below.

• Article of the month: How AI redefines Self-Service Analytics. I recently wrote an article exploring a new perspective on self-service analytics and how AI is changing things. In it, I challenge the common idea that self-service is all about DIY analysis. Instead, I propose that true self-service should focus on using AI to share expert knowledge across organizations. The article explains how AI, particularly large language models, can help make high-quality, pre-existing analyses more accessible to everyone in a company. I’ve been thinking and writing about these topics a lot lately - so If you have an opinion on the matter, I would be interested to hear it!

Data Meme

We posted this meme a year ago already - but we felt that it was even more accurate today. So, for the pleasure of your eyes, here it is again.

Until next time,

Louise from CastorDoc