What are the top data teams tracking and how?
Prukalpa ?
Co-Founder at Atlan –?Home for Data Teams | Forbes30 & Fortune40 lists | TED Speaker
Welcome to this week's edition of the ? Metadata Weekly ? newsletter.
Every week I bring you my recommended reads and share my (meta?) thoughts on everything metadata! ? If you’re new here, subscribe to the newsletter and get the latest from the world of metadata and the modern data stack.
?? The future of the metrics layer
The metrics layer has been all the rage in 2022. It’s just forming in the data stack, but I’m so excited to see it finally come alive. But WTF actually is a metrics layer?
Today metrics are often split across different data tools, and different teams or dashboards end up using different definitions for the same metric. The metrics layer aims to fix this by creating a common set of metrics and their definitions. But incorporating metrics is not easy...
A few weeks ago, I was lucky enough to chat about the metrics layer with two most prolific product thinkers in the space — Drew Banin (Co-founder of dbt Labs) and Nick Handel (Co-founder of Transform).
We just released an edited transcript of the discussion here, which covers:
- How would you explain the metrics layer to a beginner data analyst?
- What is the real problem the metrics layer is looking to solve?
- How should we think about the metrics layer, and how should its interplay with other components of the modern data stack?
- What metadata should we be tracking about our metrics, and why?
- What are the real use cases for a metrics layer?
- And of course, we had to touch on… “bundling and unbundling” ??
This was an amazing conversation as we had a TON of data practitioners join & a whole lot of questions (and answers) from practitioners are covered in the AMA)! Check out the transcript here | full recording: here
My favorite piece of inspiration from the day was Drew’s message for the data world:
“There are a lot of problems in data that you can solve with technology, but some of the hardest and most important ones you must solve with conversations and people and alignment, and sometimes whiteboards. Knowing which kind of problem you’re trying to solve at any given time is going to help you pick the right kind of solution.”
? Spotlight: Data metrics top data companies are tracking
I loved reading Mikkel’s post from last week! A lot of companies just focus on tracking data quality. However, I believe that tracking data team productivity is the key to actually achieving your outcomes with data, and of course, the final outcome metric: usage and engagement of data products that the team builds.
I’m a big believer in “data product shipping standards” — which I wrote about in my article on the metadata foundation for a data mesh — and systematically measuring the productivity of the data team (Candidly, I think Mikkel’s framework for team productivity could use some work, and we need better ways to measure how effective our data teams are. But more on that in a future post!)
Some excellent nuggets from his article:
Because measuring data quality helps set high standards for your data team. A Looker dashboard not being updated before 9 am when the C-level team looks at the KPIs or frequently being told about data issues by stakeholders before you identify them are both examples that reduce trust in data. Measuring data quality can help you be scientific about this and be proactive about where to improve controls.
Productivity: Time spent improving data quality is a double edged sword and has to be balanced with also doing other strategic work. You should track how much time your team is spending on work related to data quality.
Engagement: Often dashboards and data models get thrown over the fence without much consideration for who uses them. Engagement metrics help keep everyone responsible that what’s being created is also being used.
Criticality: Not all data should be treated the same. An error on a data model that’s only used by you and a few close colleagues may have a very different impact than an error in your top-level KPI dashboard or in a data service that powers a production-level ML system. Most teams have a way of knowing about this, for example by tagging data models as ‘tier 1’, ‘critical’ or ‘gold standard.
?? More from my reading list
- Product Sketch: Airflow by Stephen Bailey
- Analytics is Stuck in the 1970s — How the Modern Data Stack can Enable Better Decision Intelligence by Alvin Wong
- A Framework for Embedding Decision Intelligence into your Organization by Erik Balodis
- How to Add Value as a Data Analyst by Cassie Kozyrkov
- The Next Wave of Cloud Infrastructure by Sai Senthilkumar
I’ve also added some more resources to my data stack reading list. If you haven’t checked out the list yet, you can find and bookmark it here.
?? What is the biggest challenge faced by data leaders right now?
I recently asked this question to data leaders on LinkedIn. According to the poll results, creating a great data culture remains one of the biggest challenges. This is where I believe that it’s time for the modern data culture stack. The more I think about this, the more I’m convinced we need a role in data teams (e.g. data enablement like sales teams have sales enablement) focused on creating a data culture. I’d love to hear from you about the challenges you face as a data leader.
If you’re new here, check out the archive of this newsletter on Substack. I'll see you next week with some interesting stuff around the modern data stack.
Liked reading this edition of the newsletter? I would LOVE it if you could take a moment and share it with your friends on social.
Want to solve Problems WHICH MATTER with Data | TCS Alumni ??| On the Path to be Domain & Platform Agnostic Data Professional
2 年Handling different data with different priorities and balancing #dataquality improvement with datateam productivity are of utmost importance!! ????
Award Winning Data, Analytics and AI Transformation and Strategic Planning Executive, Keynote Speaker, Board Chair, Board Director, Committee Member, Advisory Board Member, NED.
2 年I love the cartoon, and it’s very true isn’t it!! Whilst recognising the polarising effect of linked in polls - I worry about the focus on “data culture”. The reason for this is that I think it’s currently a poorly defined term with minimal if any framework as to how we measure it, what it is or what good looks like. I am also sceptical that we have done enough work on benefits to determine what the ROI would be. Granted, we need organisations to shift quickly to making strong decisions based on fact and to start utilising data as a core asset for their business. But what is data culture 1.0 and what are its constituent parts, metrics etc. I have always had a role in my team that leads teansformation. That role always had to have people and communication skills and it was never a pure PMO type role. If we are going to lean on “culture” as a problem then we need to be clearer around the problem statement. Great work Prukalpa ? I love reading your posts, always very engaging and though provoking..
Director - Analytic Solutions and Data Science
2 年Glad you liked it!