The big picture for data professionals, data documentation, and more

Welcome to this week's edition of the ? Metadata Weekly ? newsletter.

Every week I bring you my recommended reads and share my (meta?) thoughts on everything metadata! ? If you prefer getting the newsletter directly in your inbox, you can subscribe on Substack here.

Starting this week with a meta musing triggered by a conversation with Gordon Wong — a feeling of thankfulness for how far we’ve come as a data community. Even a few years ago, there was NO community for us data people — no newsletters and blogs talking about what it means to be a data leader, no conferences for purple people as Jillzy (from dbt) calls us, no place for us ‘data people’ to figure out how we can work together or solve our collective challenges.

In contrast, nowadays, every week I hear stories about how the modern data stack is changing how data teams operate. I’ve seen that conversations in the data world are moving from talking about data architecture to data culture. I see more amazing data leaders step up and talk about their challenges and novel ways they are finding solutions! We still have a ton of challenges to solve as a community... but I feel more positive than I have ever before. ??

In this edition, I share my thoughts on some of the articles from my fav topics talked about in the data world last week – data documentation, and the big picture for data professionals. Also, check out some other readings from my data stack reading list here.

?? Fave?Links from Last Week

1. How to Feed a Data Professional by Stephen Bailey

I LOVED the analogy that Stephen used here! Except that I was hungry for more after this (pun intended). ??

Snack time for us will always be ad-hoc, do it when you want to / feel like it, eat what you are craving for. We all enjoy a good snack, but it cannot make up for a complete meal.

“...without active leadership — a Responsible Data Parent in the room— data teams will face constant pressure to become snack shacks. In industry, there is a downward force to push data work to the lowest common denominator.”

With the modern data stack, we have moved past the transactional nature of how we treated data. I believe today the biggest delta will come from how data teams build their data culture. As more and more data teams become central to an organization, collaborating with diverse teams, it’s only fair to equip them with everything that can help them understand and see the big picture.

“...data professionals crave a connectedness to The Bigger Picture that sets them a bit apart from other professionals. They want to see the line between a data product and a business strategy. They want to understand the categories of problems they face and bring in strategies from other companies to tackle them. They want to animate the business through experimentation.”

When you want a data professional to “pull a number for you” — DON’T. Instead, tell them about the problems you’re looking to solve and the decisions you are struggling with. Create a space for them to ask questions, to be your partner in problem-solving — not just someone who pulls data for you.

The next generation of data professionals seeks the big picture and the scope to contribute to the big picture, beyond just building dashboards or getting you some numbers disguised as insights.

2. Write that down: What I've learned about documentation for data teams by Brittany Bennett

In this article, Brittany introduces a new term, “documentation debt”, a term that I love!

“But my code was not commented. Most of the knowledge about our data program's compliance set up only existed in my head. I knew all the tables in our warehouse like the back of my hand...but my team had no insight of where to look for their data. I learned the value of documentation the hard way: by not investing it for the first year I was brought on as Data Director at Sunrise Movement. And as a result, I accrued a massive amount of documentation debt—the build up of documentation needs on a team by focusing on short term needs rather than long term sustainability.”

Library, documents, templates, checklists, documentation culture – these are some of the most important tools to handle the everyday chaos inside a data team. Brittany’s article has some great foundational ideas to get started with building a culture of documentation.

I’d also recommend using a framework like the 5W1H (What, Why, Who, When, Where, How) to build documentation standards for your data products, which can set the bar for what “great documentation” should look like.

Trust me (and Brittany), it's worth it ??

“My team can work independently of me. They do not have to guess what a column in a table means—they can look it up in our dbt docs. Our compliance lore is now saved in Library for anyone on my team to view. And as dark as it may sound, if something happened to me, my team would be able to carry on.”

P.S. HUGE shoutout to the leadership at Sunrise for valuing their data team so much that they allowed Brittany and her team to go dark for a month to build the strong data foundation they need! We need more CEOs and leadership to allow their data teams to actually invest in the foundations they need to become great at what they do.

???More from My Reading List

1. How to Succeed with Data by Nitish Mathew
2. How to get more power from your data analytics engine by Petr Janda
3. How to design your data stack for curiosity by Amit Prakash
4. Make using data easy by John Cutler
5. Data Product MVPs: Make Sure the Problem You're Solving Is Worth It by Eric Weber

???Podcasts to Tune In

• If you’re a data engineer reading this newsletter, check out this podcast by Tobias Macey. In this episode, he shares his learnings from running the podcast, macroscopic themes in the industry, and lessons for data engineers. Tune in here.
• There is no way the data industry can flourish without nurturing junior data practitioners. This is a great podcast to learn how Brittany and her team at Sunrise hire for mission alignment rather than the technical background, and invest in the training and mentorship of young data professionals. Tune in here.

If you haven’t yet, bookmark my data stack reading here.

I'll see you next week with more interesting updates from the modern data stack! Meanwhile, you can subscribe to the?newsletter on Substack and connect with me on LinkedIn?here.