Journey to Data Mesh ??

Journey to Data Mesh ??

It’s a problem any student can relate to. You’re learning a new concept, and you’re stuck on how best to answer a question, or even understand altogether. It’s the universality of this experience that’s made Brainly the world’s most popular education app, serving students, parents, and educators, helping them answer questions and strengthen skills across subjects like English, Math, Science, and History.

Serving hundreds of millions of users, across diverse personas and subject matter, Brainly’s business generates an astounding amount of data, and their organization depends on it to better serve their users and continue their growth.

In this week’s newsletter, we spotlight how Brainly transformed its data and analytics strategy.?You can read the full story here →

? Spotlight:?Brainly’s Journey to Data Mesh

Illustrating some of what’s worked well for Brainly’s data team, Kasia Bodzioch-Marczewska, their Domain Lead of Data Engineering, joined Atlan at the 2023 Gartner Data & Analytics Summit in London, sharing the progress her team has made on Data Mesh, how Active Metadata Management can help drive necessary cultural and technical shift, and key takeaways for data leaders considering a similar journey.

Decision on Data Mesh

Recognizing the potential of activating Brainly’s data, their team began to consider implementing Data Mesh to better organize work, and encourage full ownership and stewardship of their data.

A?data mesh?is a?technical?and?cultural?approach to building a decentralized architecture that organizes data by a specific business domain, providing more ownership to data producers.

The Brainly team began their Data Mesh journey by defining two critical dimensions to master. First was technical, investing in technology that would better enable transparency, team-to-team collaboration, and data quality standards. Second, and perhaps more importantly, was cultural, enabling ownership and accountability, despite a decentralized governance model.

Further enhancing the value they could yield from Data Mesh, and treating data as a product, Kasia and her team found important alignment with the way Brainly’s product teams were organized and run.

“A few years back we decided, as a company, to decentralize our product teams,” Kasia shared. “Every product team at Brainly is independent and has their own tools and data.”?

While this model paid dividends for innovation and agility, making Brainly the leader in education technology that it is today, the siloed nature and ownership of this data meant frustrating back-and-forth whenever one team needed another’s data.?

“If we think about this data and this setup, if a program department like Tutoring, for example, wants to utilize financial data, they would have to go and ask. There was a very long process to get access to the right data, and to figure out which data you could use for your analysis,” Kasia explained.

The Data Mesh concept, combined with Brainly’s unique model of product domain ownership, was a clear and exciting opportunity. But to eliminate the team-to-team collaboration friction inherent to this strategy, Kasia’s team evaluated the Active Metadata Management market in search of a solution.

Data Catalog – Enhance Data Mesh

“We figured out that we needed a data catalog to support us with the cultural piece of Data Mesh. We went through several proofs of concept, and through those, we chose Atlan as our data catalog.”

In the early stages of their Data Mesh journey, Atlan has proved to be a crucial partner as Brainly migrates to a new data platform that will better support their new way of working.

No alt text provided for this image

“We’ve onboarded into Atlan all of our data sets from both our legacy platform and our new data platform” Kasia explained. “And for migration purposes, we’re using Atlan to identify these sources of the products that are to be migrated to the new data platform, as well as to figure out the downstream objects affected by decommissioning in the legacy platform.”

No alt text provided for this image

Brainly’s new data platform consumes a vast array of data sources, both structured and unstructured, passing through to a raw data lake in AWS S3, Spark and Glue for processing, through to a data mart using Snowflake, and Tableau or Metabase for visualization and analysis.

As users migrate from legacy-to-modern, Atlan serves as their gateway to understanding and applying data and the new platform’s capabilities. “Obviously, we have users consuming all of this. We have all of our assets in Atlan; everything integrated into Atlan. Basically, Atlan is the place where all data sharing happens,” Kasia explained.

Ten domain teams at Brainly are already using their new data platform, and increasingly depend on Atlan for crucial context about available data. “We see more and more people defaulting to the data catalog instead of going through a lengthy process of going back-and-forth (with questions),” Kasia explained.

Read the lessons learned by Kasia the data team at Brainly in this journey.

?? From Our Reading List

?? The Big Reveal of Atlan AI – 7 Days to Go!

A few weeks ago, we announced the launch of Atlan AI as the first-ever copilot for data teams. And it’s almost time for its big reveal. The launch was also covered in?Benn’s newsletter.

If you are interested in learning how AI has completely revolutionized the way data teams discover, understand, and work with data, come join the waitlist to explore Atlan AI and save your spot for the launch event next week?here.

P.S. Liked reading this edition of the newsletter? I would love it if you could take a moment and share it with your friends on social! If someone shared this with you, subscribe to upcoming issues?here.


Cameron Price

Founder | Senior Data Executive | 30 Years of Leadership in Data Strategy & Innovation | Executive Director | Sales Executive | Mentor | Strategy | Analytics | AI | Gen AI | Transformation | ESG

1 个月

It's great to hear about Brainly's journey with Data Mesh, especially the focus on both technical and cultural dimensions. How has the transition affected team collaboration and the overall data strategy? Would love to learn more about any surprising challenges or wins along the way.

回复
(on sabbatical) Scott Hirleman (back mid next year maybe but prob not)

Data Mesh Radio Host - Helping People Understand and Implement Data Mesh Since 2020 ??

1 年

Katarzyna Bodzioch-Marczewska let's chat data mesh and look to do an episode of Data Mesh Radio. It's super easy, just grab time for a planning session and we can go from there. Happy to provide more info too: https://calendly.com/data-as-a-product/podcast-plan-session

Navindas Neroth

Database Architect @ Silicon Stack | Cloud Administration, Databases

1 年

nice one

回复
Marc H. Guirand

Volume Shooter ???

1 年

???

回复

要查看或添加评论,请登录

Prukalpa ?的更多文章

社区洞察

其他会员也浏览了