Tales from the Data and Analytics Crypt 2024 Edition is starting now...

Greetings, boils and ghouls! Welcome to 2024, a year full of frights and delights.

It's time to start some bone-chilling conversations and spine-tingling considerations. I have dug up one ghastly topic from the "lake" of data and AI world. The goal is to slice and dice it from left to right and top to bottom to confirm (or deny) its freshness and value. But beware, you might not like what you find. You might discover some horrors that will make your hair stand on end and your eyes pop out of your skull. You might uncover some secrets that will make your skin crawl and your teeth chatter. So, are you ready to join me on this gruesome journey? Then let's get started and explore the data platform in 2024. But don't say I didn't warn you. You're about to enter the realm of...Tales from the Data & Analytics Crypt!

I still remember the times when “data platform” was the holy grail of the data ecosystem. In 2015, I was hired by #Microsoft (thank you Miroslaw Szymczak, Drazen Sumic and Karl Davies-Barrett for making this happen) to focus on building data platforms with customers . It was a dedicated role of Data Platform Solution Architect (DPSA) and believe me, the investment in this motion was very visible inside Microsoft and across the market at that time.

And here we are, almost a decade later. The data ecosystem has changed rapidly over time, but the data platform as a concept is still with us. Is it? That is the first question on my list of considerations. Of course, there are many more, so I decided to spend some time on this topic this year, especially since I still hear and see a lot of talks, activities, and investments done by different companies.

The idea is simple - I will go through the topics I find relevant to cover the data platform concept from a 2024 perspective. Everything will start from:

• Introduction to Data Platform: This part covers the basic concepts and definitions of a data platform, as well as its advantages and challenges. You can expect to learn about:

What is a data platform and why do you need one?

Data platform vs data warehouse vs data lake/Lakehouse

Challenges and best practices

Trends and innovations

Once we have the same understanding of the concept, I will dive into:

• Architecture and Design: This part focuses on the technical aspects of building and maintaining a data platform, such as the tools, technologies, patterns, and processes involved. You can expect to learn about:

Tools and technologies

Architectures and design patterns

Data ingestion and data transformation in a data platform

Data catalog and data discovery in a data platform

The semantic layer and access management in a data platform

Security and privacy Scalability and performance

Testing and deployment

Monitoring and maintenance

Optimization and tuning

Migration and integration

Automation and orchestration

We could say - that is more than enough, but it is not only about the concept and technology. The decision to build a data platform is deeply connected to the business, company strategy, and goals. There is no data platform without data that comes from the business, and there is no value of a data platform without business cases. That is why one of the very important and deeply investigated areas will be:

• Use Cases and Applications: This category showcases the different ways that a data platform can be used to generate value and insights for various domains and purposes. You can expect to learn about:

Use cases and success stories

Data platform for business intelligence and analytics

Machine learning and generative AI in a data platform ecosystem

Data platform vs customer data platform

Comparison and benchmarking

Customization and personalization Innovation and experimentation

The last but not least is how to prepare organizations for data platform deployments and management:

• Management and Governance: This category addresses the organizational and strategic aspects of managing and governing a data platform, such as the roles, responsibilities, skills, culture, and ethics involved. You can expect to learn about:

Data governance and data quality in a data platform ecosystem

Data observability and data reliability, as a key element of a data platform Roles and responsibilities

Skills and competencies

Training and education

Ethics and social responsibility

Costs and benefits

Risks and mitigation

Evaluation and measurement

Maturity and roadmap

Vision and mission

Alignment and value proposition

Advocacy and evangelism

I hope this introduction gives you a clear overview of what to expect from this series of articles. I look forward to sharing my insights and experiences with you on the topic of data platform in 2024. If you have any questions, feedback, or suggestions, feel free to drop me a message or leave a comment below. I will make sure to address your ask. Let’s get started and explore the data platform landscape in 2024 together.

Hit the ?? on my profile, Bartlomiej Graczyk so you never miss a post.

Make sure to subscribe Tales from Data & Analytics Crypt on LinkedIn

Picture: Generated using DALL-E, Article introduction (Tales from the crypt style) powered by Open AI.

?