Why Data Products?

Data products is a popular topic in the data community today. A related and recurring topic of discussion in conversations with customers and practitioners in recent times has been "Why data products?". It generally goes like this...

"I am hearing and reading a lot about data products as the way to shape data investments. 'The why' is still not clear for me. We have been managing and serving data for our users for years, how is this approach new and different? What value can we get from it?"

A clarification as the above when surfaced is a humbling moment for us data geeks immersed in 'the what' and 'the how' of bleeding edge data concepts, with an implicit appreciation for 'the why'. Explaining 'the why' in relatable contexts to our stakeholders who are not data geeks, and to our community practitioners starting out on these concepts, can be a quality challenge. Data products is one such concept that has consistently surfaced 'the why?' clarification in customer and practitioner conversations on data transformation and data governance best practices. I vividly recall initial live instances when I attempted to explain 'why data products?' sans geek speak, being blooper grade comical, both in delivery and in reception :-)

Sharing here a refined over time framing and which have found to be effective in applying to navigate a conversation on this topic. A short and memorable acronym for the framing is ??????; hence the cap in the image for this writing :-) The anchor for CAP is the notion of everyday products and their semblance to data, embodied by three defining traits, ??onsumer centric, ??udience oriented, and ??urpose driven.?

Consumer centric

What do everyday consumers expect of everyday products? Value in ability to efficiently (time and cost) address their need, ease of understanding and use, safety, quality, compliance with their stated preferences, service level agreements, and timely support when needed. Data consumers are no different. Data consumers expect (and rightly so) these attributes to be the norms of the data they consume and apply, fulfilled by service grade contracts.

Audience oriented

Everyday consumers are a broad community with high diversity in demographics, needs, and product expectations. Data consumers are no different. The needs and expectations of organization-wide consumers of data spanning data, technical, business, and operations functions, are as diverse in the data context. A product needs to cater well to its target audience within a wider consumer base. Ditto for data. For instance, a clean master dataset with trustworthy customer account data is golden for marketing and sales data scientists who would otherwise have to wrangle enterprise-wide data to create such for their applied models. For a sales leader, seeing accurate and consistent revenue numbers across related reports and dashboards, is a golden state. Audience context and orientation matters.

Purpose driven

A product without a purpose is either dead on arrival or a drain. Purposes can be specific (or) capabilities and tools of generic value in uncovering possibilities yet to be uncovered. For instance, the assembly of Lego blocks can be guided for a specific structure (or) free form in discovering structural possibilities within the parameters of the blocks in hand. Data likewise can serve either purpose. What is key though is to form a good sense of the purpose(s) to be served, by and for whom (audience orientation), and the expectations of the users (consumer centricity), both stated and implicit.

For instance, the diverse purposes served by user data for directional analytics vs compliance grade privacy audit reporting, have different orientations and expectations. Shaping user data contextually to serve each purpose, can yield the best contextual outcomes, directional analytics not blocked on the grounds of compliance grade quality, and compliance grade audit reporting not compromised by the quality sufficiency for directional analytics.

As another example, an analytics dataset to serve a specific business insight has different considerations than an analytics model intended for exploratory analytics to form hypothesis for future investments.

The Data Products Practice

Shaping data investments with the CAP traits and doing intentionally, is the essence of a data products practice.

The crux of a data products practice is the intentional shaping and serving of data as products that are data consumer centric, audience oriented, and organizational purpose driven.

The benefits of the practice are wide reaching. The top line benefits that I have seen realized in applied practice include:

• Increased durability, reusability, and value accrual of data investments beyond their initial point in time application(s).
• The shaping of an intentional data estate to serve impactful purposes.
• Reducing the cost of operations and data exposure risks with data estate minimalization as an outcome of intentional shaping.
• Responsible data democratization to confidently scale organization-wide value creation from data.
• New business streams from monetizable data products.
• Fandom with organization-wide data consumers.

Data Products and the age of AI

Data consumers and audiences in the age of AI include AI actors; AI powered systems, AI copilots, and AI agents with diverse and growing purposes that were prior human served. Agnostic of type, an AI can only be as good as the data that it is trained on. Applying the CAP traits in shaping data estates with the data products practice, for AI actors and purposes served by AI, will have a profound impact in scaling responsible value creation with AI. For instance, intentionally shaping data products for RAG (Retrieval augmented generation) vs LLM (Large language model) training use cases, can have a profound impact on compliant data handling for regulatory requirements.

Persevere or/and evolve?

Organizations and practitioners who are intentionally shaping their data investments with the CAP traits, are practicing the data products practice, whether they refer to it as such or not. If you are such, you have made the evolution to intentionally shaping your data estate for greater value outcomes. Keep persevering and evolving further :-)

Organizations and practitioners applying the CAP traits but doing only for managed reports and dashboards as conduits to make data accessible for their organization-wide users, have the opportunity to expand the aperture of their data products practice to broader data modalities that serve a broader audience, in scaling organization-wide and responsible value creation from data. Clean datasets for data scientists and data analysts, semantic models for business analysts and data savvy product managers, quality ML/AI models for ML engineers, and resilient data APIs for software engineers, are as high value data product investments to consider, beyond reports and dashboards.

Organizations and practitioners navigating data investments as projects to serve point in time needs (most of us start here, I personally and every data team I have been a part of, did when starting out) could benefit greatly from evolving to apply the practice of data products in shaping their data estates for greater impact.

Getting Started

How do we get started on the data products journey? A multi-posts topic in its own right and which I will look to share applied learnings on in a future writing. Will close here now with a top line recommendation on getting started; it is all about embracing and practicing progression over perfection. Taming data estates evolved organically over several years, to shape to data products, is best navigated as an incremental journey, with each step anchored to a specific CAP value outcome that is measurable as an organizational OKR (Objectives and key results). More on this in a future writing :-)