Diary of a Data Product Manager: Considerations for a solid technology foundation

Thank you for visiting my blog.

In a previous blog, I presented the case for having a well-defined data strategy, data product definition and data stewardship for successful product management. Suppose you have done the work to define your company data strategy in line with the overall business goals and objectives. If you have also identified data products, the next step will be choosing the right technology. This blog will focus on the intricacies of selecting the right technology for your data product.

According to the World Economic Forum, Data concept goes back to 1960 with the discovery of the Ishango Bone in Uganda. The tribe used the stick to record trading activities and predict how long their food supplies would last. Another notable time in history is the emergence of Statistics and Herman Hollerith's role in creating an automated computation of US census data in 1881, which reduced the analysis of the data from 10 years to three months. The human mind has always been analytical, and there will always be a requirement for capturing, storing, and efficiently processing data.

The emergence of large data centres, the internet, cloud computing and big data has led to a new problem statement. The new challenge is choosing the right technology for managing the large volume, variety and velocity of data produced by business applications and processes. With a well-formulated data strategy, designing a good solution with the right tool will be easier.

There is a wide variety of tools and technologies for data analytics in the marketplace, and selecting the right one for your product is critical for delivering business value. For a seasoned data architect, designing the best solution is underpinned by a good understanding of the business requirements.

A discovery exercise is required for a future-proof solution and successful delivery. From my experience, this is usually a two to six-week workshop where a data consultant and other professionals collaborate with business users to evaluate the current data landscape and produce a report showing the existing data sources, data transformations and use cases. A product roadmap is usually part of the deliverables, depending on the scope and duration. All aspects of the findings are essential for designing a modern data solution.

Let's use a hypothetical example to choose the right tool for Contoso, operating in the retail sector.

The table below presents vital information for Contoso.

Note: This is a simplified summary of the business functions in Contoso for illustrative purposes.

In the case of the example above, let's assume that the underlying business strategy for Company Contoso is to be more data-driven. Here, the data strategy could provide a centralized data platform for users across the business to access near real-time data. This strategy will inform the design of the analytics solution for Contoso, and ultimately, the technology chosen will need to support the delivery of this vision.

The diagram below shows a possible architectural solution for Contoso.

For Contoso to use the design, there will be changes in the current ways of working, data stack and overall technology ecosystem. Again, this is for illustration; a project roadmap will cover each development aspect in more detail.

To summarise, you need a good understanding of the business objectives and use cases to design and build a successful data solution. From the outset, there must be a clear link between technology and business value.

My next post will focus on data quality, one of the most common gaps and issues in the data world.

References

• A brief history of big data everyone should read | World Economic Forum (weforum.org)
• Modern Data Platform using AWS and Snowflake (awsstatic.com)?

?