Data - a strategic or poorly maintained asset

OK, so we all know about the explosion in the amount of data being generated nowadays and the amount that businesses are collecting.  It's only going to continue to grow, if not accelerate.  Here are a few soundbites that may be of interest:

• In 2020, there will be around 40 trillion gigabytes of data (40 zettabytes) Source: EMC
• 90% of all data has been created in the last two years. Source: IBM
• By 2020, every person will generate 1.7 megabytes in just a second. Source Domo
• In 2020, the big data market is expected to grow by 14%. Source Statista
• In 2012, only 0.5% of all data was analysed. Source The Guardian

To me, the last point is still the most remarkable. Despite many business people acknowledging the growth of data, they still don’t appear to be managing and analysing that data to create real business value

Simplistically, you can break this problem down into two parts: 1) people manage their data poorly and therefore they don’t know what data they have and its quality or 2) they don’t have the right tools and skills to exploit that data.  Although I believe most organisations suffer from both problems, let's look at the first one today, as it would be all to easy to start to talk about analytics, AI and training and gloss over the data management issue as many people do.

So why is this?  Well, I think it is because most businesses don't think about data as a business asset and don’t invest in it and manage it accordingly.  Like any asset, physical or intangible, it needs to be managed and used by suitably trained people inline with a set of policies and it also needs to be maintained accordingly.  Most of all, it needs to be treated as part of the business, owned by the business and managed as a as service for the business to use.  

Like any asset, or service to that matter, data has a lifecycle and each stage of that lifecycle needs to be considered:

• Strategy - what business goals are you trying to achieve and how does your data strategy enable those goals.  Are you looking to create revenue from its use, reduce operating costs, improve your decision making, or all three - its important that you understand what you are trying to achieve in order to shape the decisions below.
• Create/Acquire - what data can you generate yourselves from your customers, employees or sensors and what data do you need to acquire to enrich your own sources.  Are you missing any data sources that could be of real value to your business.
• Storage - where will your data be stored: cloud, production system, archive etc and will you physically centralise the storage of your data or virtually centralise it.
• Identify Appropriate Data To Keep - even though you may collect vast quantities of data, not everything needs to be stored and maintained.
• Maintain/Manage Data Quality and Metadata - although we’d all hope that the data we capture is of high quality, it is unlikely it will be, therefore work is often needed to improve the data quality so it can be relied upon. This is not a one-off exercise, but repeated throughout its lifecycle.
• Curate/Make Usable - Organise and define your data so that end users can find and understand it - no one understands the undecipherable application names given to data fields, so often it needs to be transformed into something usable and understandable.  Creating Master/Authoritative data sets is also an essential part of data curation
• Access/Publish - make the data accessible via controlled APIs for business users and data scientists to use, with access privileges clearly defined to safeguard the business' data.  If data is not accessible or cannot be integrated into business processes, then it's value is minimal.
• Use - data environments and analytics tools need defining - there will never be one tool to rule them all, but its important that there is consistency across the business otherwise cost will spiral and reuse will be minimal.
• Archive - given the above, there is a cost to managing data, so be prepared to archive data that is not regularly used.
• Purge/End Of Life - when the business has finished using or deemed data has no value, be prepared to purge it.  Although storage is cheap, it still costs money to hold data.
• Training - all of the above needs skilled people to define, implement and manage your data strategy

 On the face of it, you'd think the above list is enough of a challenge, but there are also a number of other areas to think about if your strategy is going to be successful

• Cultural challenges of sharing data across the organisation - usually everyone is happy to consume high quality data but a far smaller number are happy to invest resources in managing and providing that data as a service for others to use
• Everyone wants to choose their own tooling and few want to have to comply to a corporate tooling strategy. The interdependencies between data and technology make this issue a key one to address
• Along with security, data needs to be a fundamental element of any solution design from the outset - today is often an afterthought.
• Speaking a common language - invariably every organisation will be starting this journey with a wide range of legacy systems and data stores, with similar data described and labelled differently.  Defining and implementing a common taxonomy and refence data sets across the business is essential to managing and leveraging data as a business rather than in silos
• Compliance - no one ever wants to invest in compliance, but with fines such as GDPR breaches now up to a maximum of 4% of Global Revenue or €20m for Public Sector organisations, compliance is something that needs to be an integral part of data management

So, a few final thoughts.

I have had a variety of roles through my career and I like to think there is an element of each role that is similar to the strategy, management and use of data:

• Finance (albeit a very long time ago) - understanding the value of something, enabling the business to create value and building a core discipline stays with you in whatever role you do.
• Product Management - understanding the end to end lifecycle of products, services and assets, bringing them to life and into service, creating value from them and knowing when to retire them from service.
• Entrepreneur - focus on the core things that are key to success before moving on to the next goal.  Don’t try to do everything at once.
• Innovation - creating new insights and new ideas from collaboration and experimentation, whilst also being able to turn those ideas and projects into exploitable benefits 
• Data, AI & Automation - well, it was bringing all of this together really.

Why the picture above?  Well, you wouldn't manage you physical assets in a way that lets you down.  You'd maintain them, fuel them and train your people to use them…….you'd treat then as a strategic asset.  It is now time for the same focus to be placed on data.

Let me know the approach being taken by your business or organisation