Know Your Data......(KYD)

Know Your Data (KYD but pl do pronounce /k?d/)

A key challenge executives face while taking business decisions - 'is the data reliable'? Unfortunately, they may end up taking ill-informed decisions, knowing data may not be trustworthy. Outcome of such decisions may result in business, governance, regulatory and many other challenging issues. Unless executives and leadership get insights basis trustworthy data (of course context and gut still matters), we are not enabling better business decisions.

How can we solve this problem? Why does it even exist, despite having programs such as data governance, data quality etc? In highly digitized and largely automated environments, will this problem get addressed or will it get further amplified? How do we know the data we rely on is trustworthy?

We need an open and happy mind, just like a school going kid, to solve any complex problem. Lets start KYDding first...(i mean learning from each other, well that what kids do...). Here we start, five fundamental things to help us know our data better.

1. Data Provenance - Where did this data come from? Not where you just found it or got it. Actual point of origin. It will be a fun exercise to do this and may become scary too, when we cant establish the origin

2. Data Lineage - what happened to the data between its point of origin till last known destination? If you were to draw swim lanes for the journey of your data, all the hops and changes it went through

3. Trustworthy Data - Why should i care about point of origin and its transformation enroute? Well, if we dont, we dont know what we are relying on? What if it got corrupt in between? What if its no longer available? What if it got intercepted? What if data is biased? who owns the data? who manages it? who makes it available in an agreeable manner? Do we have a contract with the owner?

4. Data Dependencies - What if data dependencies get impacted? Before that, do we actually know our data dependencies? How do we identify them??

Almost a decade or more back, when i worked with my Guru @Todd Aven, on understanding control design, i was surprised to experience various dependencies of a control (literally control design components). Interestingly, across these (control activity, control solution, control reference data, control reference data source - like a system of record or authoritative source and not just any copy, control data transport solution and control performance data or records evidencing control effectiveness), it is a lot to do with data and how it moves and reaches the control. I stumbled upon these dependencies and when we shared with everyone around, it was like 'Gosh, we never even know about these, forget worrying about these'.

Truth of the matter is, many start discovering these after something happens to business...

Now, imagine a highly digitized or a largely digitized environment enabling your business function. What if data being consumed by your business solution gets impacted? GIGO, needs an intensified expression of choicest words to explain the mess. Let me try, DADADADADADADADADA.........?

5. Data immutability - Can someone alter your data or modify or delete it? If they can, what happens to you and your business? Do our existing data lake, forest, store......support us here? Why we need this and how does it help?

Not completely KYDding yet. We did a start. Not sure if we know data better yet, but we now know data knows us better than we know.?

Looking forward to your views on things which help us KYD better.....