Data Quality Controls Assessment: More Than An Audit
First published on my website (Data Quality Controls Assessment: More Than An Audit — AHUJA CONSULTING LIMITED)
As a CDO or CRO, you know that your organisation runs on data. But without rock-solid quality controls, that data is not the asset it could be and, worse still, could even be a liability.
Yet so many organisations can’t seem to find a balance.
They're either being strangled by excessive controls that are inefficient or relying on partial and ineffective frameworks that leave them exposed to unexpected risks.
The price of getting this wrong isn't theoretical. We're talking:
You need a robust assessment of your data controls.
This isn't just another audit.? A Data Quality Controls Assessment is your organisations reality check on whether you’ve got the right controls— whether they're up to the job of maintaining data accuracy, completeness, consistency, and reliability across your entire critical data lifecycle - and whether you’re getting bang for your buck.
The Critical First Step?? Forget About Controls.
Your first move isn't examining controls—it's understanding risk - and there are two layers you need to dig into.
Layer 1: Framework Scope Assessment
Here’s a question: Would you install a security system without knowing exactly what you're protecting? Of course not. Yet organisations will happily build a set of data controls without clearly mapping all downstream use cases. This is not only expensive, but can leave you vulnerable where it matters.
Your mission: identify every downstream use case that needs protection. If your framework doesn't start with this, you're building on quicksand.
The best way to do this?
Think of your organisation’s critical data-dependent processes and ask yourself what happens when data goes wrong.?? Could it result in fines, brand damage, greater expense or hits to the bottom line?
You need to check whether your Framework takes these use cases into account.?
A robust Data control framework will be built on the foundation of the data use cases that can cause harm to the organisation.
But there’s a flip side.?
Bear in mind that all controls come with a price tag.? So to properly assess your control Framework, you also need to have analysed the impact of bad data hitting your critical use-cases.
Remember: Not all of your downstream uses need perfect data.? A Data Control Framework that aims to supply 100% accurate data when you only need 80% is a Framework that’s costing your organisation time and money.? It’s gold-plating.
So be clear on the level of control required (your risk appetite) and make sure you’ve designed your controls accordingly.
This is where Layer 2 enters the scene.
Layer 2: Data Flow Risk Analysis
Once you know what you're protecting and the degree of protection required, it's time to examine the granular controls focused on how data is created, augmented and flows through your organisation.
This requires two parallel tracks:
Neither approach alone is enough. You need both.
The Subjective Assessment: Control Design Based on Risk
To properly assess the risks and controls in your data flows, you’ll need to map the steps, handoffs and transformations with surgical precision.
Here's the template I use with clients for each of their critical flows, pre-populated with an example from the insurance industry:
领英推荐
Inherent Risk = Risk before controls applied
Residual Risk = Remaining risk
The example details the process steps and flow of data upon which the reinsurance recoveries process is dependent and spans both the reinsurance policy set-up process and the recoveries activation itself.?
Put simply, the process of recovering a loss is dependent on the upstream entry of data, which when accurate, triggers an automated recovery. So the process steps and consequent Inherent Risks, as well as the existing control suite are made transparent in the template to enable a robust control assessment.
The breakdown allows you to assess the Inherent Risk in each step with a high degree of precision, as well as the Residual Risk, after controls are taken into account.
For each control, you just hammer these questions:
Let’s take a closer look at the example: the QA sampling. Sure, it catches issues—but only in the selected data entries. Is that enough?? In the illustration, the sample is 40%, but, for this process, the data needs to be right 100% of the time.
Calibrating the sampling size is ultimately a business decision.? The bigger the sample, the greater the cost in terms of resource.? Therefore, in our example above, the QA was supplemented with an exception report to catch any mistakes and reduce residual risk within appetite.
If you only have a partial QA and you’re not sure whether to increase the selection size or augment it with another control, this is where the objective review can lend greater insight by allowing you to assess the risk through another lens.
The Objective Review: Data-Driven 20:20 Vision
Your controls may look fine on paper but what is the data telling you?
The objective review is a window into how well, (or not!), your controls are actually working.
How? By being data-driven.
You need to look at real data examples and profile them for completeness, accuracy and consistency.? The devil is in the detail here and it's the anomalies that you need to focus on.?
Let’s revisit the example above. We found that the spot check compensating control was uncovering a higher rate of missed recoveries than was initially envisaged. Sure, it worked great in picking up the missed recoveries, but there was still an impact to the organisation - insurance companies need to recover their losses quickly! So whilst the overall suite of controls was deemed to be acceptable on paper, a re-design of the QA sampling methodology was immediately carried out in light of what the data was telling us. We needed to make the recoveries process slicker and that meant beefing up the primary QA controls!
By reviewing your controls through both subjective and objective lenses, you’re giving yourself 20:20 vision.?
When it comes to ensuring you have the right data controls in place to give assurance, why settle for anything less?
The Bottom Line
A data quality controls assessment isn't a tick-box exercise—it's a systematic examination of your entire data ecosystem.
Get it right, and you've got a competitive advantage. Get it wrong, and you're betting your organisations future on hope.
?
Coming Next:? Testing the operating effectiveness of your data controls to ensure they are delivering value
?
Subscribe here to get future articles in this series.
--
Need Data Governance help?
?
Book a call here to discover how we can support you.