Data governance: IT projects vs. data quality

Can a well executed IT project reduce an organization's data quality? Yes, here is why.

Declaration: A coffee break explainer. No LLM additives; all is distilled from pure experience.

This is an explainer

Why does even the most well-executed projects and the best PMO set-ups not guarantee better enterprise-wide data quality? The short answer is that they cannot. The reasons are deeply rooted in how organizations and projects work.

This article lays down those reasons and aims to arm data governance professionals with another coffee break explainer: why data governance and enterprise architecture are needed. Even though the organization is SAFe, PRINCE2, Scrum, [insert your project model] savvy.

Observation

The fact is that IT projects generally do not ensure data quality, or a consistent and well-integrated application landscape. This statement is based on 25 years of observations. Why is that? I wrote down my observations 20 years ago. To this day professionals in both the enterprise architecture and data governance space continue to confirm that the causes listed below coincide with their own experiences.

Disclaimer

For the record: this is in no way a rant to belittle IT projects. Let it be clear, individual projects are in no way the target of criticism. Projects operate as per their instructions and mandates, and everybody is working hard to solve whatever is put before them. The thing is that these instructions and mandates are lacking. This causes data quality issues and technical debt to pile up. The further you step back from the projects themselves, the more obvious this pile becomes.

This is about how a do-nothing approach to data governance and enterprise architecture leave projects hung out to dry.

The Nature of Projects

IT projects are temporary constructs that provide lasting products and solutions. Solutions that aim to improve business execution in some way. This also goes for IT infrastructure projects, although the business improvements are indirect. Projects are given a charter that provides a scope and a mandate, i.e., a solution description (a range), and a budget.

There are, aside from the projects themselves, two other important stakeholders: the project sponsor and the project management office (PMO), one to set the scope and one to set the guardrails for the project.

What projects and PMOs do (not)

Projects focus on individual process activities, specific sets of data, and a singular application (and relevant integrations). A PMO’s purpose is to ensure that projects are executed efficiently and meet their defined objectives, timelines, and budgets.

The PMO is not focusing on the specific functionality, data definitions or standards. Nor does a PMO take interest in application architecture standards, integration patterns, etc. All of which are left for the projects to deal with.

Enterprise architecture and Data Governance promote an overarching and holistic view of processes, data, and IT systems. Projects as a general rule do not, due to their temporary nature.

Default Projects

If left to their own devices, projects will have a singular focus on providing a specific IT service to the business. Projects zoom in on providing this functionality within the time and budget given by the PMO. The result is a self-inflicted tunnel vision, that guides the projects’ behavior. The result is that one or more of the following scenarios will be playing out.

Scenario One

Projects are not chartered to set standards where none exist. This would impact other projects with different goals and sponsors (the mind shutters for fear of the ensuring turf wars).

It is easier if the project mandate includes full autonomy over data, including naming conventions, interpretation (definitions), integrations, and sourcing. Being skilled professionals the project team has this in hand within their own scope, but not at an enterprise level.

Lack of focus on creating value for the next project is the IT version of a high-cost short term credit loan, the interest rate is high and pay-back is coming.

Projects does not build for unknown future projects.

Scenario Two

If a project has a charter to set a standard, in addition to providing specific business functionality, this standardization is the first cut-back candidate if (when) the deadline or budget is about to slide. Setting a standard is not really needed to meet the primary success criteria.

Short term objectives will get priority when the budget is sliding.

Scenario Three

If standards do exist, they are not always followed. They might be unknown, hard to get at, not “quite” applicable (not invented here), or perhaps the standard would require a re-scope of the project. Of course, there are also the infamous loopholes and political overrides.

If standards are not rigorously enforced, they are about as valuable as a screen door on a submarine.

This can be reinforced by a governance board that is ineffective, inefficient, with no mandate, or it is simply missing.

Scenario Four

Reuse is often prevented or at least hampered by a DIY attitude. The propensity for entrepreneurship at every level, makes it rare for projects to hunt for reusable components.

People like to invent new stuff rather than reusing what already exists.

A master data repository or other applications that might already capture or hold data is seen as something that is less than cutting edge. Like a second hand car. Hunting down existing data would also require the project to invest time in interpreting and aligning to the validity (fitness for purpose) of said data. Also, the clear-cut documentation tends to be missing.

I, for one, am guilty as sin on this one. Although repentant, there is no doubt that I’ll spend time in EA purgatory.

The result

Data interpretation, formats, uniqueness, validity, and every other quality dimension are drifting in the wind on the enterprise level. Even if the project has "its own data" under control.

This eventually creates an enterprise structure reminiscent of a yarn ball fought over by three kittens.

Experience matters

A high level of experience in a team might oddly enough accelerate this. Project teams that have a high level of experience rely on what they know works. The more proven the routines the harder it is to change practices and preferred way of working. This is not by choice we live by routines and so does project teams. Which is why introducing governance might reduce productivity until the dust settles.

What to Do

Projects fight the same challenges, and individual projects are not equipped to solve them.

Regardless of which of the above scenario(s) that is playing out, the result is an enterprise that looses its ability change and therefore to adapt to the business environment.

What organizations need is governance that supplement the PMO and help the projects however you brand it: data governance, process governance, or enterprise architecture governance. Needless to say (but I’ll say it anyway), you should not try to introduce all three. Pick your battles and choose according to your sponsor’s inclination.

To bring successful governance into this particular theatre of company hodgepodge policies, you will need be helpful. Choose way where both projects and the PMO perceive the governance as a win.

More on this next week.