Why So Many Data Warehouse Projects Fail
Data warehouse projects fail when they are treated as purely technology projects and aren’t focused on the end users and delivering value. Time and time again, projects have stalled because of this: They can’t get investment, and they’re failing to attract a regular user base or different departments that are off building their own things because they’re not getting what they need.
Often, the internal diagnosis of why the project is stumbling will be that the design is at fault. Maybe, the organization believes the wrong technology has been selected, the data models are not good, or the right architectural approach hasn’t been taken. This is symptomatic of the actual problem, the technology-first mindset. Sure, almost any design can be improved upon, and sometimes there are missteps, but these are rarely the fundamental problem.
Instead, the projects have failed to find a customer and focus on their needs. This could be a marketing team who are being provided insights into customers, allowing them to target their audience better; it could be a sales team that wants a better handle on how the sales pipeline is performing; it could be the operations team that wants to understand where the bottlenecks and issues are so they can fix them.
Why do so many fall into this trap?
There are several reasons why businesses get themselves into this position.
Sometimes, it’s because data is being led from the wrong place. For a long time, data was owned by IT departments whose experience and expertise were focused on implementing systems. Inevitably, data warehouse builds got treated in the same way as everything else. Having data led by a chief data officer instead can help avoid this problem, as these leaders tend to be focused on outcomes and the use of data, ensuring the technology functions as just the enabler.
A misdiagnosis of the challenges teams face can also lead to the wrong solutions being proposed. As people’s primary interaction with a data warehouse is with tools, their complaints and issues often focus on those same tools. However, when you actually sit down with the users and peel back the problems, it is often a combination of the data they need not being available (or just plain wrong), a lack of skills, training being unavailable or just a poor understanding of what exists and where to find it.
Technology vendors also carry their share of the blame. How many times have you walked around a conference floor and seen a technology vendor position their tools as the panacea to all your data problems when really it’s just a database or BI tool? It can be a hard market to navigate, and I don’t think I’ve met anyone in tech who isn’t interested in the latest and greatest tooling. The opportunity to put something new in place can lead people down the wrong path and focus them on the wrong things.
So how do you prevent your project from failing?
Thankfully, not all data warehouse projects are destined to fail. There are a few mindsets and approaches you can adopt that will help ensure success.
The most important thing to do, which we’ve already touched on, is to focus on delivering value. Before even thinking about designs or technology, data teams successful at building data warehouses start with understanding the business use cases and work on defining a data strategy. Without a focus on value, and demonstrable results over time, data warehouses quickly become giant cost centers.
Framing the work against a wider data strategy also helps make sure you’re considering more than just technology. A good data strategy will look at value, organizational design and skills, processes and data management. All these things must come together to make a data warehouse project work.
Throughout the whole build process, it’s crucial to establish a close working relationship with the end users. Product mindsets and approaches are incredibly valuable here. Work with your business to understand their needs and requirements, build MVPs and iterate the solution piece by piece, constantly collecting and reflecting on feedback.
Ultimately, if you want to make sure your data warehouse project isn’t going to fail, then don’t treat it as just another technology implementation. Find your customer, focus on the value it can deliver for them and make sure not to forget how the platform will operate in the long term.