How to design enterprise analytical applications that users trust and love?

 Analytics is the transforming power in the business of today. Therefore, pretty much all companies are currently designing, and redesigning how to serve user base with analytical solutions. While there are many successes, frustratingly often there are also failures.

We at Sievo have been in the business of creating procurement analytics software for last couple of decades already. We really have had the opportunity to learn, together with our clients and often in a hard way, how to create sustainable, value-adding analytical solutions.

There are many ways in how designing an analytical solution is similar to designing any good software. However, there are also multiple aspects where there are clear differences to generic software development. Whether you are building your in-house solution or are considering a 3rd party software vendor, we hope that these guidelines help. Based on our experience, there are twelve things people get wrong when designing data driven analytical solutions.

1.   Accept that data is always dirty

Data is the fuel for insights and basic building block of analytical applications. However, way too often designers and creators foolishly expect that data entering the application is of high quality. When this is not the case, users get frustrated as they cannot trust the numbers. At this point, solution owners wash their hands as it is ‘not their fault, but master data is not good enough – garbage in, garbage out’. If you want to create an analytical solution that actually works, you need to design it based on the assumption of dirty data – not use dirty data as an excuse. Seriously – if you design your application assuming high quality data inputs, you are setup for failure. Your task is not to create fancy charts, your task is to create reliable analytics. And, no, the solution is NOT to start with fixing the master data.

The solution is to build the analytical solution based on assumption that data will be dirty. In practice, this often means both building ‘in-analytics data cleansing’ solutions (e.g. spend classification in our context), and feedback loops to source system master data fixing (these master data fixing loops work for internal data only, though, see #5). You can only create demand for better data by exposing the business challenges created by bad data. The key is that these feedback loops are near-immediate – the delay between a user seeing an issue and it getting fixed should be between minutes (easy fixes) and days (more complex issues), not days, weeks, or months. Too slow feedback loops take out the incentive from users to report issues, which leads to a deadly spiral of non-credible numbers, diminishing trust, and, ultimately, solutions that fail to deliver value and become jokes.

2.   Don’t deliver death by dashboards – analytical use cases have a long tail

Good software products are designed to target certain use cases. This also applies for analytical solutions, however with a twist. When designing systems to support operational processes, it is relatively easy to isolate certain use cases, and design functionality to match that. However, in analytics, the number of relevant use cases is large, and the frequency of most of those use cases is very low – analytical use cases have a long, long tail. This is obvious when you consider how people use Excel – it is not an accident that pivot table is one of the most powerful and popular functionalities. It is not designed with any specific use case in mind, but it provides users a way to look at data from different angles.

When designing analytical applications, those on the building side have good intentions to cover the use cases, and approach users. And the potential users come up with dashboard designs, that they believe will solve their use cases. It is not that all dashboards are useless, but most of the dashboards designed get never used, and 2-3 years down the line, nobody understands why they were created in the first place. The lesson ? Invest in building great, flexible self-service tools, rather than delivering suffocation by dashboards. It is hard to avoid creating dashboards (it’s fun, visual, and effective), but sometimes less is more.

3.  Don’t care if it’s powered by hamsters – as long as it works

? Powered by AI ?. It’s not possible to find an analytical solution nowadays that would not claim that it’s powered by AI. Most of them have only a relatively small, if any, AI-component in them. There is just so much hype around AI (and before that chatbots, predictive analytics, mobile apps, etc.), which pushes boardrooms to pressure executives, who pressure analytical solution designers to have AI/ML component as part of solution. This, combined with vendors’ marketing teams who feel that they must join the game – and, they must, as any vendor not having AI as part of stack will be excluded as old-fashioned.

This just doesn’t make sense. Yes, for some applications AI/ML adds value – but for many it does not. And, even when AI/ML is in the game, it’s a small part of an overall solution. Nobody is asking if the analytics UI is developed with Angular or React. But many are asking if there’s AI in there. These are equally relevant, or perhaps irrelevant, aspects. I once heard a very clearheaded comment – ? I don’t care if your solution is powered by hamsters, as long as it works ?. We should focus on delivering business value, not insisting on the use of certain technology.

4.  Design for human-machine collaboration

When, and only when, there is a use case for AI/ML, you need to design for human-machine collaboration. In almost all analytical business applications where AI/ML is applied, it has a role, but a limited one. Input from humans is required, and not only as one-off exercise, but also on continuous basis. A good example in the procurement context is spend classification – AI/ML can provide value, but it requires feedback from humans, both during the initial training phase and on a continuous basis to keep data quality high. And there are both areas where manual interventions should dominate and areas where AI/ML should be the primary driver. When designing analytical applications, the challenge is not to apply AI/ML – the challenge is to create the feedback loops where users (and not data scientists) can continuously provide feedback for AI/ML.

5.  Design for not just drill down, but alert up

Good analytical applications provide an overview, and enable users to drill down, and slice&dice data in different ways. Great analytical applications go a step ahead. The business logic captured in analytical application looks for opportunities, risks and irregularities, and highlights those for users for further action.

6.  Don’t design applications, design systems of engagements

Yes, you have designed a great, usable analytical application. As you want users to use it, you maniacally measure user adoption rates, usage session lengths, collect NPS data and compare how popular different dashboards and reports are. You interview users to get qualitative input. You conduct trainings and internal marketing to drive adoption. All of this is great, but it misses half of the potential value that your analytical application can deliver.

Yes, as users interact with your analytical application, they capture value. But, what you actually should do, is to design systems of engagement. How the data enriched by your application, as well as analytics and insights created, can be used both directly (using application), but also indirectly (embedded in other solutions). In other words, you need to design for systems of engagements, not just design for application.

Your job is NOT to get happy users on your platform. Your job is to create business value – and often this happens best outside your application. Some examples of indirect engagement are:

• Enabling download of data to Excel pivot :)
• Sharing findings and data with e-mail (automated push reports and manually)
• Embedding analytical data in operational systems
• Pushing your domain specific data to corporate-wide data lakes
• Integrating your analytical solution to collaboration tools, e.g. Slack
• Creating ways to share data with external stakeholders (customers, suppliers, partners

7.   Cure from internal data myopia

Traditional data warehousing systems were based almost solely on internal data. Well, previously, that was pretty much all the data that was available. This has changed drastically. For most business domains, there is an increasing amount of external data available, both on free and subscription basis. However, as designers of analytical applications are often internal IT people with limited domain expertise but good understanding of internal operational systems, they are greatly aware of internal data assets, but very poorly aware of potential external data assets. There’s substantial internal data myopia. Look up, see all the domain data available, and combine that with your internal data.

8.  Identify the core data structures for your analytics

There’s lot of data, and multiple different aspects. At some point you need to separate core data structures from the important but less relevant ones. Clear understanding of what are the core data structures helps you to design both the data models and the UI to meet business needs. In the procurement context there are three:

• Categories – to help procurement people. Pretty much all procurement work is centered around the concept of purchase categories. What services and products are being used.
• Suppliers – to make things actionable. Categories exist only in people’s heads, whereas suppliers are real. That’s where the action – savings, innovation, risk lives.
• Business units – to relate to the rest of organization. Nobody outside procurement cares about categories or suppliers – procurement’s task is to relate insights to how it impacts different business units.

 9.  Apply the super glue to link unrelated data sets

The more you source for external data, the more data you will have that is specific to certain core data structure only. For example in procurement context external risk scores are related to a specific supplier, but do not relate to category. Market prices relate to a certain commodity, but have no direct link to suppliers.

It is great to have that new data, but it needs to be linked to internal data sets. To do this you need a super-glue with two components :

1.     You need to match external data records to internal data records – for example linking external system vendor ids to internal system vendor ids. It’s easy to do as a one-off exercise, but you need to create a process around this as both internal and external data sets evolve over time.

2.    You need to find way to relate data that is specific to certain core data structure only (e.g. vendor risk scores) to the overall data set, so that you can for example analyze ? How big share of our sales is at risk due to this supplier having issues ?. In the procurement domain, spend transactions are this second component of super glue.They help make "supplier-specific" external data to apply also to category level information.

 10. Create a support model that can explain the data behind the analysis

The clear #1 reason why analytical applications fail is that users do not trust data - too slow performance is probably a distant #2 when working with large data sets. Users’ challenge is primarily not that they would not know how to use the tool. Their challenge is that they cannot recognize and trust the numbers.

 Sometimes users are right, there is a mistake in data – this you need to fix (see #1). More often, data is technically speaking right, but users feel it is wrong. There are million reasons for this – typical ones include : (i) user assumes a different definition of a metric than what has been applied ; (ii) different concept of time (e.g. user is assuming ‘when invoice is paid’, analytics is built based on ‘when invoice was posted’) ; (iii) different currency rates applied; (iv) user is seeing just part of data due to security restrictions or filterings. Documentation and training can help, but there will be these questions – and lot of them. And, unless you can address these questions quickly, distrust in numbers will grow quickly, even if they would be technically correct.

But – your task is not to create reliable numbers, your task is to create business value. And one crucial component in this is having a user base that trusts the numbers. To address this, you need to build a support team that has both competence to answer these questions, and empathy to serve the users. The competence part is surprisingly hard – the right people will have enough domain expertise to understand the user’s business problem / view, enough analytical tool capability to understand how analytics work, and also an understanding of how the data has been created in underlying source systems. What makes getting this right even harder is that you need to find people who have both competence and empathy – either alone will not make it.

The success of your analytical application is not only driven by how your good software and wider system of engagement are, but also by how good person-to-person customer interactions you can create.

 11.  Design for continuous operations and change

Building an analytical application is one thing. However, the true lifetime cost of analytical applications comes from continuous operations and managing change. When operating a large analytical application, there will be surprises down the way. It is relatively easy to keep integrations technically running and monitor them. The tricky part is that the data content keeps changing.

New master data records are created, but this, I hope, you saw coming. However, there are more complex changes – source system configurations are changed, new document routing practices are established, source systems get upgraded, and new data sources pop up. Some of these are breaking changes, and are easy to spot, but nevertheless require urgent attention. Some of these bring new kinds of data to analytics, that you did not plan for, and once somebody spots it, you need to adjust. You should plan for continuous development of analytics application – initial release is just the start.

 12. Fight for free data

I really love to hate user specific data security limitations. Partly I get it – large organizations need to limit risks by limiting access to data by regions, business units and so forth. However, I feel most companies overdo this. Mediocre managers who are afraid to make a mistake, play it safe, and apply an ever-increasing amount of security constraints. User x can only see data for this country, but when it is these metrics, only for this country in this business unit.

There is pain to this, lot of it. Creating and maintaining user access settings becomes an operational burden. Multiple security limitations result in lot of confusion (two users seeing different numbers), driving trust in data down. Those well designed dashboards look embarrassingly silly when data set is very limited. But it’s more the lost value I’m afraid of. When seeing only data for your country or unit, you miss out on all the benchmarking opportunities. In short – fight for free data.