The Pitfalls of AI-based Data Analysis

As companies start to turn to AI to gather and analyse their data, there is the risk that much of the subtlety and nuance needed to make the best use of the information will be lost. The previously vital information companies received from things like employee surveys and customer feedback will become bland, generic, and over time, useless for businesses.

Continuing with the example above of employee surveys, knowing that some people aren’t always truthful with their responses, good data collection methods can produce rich and valuable information. If respondents are able to add their own comments and share their true opinions, the data collected can be high quality, no matter who (or what) is doing the collecting. If not, the results may not truly reflect what’s really going on.?

Additional problems can occur when companies decide what to do with the data they have painstakingly collected. Leaders who truly want to understand the data and its meaning to improve the working lives of their employees, or enhance performance, or improve the customer experience, will use the data as a starting point for more meaningful conversations and analysis. Others, however, may see it as a tick box exercise giving the information, gathered for example via a chatbot, a cursory glance, never to be discussed or looked at again. This loss of potential benefits or positive impact from analysing the data will, of course, be lost. In such cases the lack of understanding at grass-roots level, could lead to lost opportunities to rectify, for instance, potential staffing issues, or business process problems, or causes of employee attrition.

Analysing The Data

The limitations of questionnaires are well known - the reluctance to complete lengthy surveys to dig deep into the employee experience is clearly demonstrated in the poor response rates. To truly understand the attitudes of everyone in an organisation, surveys would need to have hundreds of questions - something no employee would relish. As such, questionnaires are whittled down to the leanest - the bare minimum of questions, to make it quick and easy to complete, so that as many people as possible can participate in the time they have. Average surveys are dropping from around 45 to 30 questions, with pressure to have even fewer and take even less time to complete. Anything taking more than 10-15 minutes is often frowned upon. With all the best will in the world, and even with high response rates, there is only so much information that you can yield from such a short questionnaire.

The real value of any data comes when you actually get a group of people together and ask, ‘What does this data mean? What is really going on here? Is this a problem? If so, why?’? The value lies in the exploration and deeper dive into what is important for people, and how things can be improved - even just a little bit. People want to be heard and feel their opinions matter. Even if things can’t change, they need to know someone cares enough to listen to what they have to say. This is likely to be lost when we rely solely on machines to analyse such potentially valuable information.

At WorkingWell, we deal with data all the time. We create anonymised reports from aggregated data - meaning that we can guarantee individual anonymity within our reports? This creates a level of ‘safety’ for respondents, as they know they can be honest without their individual responses being highlighted. Creating this level of trust enables frank and honest discussion about what the data means to those discussing it. Offering a safe space to break down the data into small components, to explore what the aggregated responses mean in real terms, and to be able to lay ‘cards on the table’, enables issues often deeply buried, to be brought up for discussion and steps towards resolution.?

Clearly such data is just a snapshot in time. However, as we have seen, it does provide a very good basis and starting point for important conversations. If the opportunity to look more closely at the data and discuss it is ignored, it is likely there will be missed opportunities to understand how to make improvements. Such improvements are not limited to an individual level - they can also affect the team level and even the organisational level, if those at the top are brave enough and inquisitive enough to know what really goes on in their organisation.

The Impact on Collective Corporate Intellectual Capital

Data gathering and analysis is, of course, not limited to employee surveys, it applies to data in all aspects of business. When companies only use generic methods to gather and analyse their data, whether it’s competitor information or predictions of future trends, this often leads to generic intellectual capital. This means, in theory, every company could gather and have access to the same sort of data. There is nothing ‘special’ about the data collected, if that data is not discussed and analysed further.

If we start relying too much on AI, we may all become a bit ‘vanilla’. Companies lose their USP. Let’s use the example of consultancy firms - why would people pay big money to a company, when they have access to the same information themselves??

Generic Information Leads to Generic Solutions

At WorkingWell, we know from experience that when companies hire a consultant, they’re paying for personalisation, knowledge, and real-life experience. People buy people.

Using AI to collect data is a great way to save time and costs in the short term. However, when applying the data to areas such as learning and development, coaching, or consultancy, our clients are buying our skill, experience, and knowledge of the nuances of human interactions and behaviours, enabling us to create bespoke, personalised solutions to meet their very specific and unique needs. Every group of people has its own dynamic and requires a tailored solution, and no two individuals, no two teams and no two companies are the same.

When we walk into a company’s offices for the first time and meet the team we will be working with, we’re constantly taking notice of the specificities and intricacies of the company. We consider the internal language, internal politics, and the types of personalities in that company. We make it our business to become aware of the relationships between team members and managers, between managers and leaders, between leaders and support services. We use our people skills to help us to understand the unique pressure points each stakeholder group may be experiencing. This allows us to create a truly bespoke strategy for each business that actually works.

Conclusion

We believe AI technology can bring many benefits in a wide variety of areas to businesses that know how to use it properly. When we rely solely on AI to capture and analyse data, we are in danger of missing much of the nuance and subtlety that humans bring to the table. However, when we use AI effectively, as a part of our toolbox, and make time and space to dive deeper into the data, we retain the benefits of the human touch, while still making the most of the amazing new technology at our fingertips.