The Seven Steps to a Better Understanding of Data in the Articles you Read

More than ever before, the every-day person is inundated by news articles featuring or based on the results of research. And yet like most advancements (because the depth and breadth of scientific and consumer research now undertaken is indeed a step forward), it feels like it’s made our life more complicated. It can definitely feel like it’s only ever used to tell one side of a story (which of course is because you’re reading an article driven by the fact that something newsworthy had shown up in some published research), so it’s up to us, the reader, to add in layers of context to understand what we’re reading.

In this article, we’ll take you through some steps that with practice, you will be able to do almost instantaneously every time you read an article.

Check Your Ego

Step 1: Check your Ego. You may be reading something that you don’t like to see, or that is contrary to your previously held beliefs. And heaven forbid, it may well be right, so check your damn ego first.

Why am I reading this?

Step 2: Where am I reading this? Alas, not all news sources are of the same quality. Not all journalists are well trained in how to interpret and explain research, and not all journalists have the time needed to examine and question the research they are seeing. In fact they may not even have the space in their article to include these comments.

What’s the data source?

Step 3: What’s the data source? Is it reputable, is it even referenced in the article? Scientific, medical, and other research journals go through extensive work to confirm that data is correct and hold themselves and their writers to very high standards, with massive implications for the contributors and themselves if these are not met. Also all of these fields have had long histories of people who have produced findings that have contradicted previous understandings. More than the rest of us, they understand. I’m just saying.?If you have the source, and you want to know more… well that’s what the internet is for, go off and find it.

Why was the data collected? Why was this data released? Why are you reading about it?

Step 4: Why was the data collected? Why was this data released? Why are you reading about it? Just because research is sponsored (it’s expensive to do things right you know), doesn’t make it wrong (See Step 1). But… you should definitely keep in mind that you are seeing this information for someone else’s reason. Otherwise research happens because there are questions that someone wants to know. And you’re most likely reading about it because it’s shown something that is interesting or different.

What were the questions the respondents were asked?

Step 5: What were the questions the respondents were asked? We’ve got through all of the external factors, now we need to look at the information itself. It’s rare for the actual questions to be published in the article you’re reading, all you see is the results. Which is fine but not all questions are the same. Yes/No questions produce different results from “select 1 out of X options” and even more when you see a question asking you to “select any of the above”. The problem for the casual reader is that all results are expressed as a percentage, and if how the research is gathered is not explained clearly, then it’s easy for the reader to misinterpret it. I would like to say that it’s not common, but alas, by the time the research has made it through the researcher, through the writer of the article… it may not be so clear.

Who was the sample? When did the research take place?

Step 6: Who was the sample? When did the research take place? The sample used for the study, or the period in which it was completed is also important. This will not always be shown in the article you are reading but both are important to consider. If the data refers to consumer behaviour from 1990… it might not be as relevant as you’d like. If you are seeing data on vaccination update from Japan, it might not be that useful for you in another country. If the article doesn’t tell you either… then you might want to question all of what you’re reading.

What are the numbers telling me?

Step 7: What are the numbers telling me? I once read an article that told me 45% of boys said their family had a tablet, while 50% of girls said their family had a tablet. The only logical solution to this piece of data came from a colleague who replied with, ‘We’ve bought a tablet. Don’t tell your brother.’

In order to explain some of the mathematics of statistics, we’re going to use a real-life example. I won’t provide the source because the culprit really, really should know better.

In a nutshell, the research was trying to find “the most likely reason to cancel or stop using a paid [online] service”.

The reported results – shown via a graph and table – told us that 49% of respondents said, ‘If the price for the service increased’, 31% said ‘content I liked being removed’, 31% said ‘finding another service that has content I want more’.

Let’s have a think about this.

1.??????The report told us the information we’d be reading was “the most likely” reason consumers would stop using a paid [online] service. This implies that respondents choose only one reason. But (I’m sure plenty much all of you spotted this straight away) quick mathematics will tell you that 49% + 31% + 31% is greater than 100%. And therefore the question must have been phrased along the lines of “pick any answers you think”. Not only does this negate the “most” bit of the information, but if this type of question was used, it creates other less-discussed impacts.

2.??????If the respondents were asked to rate from “highest to lowest” of a list of reasons, they would be forced to consider all the options. However when you ask a respondent to select as “many as you like”, the way in which a respondent behaves can be very different.

a.??????There is no way of telling the order or value a respondent places on any selection?

b.??????Each respondent can behave differently. They could feel uncomfortable selecting all the options available even if true, so change their behaviour to satisfy the moderator even if they are doing it online (something called acquiescence bias). They might misread the question and only selected one answer. But how can you distinguish a mistake from someone who honestly only meant to select one response.

3.??????Asking questions about price is fraught with difficulty because price operates on a linear curve. Is the price increase mentioned in the question above one cent, or one hundred dollars? As the law of supply & demand shows, everyone is affected by price. If Netflix started charging a million dollars a month, or if it increased its?price from 7.99 to 8.00, you can be sure that the number of customers dropping the service in either situation wouldn’t be 49%. So the true answer to the question is that every customer is influenced by price, and we don’t need a survey to tell us this. What of course is more interesting for any VOD provider is exactly how price sensitive their customers are.

Hopefully this article will provide you with different questions to ask yourself the next time you see an article based on new research findings. And feel free to message me to find out who inspired this article with their misquoting of their research findings.

The author has worked in Finance Transformation for twenty years, with experience in the licensing, media, and advertising worlds. With a past academic background in Consumer Behaviour he loves to hear from all sorts of people, their stories, challenges, and interesting insight they have, that can help others and make us think.