Big Data, Big Decisions - Where do I start?

To get to the finish, one must start.

When asked what insights derived from big data would be helpful to respondents, the answered varied. What is important to one firm, may not be important to another, but it was evident the common thread among all participants was a more comprehensive understanding of their customer.

68% of respondents listed a 360-degree view of their customer derived from data science as “very important." 59% of respondents listed retail habits of their customer as “very important.” Advertising insight was of interest to this survey group as well, with 36% listing it as “very important’ and 41% listing it as “important.”

One respondent stated that “real-time content consumption/sharing data” and “return on investment on content and sales initiatives” were incredibly important to them. Another stated that "a better understanding of what drives our customers to their marketing funnel” and “what interactions our prospective customers have with us” as crucial to their success.

41% of respondents listed cost savings through operational improvement as “very important” with an additional 41% listing it as important. 36% of respondents listed information security as “very important."

The types of insights valued by participants are becoming increasingly accessible as embedded sensors in consumer products, mobile phones, and physical spaces proliferate (Power, 2015). Beacon/sensor technology connects the physical and digital world and provides firms with unique view of their customers. Proximity awareness platforms allow businesses to deploy their own beacon/sensor technology, which tracks movement, allows for two-way dialogue, and customized experiences for customers. This technology can lead to better passenger travel, a customized car buying experience, better flows of traffic, and more efficient resource deployment. This web of beacons/sensors, coined the internet of things, will generate massive quantities of new data, leading to insights never before thought possible.

WHAT FIRMS COLLECT

The firms interviewed collect a variety of data including inventory data, in app analytics, construction cost data, and sales data. The survey asked businesses about the type of data and the results, shown in the graphic below, reveal that financial, customer surveys, social media, and website analytics data are the most popular types of data collected. An average of 64% of respondents collect financial, customer surveys, social media, and website analytics data. An average of 31% of respondents collect geographic, business system, and machine-generated data.

An argument could be made that the majority of the data that participants are collecting is thick data. Thick data is defined as “data generated to observe behaviour and its underlying motivations” (Rasmussen and Hansen, 2015). Organizations are collecting large quantities of data that is “either too voluminous or too unstructured to be managed and analyzed through traditional means” (Davenport et al., 2012). This includes traditional structured data like financial, customer surveys, social media, website analytics data discussed in the survey, but also unstructured data such as geographic, sensor data, call centre, biological, and search data. Often, the advantage comes from analyzing different types of data together.

Firms that capitalize on big data “pay attention to data flows; rely on data scientists and product and process developers rather than data analysts; and are moving analytics away from the IT function and into core business, operational and production functions” (Davenport et al., 2012). This is because “data-driven decisions are better decisions” and “using big data enables managers to decide on the basis of evidence rather than intuition” (McAfee & Brynjolfsson, 2012).

Those firms who are leveraging structured and unstructured data to make decisions have been shown to perform better than those who focus solely on structured traditional data. “The more companies characterized themselves as data-drive, the better they performed on objective measures of nancial and operational results” with 5% increases in productivity and 6% increases in profitability (McAfee & Brynjolfsson, 2012).

The article Making Advanced Analytics Work for You (2012), suggests that performance of firms is positively correlated to their use of advanced analytics and data driven decision making, but extracting insight goes far beyond simply collecting the right kind of data. The authors state that “developing analytics tools that focus on business outcomes and that are relevant and easy to use for everyone from C-suite to front lines” (Barton & Court, 2012) is equally important. But, what is the point of acquiring a large quantity of data if you do not have a way to analyze or make sense of the results?

My research shows that there is a lack of tools and systems in place to analyze results from unstructured data. Because of the lack of analysis, many firms have yet to realize the value that can derive from data science. Firms view big data as a “black hole” as one study participant stated. This research suggests that there is “knowing-doing gap” (Pfeffer & Sutton, 2000) when it comes to big data.  

Firms can read about the potential of adopting the study of big data as part of their decision making, but are overwhelmed and do not know where to begin. One participant stated that all of their data was “dirty, garbled, and mixed between multiple databases.” They went on to say that their business analysts spent “months reviewing and cleaning the data so that they could try to extract some value” from it. Interviews with data scientists suggest that this is a common obstacle that firms present, but it is not as big an issue as they may be lead to believe. Data scientists are comfortable extracting value from chaotic data.

The academic literature suggests that firms should start small. McAfee & Brynjolfsson (2012) suggest that firms should select a department as a test case and that the departmental leader should have a “quant friendly leader” in place, with proper support from a team of data scientists, engineers, and visualization experts. Firms must take an agile approach to their test case by identifying “five business opportunities based on big data, each of which could be prototyped within five weeks by a team of no more than five people” (McAfee & Brynjolfsson, 2012). This iterative approach to big data is an accessible and low-risk path.

Barton & Court (2012) speak of the three strengths that need to be developed before firms can realize the full potential of advanced analytics. They state that using internal and external data requires upgraded infrastructure for easy merging of data. With the proliferation of enterprise cloud storage, this is within reach of all firms and has resulted in significant cost savings over traditional server warehouses. The authors suggest that models must be developed that optimize performance and predict future opportunities. Finally, the authors indicate that organizational transformation can be realized by creating simple data visualization tools for front line staff and reinforcing the benefits that come from leveraging those tools (Barton & Court, 2012).

It is important for firms to focus on data collection and analysis, but it is also very daunting when it need not be. Rather than undertaking massive overhauls of their businesses, executives may consider concentrating on targeted efforts to source data, build models, and [slowly] transform the organizational culture” (Barton & Court, 2012). This incremental approach is suggested by experts and must follow a traditional change management process to be successful. Firms must unlock small insights and solve small problems through data science before they are capable of acting on the big insights that are possible through the adoption of data-driven decision making.

___

Jonny Stevens works in a data consultancy and is using his research on advanced analytics and decision making to help his clients implement data science solutions that predict, optimize, and visualize.