Taking advantage of Big Data in the mining industry. It just got a reboot.

While simple in its brevity, ‘Big Data’ is a nebulous term. It refers to data so voluminous and complex that traditional data processes are inadequate to deal with it. It is also not necessarily always just about data size, but the speed at which data is processed, and the type of information. So Big Data is not a one size fits all term. It is vague jargon which means different things to different organisations.  A seemingly nefarious concept by nature, Big Data is an integral fragment of understanding the digital age in mining by providing insight from information sources that are diverse, complex, and of immense scale.

Big Data enables timely decisions that are customarily difficult to execute without intensive workloads. It also amplifies organisational value, which among other things includes saving lives, determining patterns in abstract data (such as imagery), and exploring complex relationships between unrelated data sets that would not otherwise be detected. In other words, Big Data has the capacity to offer substantial benefits we cannot normally see. 

Big Data consists of 3 different types of data structures.  1) Structured data: any data stored in a specific data scheme and typically recognised as tabulated such as SQL, Oracle, Excel, and Access. 2) Semi-structured Data: which does not conform to a regular data model, but through text based tags and markers enforces hierarchies of records and fields within the data. 3) Unstructured Data: data without a pre-defined data model such as reports, word documents, images, and video. These classifications are fundamental in understanding Big Data. 

A Data Lake supports an organisations ability to harness diverse types of data stored in a multitude of data structures. Data Lakes are crucial for understanding Big Data because they represent a holding place for data in its native format. Another way of thinking about it is that instead of a lake being fed by multiple water sources such as streams, waterfalls, and tributaries, a lake of data is an accumulated body of data made up of structured, semi-structured, and unstructured data sources. 

The above streamlined example demonstrates how Big Data can be managed in the mining industry across the value chain. Company information can be so extensive, over-powering, and complex that it becomes difficult to comprehend. Big Data analytics such as predictive modelling, pattern recognition and business intelligence can be applied to information stored within a Data Lake. And when managed correctly, organisations can focus on more relevant and flexible data sets.

A Data Lake gives rise to a system with large storage capacity of multiple formats, and timely processing speeds – the indispensable pre-requisites for Big Data.

‘Big Data’ is now ubiquitous terminology when discussing data volumes that teeter on the border of cutting edge technology. However data has been pushing technology to its limits for decades, since the late 70’s. ‘Big Data’ was in fact first coined in 1998. So why is the mining industry only now getting involved with Big Data? Primarily because enterprise systems that encompass the necessary requirements for Big Data such as cloud storage, IoT, machine learning, and data lakes, have only been available in the last 2-3 years through providers such as Microsoft, Google, and IBM. 

So really, Big Data just got a reboot. 

And timely enough to be instrumental in aiding the transition to Industry 4.0.

Being proactive rather than reactive is a catch cry for Big Data. Take a processing plant for example. Collecting and reading large amounts of sensor data is becoming more synonymous with plant and machine operations every year on mine sites. Instant data analysis can aid in making critical autonomous decisions at any moment, such as plant shut downs to protect catastrophic mechanical failures, and more importantly to protect employees from harm.   These are decisions that can be made instantly or in the short term. While using data for short-term decision making is becoming prevalent across the industry, what is less prevalent is recording and analysing vast amounts of data, historical and real-time, to predict an incident before it occurs rather than coping with it as it happens.

But how can an organisation effectively use this cauldron of raw information with no immediate practical use? Not only is there too much of it to make any sense of meaningfully, knowing what to look for is challenging. This is where the interaction between Big Data and Machine Learning entwine - making sense of huge stores of information very quickly, and enabling companies to produce outcomes of valued significance. 

To explain Machine Learning simply, there are traditionally 2 main types of technique used in Machine Learning : 1) Unsupervised Learning, and 2) Supervised Learning. Understanding the difference between the two goes a long way to comprehend the basics of using Big Data for machine learning. 

Unsupervised learning typically is the raw and unmanaged data which organisations use with Machine Leaning algorithms to determine patterns that will give meaning to data. Because organisations often do not initially know the data relationships they are looking for, algorithms are used to make sense of the data and find relationships through techniques such as clustering.

Supervised learning uses data inputs and defined classifications from which algorithms can map answers based on historically defined data.  A simple example of Supervised learning may be that a relationship is uncovered though analysing historical sensor data between 3 completely different mechanisms in a processing plant. They have resulted historically in the failure of a fourth mechanism further along the plant cycle. A faulty mechanism can result in catastrophic plant failure. Through extensive processing of historical data, it is determined that when the three mechanisms behave inconsistently at the same time, a downtime classification is determined and failure of the fourth mechanism can be predicted and managed prior to a possible incident. It is a proactive measure to predict failure rather than reactive.  

Without doubt, Big Data is relevant across the mining value chain. From exploration prospectivity mapping to managing autonomous machinery and safety. It offers the capacity to harness previously superfluous data, and make decisions that will improve safety, costs, and productivity over time.

The term Big Data will not be disappearing any time soon. By its very nature, Big Data infers massive volumes of data processed and analysed at speed beyond the realm of traditional data practices. Big Data platforms will always be searching for new technologies and innovations that are continuously improving. 

In other words, it’s just getting started.