Drowning in Data: IoT’s Impact on Big Data

As billions of devices, things, and processes become connected; they will generate a massive amount of data. IDC predicts that 50 trillion gigabytes of data will be generated by 2020 to support the 20 to 30 billion devices that are commonly predicted to be connected as part of the Internet of Things (IoT) by 2020.

The data generated from inter-connected “things” create value when that data is used to reduce inefficiencies and optimize productivity. It can also help to create new products and services and also reduce fraud, even save lives.

However, with the increase in the number of connected “things” and the data it generates, comes increased responsibilities for companies, service providers, and others to address challenges such as privacy, storage, security and ownership of the data. But, before we get ahead of ourselves, let’s understand how and why so much data is being generated from IoT.  

A Fleet Example

I discussed the connected car in my earlier post.  Another related and interesting example is the transportation industry as its one of the first to adopt IoT in a widespread manner. The fleet industry’s trucks, busses, and vans generate quite a bit more data than its counterpart just a few years earlier. Fleet telematics systems collect data throughout the truck including via the tires, the engine, the cameras, etc. Data from the engine is sent as often as every 10 seconds, in FleetUp’s case, to determine, for example, if the tire pressure is at the right level, if the engine is overheating, if the driver is braking too hard or accelerating in an unsafe manner. If the truck is involved in a crash, video can be sent to the company headquarters 10 seconds before and 10 seconds after the crash, for example. This information provides valuable clues about what took place, when and perhaps, what caused the crash.  Such information helps the fleet owner determine if the accident should be disputed.

In addition to tracking the number of hours a truck has been operating, its gas mileage, whether the engine is idling and for how long, fleet telematics also generates and/or consumes data to assist with vehicle routing,  dispatching, vehicle navigation including providing information on traffic congestion. Whew! That’s a lot of data.

Characteristics of IoT Data

While we can agree that much data is generated from IoT, not all data is important nor needs to be treated the same. There are four primary characteristics that determine the significance of the data generated from IoT and an appropriate action.

• Volume – The volume of data that is generated per event and the amount that requires processing.
• Situation – The background or context within which the data is generated and managed.
• Structure – The different types of data generated and its structure.
• Speed – The rate at which the data is delivered and processed.

Buildings with security systems, for example, generate a lot of data (volume) including video data (structure). Depending on what is taking place (situation), the data may or may not be significant. If a theft is taking place, the data significance goes up hence the need to process and react to the data very quickly (speed) by calling the police.

Challenges of Managing IoT Big Data

As with any new technologies, there are challenges and IoT is no exception. Here are some of the issues facing companies as they wrestle with how to make sense of all the data generated by an ever-connected world:

• Privacy – Data privacy is different from security so deserves mention by itself. When designing an IoT solution that is capable of capturing an endless amount of information, be it in data or video format, be sure that it doesn’t violate an individual’s rights to privacy. While many of the laws around privacy are already defined, IoT is new and evolving so don’t assume that if there isn’t a law or regulation surrounding privacy today that there won’t be one tomorrow.
• Storage – Conventional data storage technologies and approaches are simply not enough for most deployments, except the smallest. The data storage solution has to be capable of scaling and a plan to back-up the data also needs to be in place.
• Security – Each use case is unique and needs to be approached individually to ensure that access control, system availability, data integrity and audit procedures are designed into the IoT solution from the beginning. While it’s possible to address the above issues after the solution is built, it will be much harder so don’t overlook data security when starting.
• Data Ownership – The question of who owns the data can be sticky. Is the data owner the company that owns the building where the camera is placed or could it be the solution provider that is providing the video surveillance solution? What about the individuals that are being filmed? Who monetizes the data that is captured from a certain building and what if a solution provider puts it together with data from other buildings and in other cities along with that provider’s proprietary data? I don’t have an answer, but these are issues that need to be considered when designing and deploying an IoT solution.

The Future of IoT: Big Data

Given a large amount of data that is being generated, it’s crucial to identify what is important and what is just noise. Hence, the need for data analytics will be great such that it will drive demand for jobs for individuals that specialize in data science.  Currently, the issue of how to get data off the devices and into back-end systems is increasingly being resolved through edge computing. However, the challenge of who gets to monetize the data (The solution provider? The company? The network connectivity provider?) is not yet resolved. Also, the industry needs to figure out who plays the role of the data broker and who manages the data repository. The answer might involve multiple roles or one. As the increasing demand for IoT analytics providers and data scientists to help make sense of the data is met, organizations will start to utilize the data for predictive purposes, so it drives better, more efficient organizational decisions rather than as a passive activity of analyzing the data after the fact.

If you want to see an example of how the fleet industry is using big data to manage fuel efficiency, truck routing, navigation and engine performance, go to FleetUp’s site.