IoT Big Data Analytics

The development of big data and the Internet of things (IoT) is rapidly accelerating and affecting all areas of technologies and businesses by increasing the benefits for organizations and individuals. The growth of data produced via IoT has played a major role on the big data landscape. The more opportunities are presented by the capability to analyze and utilize huge amounts of IoT data, including applications in Smart Lighting, Smart Home, Smart cities, Smart transport and Smart grid systems, energy smart meters......

The widespread popularity of IoT has made big data analytics challenging because of the processing and collection of data through different sensors in the IoT environment. IoT big data analytics can be defined as the steps in which a variety of IoT data are examined to reveal trends, unseen patterns, hidden correlations, and new information. Companies and individuals can benefit from analyzing large amounts of data and managing huge amounts of information that can affect businesses. Therefore, IoT big data analytics aims to assist business associations and other organizations to achieve improved understanding of data, and thus, make efficient and well-informed decisions. Big data analytics enables data miners and scientists to analyze huge amounts of unstructured data that can be harnessed using traditional tools. Moreover,big data analytics aims to immediately extract knowledgeable information using data mining techniques that help in making predictions, identifying recent trends, finding hidden information, and making decisions.

Techniques in data mining are widely deployed for both problem-specific methods and generalized data analytics. Accordingly, statistical and machine learning methods are utilized. IoT data are different from normal big data collected via systems in terms of characteristics because of the various sensors and objects involved during data collection, which include heterogeneity, noise, variety, and rapid growth. Statistics show that the number of sensors will be increased by 1 trillion in 2030. This increase will affect the growth of big data. Introducing data analytics and IoT into big data requires huge resources, and IoT has the capability to offer an excellent solution. Appropriate resources and intensive applications of the platforms are provided by IoT services for effective communication among various deployed applications. Such process is suitable for fulfilling the requirements of IoT applications, and can reduce some challenges in the future of big data analytics. This technological amalgamation increases the possibility of implementing IoT toward a better direction. Moreover, implementing IoT and big data integration solutions can help address issues on storage, processing, data analytics, and visualization tools. It can also assist in improving collaboration and communication among various objects in a smart city. Application areas, such as smart ecological environments, smart traffic, smart grids, intelligent buildings, and logistic intelligent management, can benefit from the aforementioned arrangement.

Relationship Between IoT and Big Data Analytics

Big data analytics is rapidly emerging as a key IoT initiative to improve decision making. One of the most prominent features of IoT is its analysis of information about “connected things.” Big data analytics in IoT requires processing a large amount of data on the fly and storing the data in various storage technologies. Given that much of the unstructured data are gathered directly from web-enabled “things,” big data implementations will necessitate performing lightning-fast analytics with large queries to allow organizations to gain rapid insights, make quick decisions, and interact with people and other devices. The interconnection of sensing and actuating devices provide the capability to share information across platforms through a unified architecture and develop a common operating picture for enabling innovative applications.

The need to adopt big data in IoT applications is compelling. These two technologies have already been recognized in the fields of IT and business. Although, the development of big data is already lagging, these technologies are inter-dependent and should be jointly developed. In general, the deployment of IoT increases the amount of data in quantity and category; hence, offering the opportunity for the application and development of big data analytics. Moreover, the application of big data technologies in IoT accelerates the research advances and business models of IoT. The relationship between IoT and big data can be divided into three steps to enable the management of IoT data. The first step comprises managing IoT data sources, where connected sensors devices use applications to interact with one another. For example, the interaction of devices such as CCTV cameras, smart traffic lights, and smart home devices, generates large amounts of data sources with different formats. This data can be stored in low cost commodity storage on the cloud. In the second step, the generated data are called “big data,” which are based on their volume, velocity, and variety. These huge amounts of data are stored in big data files in shared distributed fault-tolerant databases. The last step applies analytics tools such as MapReduce, Spark, Splunk, and Skytree that can analyze the stored big IoT data sets. The four levels of analytics start from training data, then move on to analytics tools, queries, and reports.

Contents from https://ieeexplore.ieee.org/abstract/document/7888916