BIG DATA

There is no place where big data does not exist! The curiosity about what is big data has been volatile in the past few years. Let me tell you some facts ! Forbes reports that every minute , users watch 4.15 million you tube videos, sends 98000+ tweets, 11 million instant message, post 46750 photos on instagram and 293000 statuses posted on facebook !

This huge chunk of data that is produced with such activities of social media, business applications, telecom and various other domains leads to the formation of big data.

EVOLUTION OF BIG DATA

When the last time you guys remember using a floppy or a CD to store your data? we had to go back to the early 21st century. The use of manual paper records, files, floppy and discs have now become obsolete.The reason is the growth of data. people started storing their data in database systems but due to new inventions , technologies, and applications with quick reponse time that becomes insufficient now. This generation of continuous and massive data \can be referred as big data. forbes reported that there are 2.5 quintillion bytes of data created each day with a fast pace.

WHAT IS BIG DATA?

Big data refers to the large amount of data which is pouring from various sources and has different formats. In this , data is so large and complex that none of the traditional data management tools are able to store it and process it efficiently.

Examples of big data :- 1) SOCIAL MEDIA- statistics shows that 500+ TB of new data comes into the social media site facebook everyday. it is generated in terms of photos, videos,message exchanges , putting comments etc.

2) A single jet can collect 10+ TB of data in 30 minutes of flight time. With thousands flights per day, data reaches upto to petabytes(PB).

TYPES OF BIG DATA

It can be found in three forms:-

1.) STRUCTURED DATA - Any data that can be stored, easy to access and to process in the form of fixed format is termed as " structured data". In this , the format is well known in advance by which we can easily derive value out of it. eg. excel sheet, An 'employee' table in a database is also an example of it. Some types of structured data can be machine generated, like data that comes from medical devices (heart rate, blood pressure), manufacturing sensors (rotation per minute, temperature), or web server logs (number of times a page is visited). Structured data can also be human generated - data such as age, zip code, and gender.

2.) UNSTRUCTURED DATA - Any data which has no structure is known as unstructured data. It is difficult to derive value out of it. only 20% of data available to businesses are structured and rest 80% is unstructured. Example: E-Mail, text files, photo sharing sites, flickr, images also because Digital photos are stored in a structured format such as JPG and PNG. but this image data doesn’t tell us what is there in the image. It needs to be processed in order to understand its meaning.

3.) SEMI-STRUCTURED - This data can contain both forms of data.we can see this as a structured in form but it is atually not defined. eg- data represented in an XML file , JSON.

CHARACTERISTICS OF BIG DATA

VOLUME- A particular data will be considered as a big data or not, is dependent on volume of data. If volume is in the gigabytes it might not be a Big Data, but at the terabyte and petabyte level and beyond it may very well be. (1.) Wal-Mart records more than 1 million customer transactions per hour, generating more than 2.5 petabytes of data.(2). And in one survey 17% of companies report managing more than a petabyte of data with an additional 22% reporting hundreds of terabytes.

VARIETY- Variety determines different formats of data i.e both structured and unstructured data such as documents, emails, social media text messages, video, images, audio, graphs, a devices, RFID tags, machine logs, cell phone GPS signals, DNA analysis devices, and more. Through analysis it gives new and valuable insights which is not previously available. Gartner estimates that unstructured data doubles every three months and offers seven million web pages added each day.

VELOCITY- It refers to speed that how fast the data is generated and processed to meet the demands. A second dimension is how long the data will be valuable. When you sign in to eg. Amazon or Netflix use big data to make shows and movie recommendations to their users, E- promotions, healthcare monitoring.

VARIABILITY- It refers to inconsistency which can be shown by data at times, and hamper the process in handling and manage the data effectively. And which keeps on changing constantly. Example – A soda shop may offer 6 different blends of soda, but if you get the same blend of soda every day and it tastes different every day, that is variability. The same is in the case of data, and if it is continuously changing, then it can have an impact on the quality of your data.

VALUE- Value does not differentiate Big Data from not so big data. It is equally true of both big and little data that if we are putting the effort to store and analyze it then it must be perceived to have value. it is clinically relevant data.

VERACITY - In this data is unreliable and come from uncontrolled environments thats why in this data is in doubt. it should be reliable otherwise it would be of no use.

VISUALIZATION- Visualization refers to how you can present your data to the management for decision making purpose. It can be presented in many ways, such as excel files, word docs, graphical charts, etc. Irrespective of the format, the data should be easily readable, understandable, and accessible, and that’s why data visualization is important.

OVERALL GOALS OF BIG DATA ANALYTICS

1.) IN HEALTHCARE - Healthcare sector take advantage of massive amount of data and provide right intervention to the right patient at the right time.

2.) STARBUCKS - Starbucks made use of big data to analyse the preferences of their customers. they analyse the coffee buying habits of customers along with preferred drinks to what time they are usually ordering. so when people visit new starbucks location that store able to identify customers through their smartphones and give barista their preferred order. on this ordering preferences, their app will suggest new products in which they might be interested. this we call it as big data analytics.

TYPES OF BUSINESS ANALYTICS

1.) DESCRIPTIVE - It expalins what happened. it summarises the existing data using business intelligence tool to better understand what is going on or what has hppened.

2.) DIAGNOSTIC - in tis we focus on past performance to check what happened and why.

3.) PREDICTIVE- It predicts future demand or possible outcome using statistical methods.

4.)PRESCRIPTIVE - It is used to recommend one or more course of action to analyse the data.