Market Mania of Hadoop & Big Data

Today almost every one of us is active on Social Media. Every day we post our thoughts, ideas, share images, videos, and demand by the user is mounting day by day. Have we ever thought what will be the scenario if all these data got processed manually or with traditional computing algorithms!

It’s truly quoted – Need is the mother of all inventions. And no doubt the factual craving to give uninterrupted & ultimate experience to the End User has given birth to Big Data.

So what is Big Data?

Big Data is a term that describes the large volume of data – both structured and unstructured – that inundates a business on day-to-day basis.

Who generates such Big Data and why?

It’s none other than us who generate Big Data. And reason of generating is very simple – getting connected and sharing information. Let’s have a look on the below Market fragmentation of Big Data for better understanding:

Below real time data is enough to understand the size of Big Data:

• Every day we create 2.5 quintillion bytes of data
• New York Stock Exchange generates 1 TB/Day
• Google processes 700 PB / Month
• Facebook hosts 10 Billion Photos taking 1 PB of storage

Now business success of these types of organizations is highly depended on –

• The ability to store and analyze these kind of bulk data
• The ability to Storage, Searching, Sharing, Transfer, Analysis, Presentation

To fulfill the above challenges, organizations normally take the help of enterprise servers.

But dealing with huge amounts of scalable data, it is a hectic task to process such data through a single database bottleneck.

Many thanks to Google, who solved this problem by implementing an algorithm called MapReduce. It divides the task into small parts and assigns them for many computers, and collects the results from them which when integrated, form the result dataset.

Then what does Hadoop do in Big Data?

Above concept are the root grounds for development of a revolutionary software framework – Hadoop. The genesis of Hadoop came from the Google File System Paper that was published in October 2003. Using the solution provided by Google, Doug Cutting (who was working at Yahoo! at that time) and his team developed an Open Source Project called HADOOP (named after his son’s toy elephant). It runs applications using the MapReduce algorithm, where the data is processed in parallel with others.

Hadoop is an Apache open source framework written in java that allows distributed processing of large datasets across clusters of computers using simple programming models. The Hadoop framework application works in an environment that provides distributed storage and computation across clusters of computers. Hadoop is designed to scale up from single server to thousands of machines, each offering local computation and storage.

At its core, Hadoop has two major layers namely:

(a) Processing/Computation layer (MapReduce), and

(b) Storage layer (Hadoop Distributed File System).?

No doubt there is a huge market opportunities for IT services in Big Data. The McKinsey Global Institute estimates that data volume is growing 40% per year, and will grow 44x between 2009 and 2020. Around $15 billion opportunity is available on software firms only specializing in data management and analytics. This industry on its own is worth more than $100 billion and growing at almost 10% a year which is roughly twice as fast as the software business as a whole. This is the reason why there is a Market Mania for Big Data.

Thanks for giving your valuable time for this article. Hope it was a good read!

** Photos in this article are from web communities, who allow their work to be embraced by everyone for FREE. Thanks to all such people out there !

Best Regards

Ravi Singh

Client Partner

Sasken Technologies Ltd.

www.Sasken.com

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Founder & Customer Service Evangelist

SalesMatinee

www.SalesMatinee.com

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~