Understanding Hadoop in 8 Minutes

Coming from a strong business background, I found many other business-minded peers who struggled to make sense of the technical world of data. Here I will try to bridge the gap and breakdown Hadoop to the best of my knowledge to explain the many possibilities it can bring to a business.

What is Hadoop?

Hadoop is a set of technologies that are used to store and process huge amounts of data.

It is important to note that Hadoop is not a replacement for the traditional relational database management systems (RDBMS). Rather, Hadoop can be seen as a supplemental data system that can handle extremely big data sets and Big Data operations that an RDBMS may have difficulty delivering on. Unlike RDBMS, we cannot call Hadoop a database. Hadoop is more of a distributed file system that can store and process a huge volume of data sets across a cluster of computers. Both RDBMS and Hadoop system have similar functions such as collecting, storing, processing, retrieving, extracting and manipulating data. However, both Hadoop and RDBMS process data differently. An RDBMS focuses on structured data whereas Hadoop specializes in dealing with semi-structured, and unstructured data. One of the major reasons that Hadoop should not be considered as a replacement for an RDBMS is because an RDBMS is transactional in nature and is used for applications where data validation such as dates, dollar amounts, etc. is needed for a single record or a small number of records. Additionally, Hadoop is used more for batch data processing as it takes much time and computes resources to crunch through large volumes of data. Basically, Hadoop is an extension when combined with an RDBMS it creates a powerful, faster and more valuable data solution.

Should I use Hadoop?

Some might ask, “So when does Hadoop come into play and should I use it?" [click to continue reading...]