Spark - Firing up Technology
This article is for nerds.
I was greatly intrigued by Apache Spark, the innovative tool that brings a whole new dimension to the way ETL (Extract Transform Load) is handled in data warehousing. IN an earlier article I had mentioned that Big Data advances will spell a doom for existing high priced ETL tools like Informatica and this is one more nail into their coffin.
Providing high speed ETL and analytics on data as it arrives in packets, Spark makes traditional tools look like outdated teddy bears in an electronic toy shop.
Spark complements Hadoop and combining the two, one can trash all prevailing high priced clutter of ETL tools, databases and hardware that cost companies millions in licensing.
I am fascinated by the way it is transforming data movement and analytics and the tremendous potential it holds. With data arriving every second from sensors like wristbands, fit-bit smart watches and other fancy gadgets, Spark can present statistics, charts and graphs on the data arriving and combine it with data at rest in the warehouse to present even complex patterns. As a use case think of being able to see your EEG, ECG, heart rate, BP, glucose level etc on the click of a button. Or think of being able to detect movement in your home and sound a loud alarm in your home while you are on vacation at the beach. Enough to send shivers up the spine of an intruder! Perfect solution to recent thousands of homes that have been robbed of cash and gold while they were away to a party. Or think of being able to track every movement and location of your child to keep kidnappers at bay.
For those who are more technically inclined I have this one hour video on the free lecture I gave recently to my students. Read and enjoy. The world is changing!!
Apache Spark introduction. Visit any of these links to see the lecture video or download the presentation:
https://www.hadoopexpress.com/download/Spark/
Thanks for sharing Shashi.