Building a Smarter Home with Nifi and Spark

I submitted an abstract for the hadoop world summit.  Check it out and vote for it here

Join us as we discuss what life is like when your front door knows you are going to be late for work, the wifi router tells you guests have arrived, and Orwell's dream is alive and well. Everything connects to the internet, everything generates logs, and everything has a different interface. How do we leverage multiple disparate sources of information to get a more complete picture of our lives? Modern tools allow us to quickly integrate event logs from different sources to feed analytic platforms. We will go through the challenges and surprising patterns that emerged the perspective of our devices. Apache Nifi enables disparate sources of information(sensor, wifi log, Rasberry Pi) to be easily acquired, analyzed in stream, and sent offsite. The integrated platform management that Nifi provides turns this task from a multi-application ecosystem to a single interface that serves as a nexus of incoming data. Utilizing Site-2-Site capabilities we are able to transfer data from the edge to the cluster for exploration in Spark. This allows us to focus purely on the data and not the technicalities normally present in complex event processing.