Why I joined Confluent (and ? it!)

A lot of people (friends and ex-colleagues) asked me why I joined Confluent (https://confluent.io/).

When finishing my studies and when I started working, I have spent about 10 years on techniques and technologies to model and publish data on an Intranet or on the Web at large (RDF, Semantic Web, Linked Data, Open Data, Linked Open Data, graph storage and processing systems, etc.).

Towards the end of that cycle, I realised that distributed and horizontally scalable storage and parallel processing systems were a big deal, that turned out to be a Big Data deal: the shift of the industry in that direction turned out to be massive. I've spent close to 10 years with those tools and technologies (some of those years overlapping with the previous 10 and using big data tools in the context of semantic web and graph storage/processing systems).

Entire industries are now refocusing their attention from big data to fast data and stream processing. Needless to say that Apache Kafka and Confluent are at the heart of this massive industry shift. After having tried many distributed stream processing frameworks, I can honestly say that Kafka's Streams APIs is the simplest, most innovative, and more interesting I've seen so far. This is where I'll be spending my next 10 year cycle: fast data, stream processing and Kafka.

To put some images alongside words and expand on the reasons why I think Kafka's Streams APIs are a beautiful example of simplicity and, at the same time, sophistication, I put together some slides, you can find them here: https://www.slideshare.net/ConfluentInc/apache-kafkaa-distributed-streaming-platform