The Great Wall of China and Apache Kafka

During Community Over Code Asia 2023, I had the opportunity to do a quick side trip to an intact section of the Great Wall of China near Beijing, at Mutianyu. I was struck by how long the wall is (it stretches into the distance, and you can see watchtowers on other sections of the wall on distant mountain ridges), and also that it seemed to be designed to be symmetrical - there is no obvious inside or outside of the wall, it is actually a castle, and could therefore be defended against enemies from within and without the Chinese Kingdom. The Wikipedia article mentions this feature, and says that this is rare on other parts of the wall. Reading between the lines it seems that this Ming dynasty section of the wall may therefore have been designed to protect against both external and internal threats.

So, what does the Great Wall of China have to do with Apache Kafka? Well, I thought the symmetrical nature of the defences was interesting, and Kafka also supports message producers and consumers (for messages in and out). The Great Wall also functioned as a communications system (20,000 km long!) using beacons, fires, smoke and horses.

And oddly enough, Franz Kafka wrote an essay on the building of the Great Wall of China, available here (It's a work of fiction - I don't think Franz ever visited the Wall).

The final thing I appreciated about visiting the Great Wall was that I could enjoy it (although it's pretty hard work walking even a small section of the wall as it follows the contours of the mountain ridges exactly - very up and down) - without having to build it myself. It was also built to last - Kafka is often used as the foundation for bigger/long-term enterprise applications so it's also a good choice if you want reliable middleware with lots of community support. If you want to find out more about Apache Kafka and Kafka Use Cases check out my Kafka for Developers blog here. You don't need to write Kafka or even run it yourself, which makes it very easy to use for a variety of large-scale messaging and event use cases.