Azure EventHubs - What to consider ?

Hello Everyone - Before we start stalling the product let us see the good points first.

Good highlights:

• Finally fully PaaS solution in place.

Setting it up from Azure Portal is fairly simple and easy. You can create an auto inflate cluster if you like and can also go with geo disaster recovery.

Once the cluster is created you can access it via FQDN which is pretty impressive and also supports terraform / ARM templates.

• Security is plus - SSL enabled and fully integrated AD

The administrator(s) can generate access policies and distribute the keys to the correct persons in the organization. If you think a key is compromised you can easily delete it and regenerate a new one.

So all those who need to argue with architects and cyber security, this is definitely a big plus.

• Comes with Schema Registry now

An Event Hubs namespace now can host schema groups alongside event hubs (or Kafka topics). It hosts a schema registry and can have multiple schema groups. In spite of being hosted in Azure Event Hubs, the schema registry can be used universally with all Azure messaging services and any other message or events broker. Each of these schema groups is a separately securable repository for a set of schemas. Groups can be aligned with a particular application or an organizational unit.

• Kafka Enabled

Basically it means that your consumer and producer for Kafka now also work for Event Hubs. Keep in mind that you have to explicitly enable this feature during cluster creation. You can not do this afterwards.

• Capture Mode

With this feature you do not have worry about service that is reading your data and writing to storage , you can choose where to offload data.

Now comes the cons:

I don't really like to call out as cons but instead as limitations or what to know before we use it

With more and more events been written down every second and with world moving all towards data I cannot imagine anything less than a super bright future for solutions that are focused on event captures and messaging and hence lies the growth for this aspect.

• Quotas

There are quotas in place everywhere but with this it is fairly low. You will run into something like messages cannot be more than 256KB , number of event hubs is limited to 10 per namespace and the list goes on and on, more info can be found here.

The extension though can be done but there is always loss of few days in getting it done.

• Retention Period

The retention per EventHub is not more than 7 days even in standard tier which is not pretty impressive. This ideally I believe should be left on customer with a limit of not more than 30 days.

• Fixed port

Event Hubs namespace listens only on port 9093 for the client libraries and port 443 if you produce messages through an HTTP client. While this has some advantages for Microsoft it also gives some disadvantages to customers. Other technologies let you change the port, why hardcode this port ?

• Kafka Connect and Kafka Streams

I think most of enterprises leverage these frameworks a lot and should be supported but it is yet in there.

My Perspective:

Well I do find this article interesting where it was attempted to compare with Kafka by Microsoft. There are some really interesting features which Azure Event-hub brings to the table and are easier to do than Kafka but maturity will come with time.

If you really relying on heavy duty streaming and have super complex operations in place then go with Apache Kafka but consider setting it up is more complex. With most of solutions moving to cloud I do see this as promising product and with recently dedicated Event-hubs in place it is gaining its own piece.

Disclaimer

I wrote this article from a personal view while trying to be objective as possible. These express my personal experience and these thoughts might not necessary be those of my firm.