Building Your Own PubSub System - Your Very Own Personal Zoom App

Another article in the series of Queue based Systems, today we will see what a PubSub is and when and where to use them. Last time, we saw how a Queue Like Structure is used by Apache Kafka and where is Kafka used.

What's a PubSub?

Going over the name, PubSub refers to a concept of Publisher and Subscriber. If you talk about english, there isn't much difference between Publisher/Subscriber and a Producer/Consumer words. But in the world on computer science, there is a HUGE difference between the two. Le's see how -

Don't forget to Like ??, Comment ?? and Share ?? with your LinkedIn network ??

Understanding PubSub

Let's try to understand PubSub from a use case perspective and try to build a Zoom/GMeet Broadcast application.

We have a Townhall (or All Hands) in our company, and our CEO wants to address all our employees via a Zoom/GMeet Broadcast. All the users are listeners and the CEO is the person allowed to talk / show video.

Various Components of a PubSub System

Over here, our CEO becomes the Publisher which means, they are Publishing events into the system - which includes video (image frames) and audio.

Every user attending this broadcast is a Subscriber, who has joined the meeting link (or channel) and subscribes to each incoming event -- video (image frame) as well as audio.

There is the Zoom/GMeet server (backend/cloud hosted) which routes and forwards each single event (or message) from the producer to each active subscriber at that instant.

Each message sent by each Producer is received by each Subscriber of the same topic.

Difference between PubSub and Queues/Kafka

The main difference is that PubSub does not store any data within the system.

It is just a rerouting mechanism which routes incoming messages to each subscriber, connected at that instant.

If you as a Subscriber are joining late, you cannot retrieve older messages.

Under The Hood of a PubSub

Now, you would think that a PubSub would be a very complex thing to build..! But no..! It's actually just a For Loop and a socket listener..! xD

connected_subscribers = [

    <user1_socket_obj>,
    <user2_socket_obj>,
    <user3_socket_obj>,
    ...

]

while True:
    message = listen_on_producer()

    if message:

        for subscriber in connected_subscribers:
            send_message(subscriber)

This is the high level overview of building a very simple toy PubSub. Obviously, here we assume we have only 1 channel (namespace or topic) whereas a production PubSub can have multiple channels (one for each Zoom broadcast). Also, it will have socket connections, threading and much more complex stuff to make it prod ready..!

A very simple, production ready PubSub was created by me long back, which you can use to simulate Whatsapp/Messenger chat system and chat with your friends on a local network..! ??

If you like this article, don't forget to Like ??, Comment ?? and Share ?? with your LinkedIn network ??