MQTT: The protocol powering IoT

You’ve heard of HTTP (the protocol behind websites), but MQTT is HTTP’s minimalist cousin, built for machines, not humans.

What is MQTT?

MQTT is a messaging protocol designed for low-power devices (like sensors) to communicate over unreliable networks. Think of it as a walkie-talkie system:

• Devices “publish” messages (transmit) to specific channels.
• Devices “subscribe” to channels (listen) to receive messages.
• A central broker (like a radio tower) routes messages between devices.

Example:

• A temperature sensor publishes to the channel home/kitchen/temp.
• Your phone app subscribes to home/kitchen/temp to get updates.

Unlike HTTP, MQTT is event-driven, meaning data is pushed in real-time—no need for constant polling!

Why MQTT Dominates IoT

Lightweight

• HTTP sends bulky headers (like addressing an envelope with a novel).
• MQTT sends tiny packets (like a sticky note).

Fast

• Perfect for battery-powered sensors that send data every few seconds.

Flexible

• MQTT works over Wi-Fi, 4G, 5G, LoRa, and even satellite connections—ensuring messages are delivered even on unstable networks.

Core MQTT Concepts (Simplified)

a) Broker: The Post Office

The MQTT broker is the central hub that receives and distributes messages. Popular brokers:

• Cloud-based: AWS IoT Core, HiveMQ, EMQX
• Self-hosted: Mosquitto (Windows/Linux/Raspberry Pi)

b) Topics: Mailbox Labels

• Topics are hierarchical labels (like folders):

home/kitchen/temp
home/livingroom/humidity        

• Subscribers use wildcards:

home/# → Subscribe to all topics under home/
+/temp → Subscribe to any room’s temperature        

c) QoS: Delivery Guarantees

• QoS 0: “Fire and forget” (no confirmation).
• QoS 1: “At least once” (guaranteed delivery).
• QoS 2: “Exactly once” (no duplicates).

d) Retained Messages: The Last Update

• The broker saves the last message on a topic. New subscribers get it instantly.
• Without retention, your dashboard remains blank until the sensor publishes the next update—which could be hours later.

e) Last Will & Testament (LWT): The “Goodbye” Alert

• If a device disconnects abruptly, the broker sends a pre-set message (e.g., “Device offline!”).
• No more guessing if a sensor crashed or just went silent!

f) Security

MQTT supports authentication & encryption:

1. Username/Password authentication (like logging into an account).
2. TLS Encryption (SSL) for secure communication.
3. Access Control Lists (ACLs) to restrict topic access.

• Example: Prevent unauthorized devices from subscribing to critical topics like /security/camera-feed.

MQTT in IOT and embedded systems

MQTT’s lightweight design makes it perfect for microcontrollers (MCUs) like ESP32, Raspberry Pi, and STM32:

1. Eclipse Paho: The Most Popular MQTT Library

The Eclipse Paho library provides MQTT client implementations for almost every language and platform. For embedded systems:

• Paho Embedded C/C++: Designed for resource-constrained devices (supports QoS 0/1/2, TLS).
• Paho Arduino: Simplified wrapper for ESP32/ESP8266.

2. MQTT on ESP32

ESP32’s built-in WiFi and low cost make it a favorite for IoT projects.

• PubSubClient: (simplified Paho wrapper for Arduino).
• WiFiClientSecure for TLS.

3. MQTT on STM32

1. Choose a Network Interface:

• Ethernet
• WiFi
• Cellular
• LoRa

2. Select an MQTT Library

• Paho Embedded C
• NanoMQ
• AWS IoT SDK

How I Applied MQTT (Project Walkthrough)

Step 1: Simulated Sensors (Python)

I wrote Python scripts to act as temperature/pressure sensors. They publish fake random data to topics: /sensor/temperature and /sensor/pressure, and status_publisher to publish the status of the device (the fake device I guess!)

temp_publisher.py:

import paho.mqtt.client as mqtt
import time
from dotenv import load_dotenv
import os
import random
load_dotenv()

LOCAL_BROKER = os.getenv("LOCAL_BROKER")
LOCAL_USERNAME = os.getenv("LOCAL_USERNAME")
LOCAL_PASSWORD = os.getenv("LOCAL_PASSWORD")
LOCAL_PORT = 1883

TOPIC = "/sensor/temperature"

def publish_temperature():
    client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2)

    client.username_pw_set(LOCAL_USERNAME, LOCAL_PASSWORD)
    client.connect(LOCAL_BROKER, LOCAL_PORT)

    while True:
        temp = round(random.uniform(20, 30), 2)
        print(f"Publishing Temperature: {temp}°C")
        client.publish(TOPIC, str(temp), qos=0, retain=True)
        time.sleep(3)

publish_temperature()
        

pressure_publisher.py:

import paho.mqtt.client as mqtt
import time
from dotenv import load_dotenv
import os
import random
load_dotenv()


LOCAL_BROKER = os.getenv("LOCAL_BROKER")
LOCAL_USERNAME = os.getenv("LOCAL_USERNAME")
LOCAL_PASSWORD = os.getenv("LOCAL_PASSWORD")
LOCAL_PORT = 1883

TOPIC = "/sensor/pressure"


def publish_pressure():
    client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2)
    client.username_pw_set(LOCAL_USERNAME, LOCAL_PASSWORD)
    client.connect(LOCAL_BROKER, LOCAL_PORT, keepalive=300)

    while True:
        pressure = round(random.uniform(990, 1020), 2)
        print(f"Publishing Pressure: {pressure} hPa")
        client.publish(TOPIC, str(pressure), qos=0, retain=True)
        time.sleep(0.5)

publish_pressure()
        

status_publisher.py

import paho.mqtt.client as mqtt
import time
from dotenv import load_dotenv
import os

load_dotenv()

LOCAL_BROKER = os.getenv("LOCAL_BROKER")
LOCAL_USERNAME = os.getenv("LOCAL_USERNAME")
LOCAL_PASSWORD = os.getenv("LOCAL_PASSWORD")
LOCAL_PORT = 1883

TOPIC = "/device/status"

client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2)
client.username_pw_set(LOCAL_USERNAME, LOCAL_PASSWORD)
client.will_set(TOPIC, "offline", qos=1, retain=True)
client.connect(LOCAL_BROKER, LOCAL_PORT)

client.publish(TOPIC, "online", qos=0, retain=True)

print("Device is online. Press CTRL+C to stop...")
try:
    while True:
        client.publish(TOPIC, "online", qos=0, retain=True)  # Keep updating status
        time.sleep(30)  # Adjust interval as needed
except KeyboardInterrupt:
    print("Disconnecting...")
    client.disconnect()

except KeyboardInterrupt:
    print("Disconnecting...")
    client.disconnect()
        

Step 2: Local Broker (Mosquitto)

I installed Mosquitto, a free broker, on my laptop. It’s like setting up a local post office for my sensors.

Step 3: Local subscriber

I created a python script to subscribe to the topics on which data is being published

from pymongo import MongoClient
import paho.mqtt.client as mqtt
import time
from dotenv import load_dotenv
import os

load_dotenv()
CLIENT_ID = "mqtt_subscriber_1"


LOCAL_BROKER = os.getenv("LOCAL_BROKER")
LOCAL_USERNAME = os.getenv("LOCAL_USERNAME")
LOCAL_PASSWORD = os.getenv("LOCAL_PASSWORD")
LOCAL_PORT = 1883
TOPIC = "#"

def on_connect(client, userdata, flags, rc, properties):
    if rc == 0:
        print("Connected to MQTT broker")
        client.subscribe(TOPIC, qos=0)  # Resubscribe if necessary
    else:
        print(f"Connection failed with code {rc}")


def on_message(client, userdata, msg):
    data = {"topic": msg.topic, "value": msg.payload.decode()}
    print(f"?? Received: {data}")


def on_disconnect(client, userdata, rc):
    print(f"Disconnected with code {rc}, reconnecting...")
    client.reconnect()

client = mqtt.Client(client_id=CLIENT_ID, clean_session=False, callback_api_version=mqtt.CallbackAPIVersion.VERSION2)
client.username_pw_set(LOCAL_USERNAME, LOCAL_PASSWORD)
client.connect(LOCAL_BROKER, LOCAL_PORT, keepalive=300)  # Set to 5 minutes
client.on_connect = on_connect

client.on_message = on_message
client.on_disconnect = on_disconnect


client.loop_forever()
        

Step 4: Bridging connection

I connected my local broker to a cloud broker (like forwarding mail to a central hub).

I used a Linux server (from digital ocean) as the cloud broker, so now all data being published to the topics through the local broker, is also being published to topics on a cloud broker (same topics names).

Step 5: Save to Database (MongoDB)

A cloud-based Python script subscribes to sensors/# and saves data to MongoDB.

Step 6: Visualization using Node-RED

To visualize data, I used Node-RED—a drag-and-drop tool—to subscribe to MQTT topics and display gauges/charts. But this was just icing on the cake. The real hero was MQTT

Final Thoughts

MQTT is the ultimate protocol for IoT, embedded systems, and real-time communication.

If you’re working with microcontrollers, smart devices, or industrial automation, you should definitely consider MQTT for fast, reliable, and lightweight messaging.

I'm looking forward to learning about more protocols used with IOT!

P.S. You can simulate ESP32 running MQTT clients on wokwi.