Intro to MQTT

MQTT: The basics

There are plenty ways to move data around. Different use cases present different challenges. The Internet of Things (IoT) brings in a unique set of applications that don't fit well with many conventional ways of transporting data. The 'things' components are usually constrained devices in terms of computing performance and often battery power. MQTT is a messaging protocol that has proved to be exceptionally useful in IoT ecosystems. This article will explanation the basics of MQTT, how it works and why it is so useful.

As a messaging protocol, MQTT is lightweight and works in such a way that it allows for reliable communication across multiple devices. It even allows scaling to a large degree, something difficult to achieve with other protocols. This becomes clear when you understand how it works and how it compares to other protocols. It was designed for machine-to-machine (M2M) applications and meant to be robust and reliable. The MQTT protocol works on a publish-subscribe model, which is one of it's core concepts . This stands as an alternative to the traditional client-server model. So instead of communicating to a server the client device publishes and subscribes to a topic that is hosted by an MQTT broker. This model also allows for bi-directional communication between a client device or "thing" and the application. The following figure illustrates this model.

Why MQTT?

So why is MQTT suitable for IoT applications? Well there are issues in using the client-server model mentioned when it comes to sending data to 'things' on a network. These devices be installed anywhere, including remote locations where the environmental conditions and the unreliable networks can cause breakdowns in communication. MQTT provides some useful features that help to mitigate issues and solve some difficult problems.

MQTT provides a useful mechanism for reliability. Since radio connectivity is usually lost on the device side, MQTT allows messages to be stored on the broker until it is ready to be received. QoS(Quality of Service) is what allows messages, even if stored, to get to where they need to be and to make sure they only get there once. Which actually adds efficiency on top of reliability.

Another reason is that it's omnidirectional messaging feature makes it truly powerful. How is it omnidirectional? Any device or application can both publish or subscribe to any topic in the network. This means that it is just as easy to send a message to a hundred devices as it is to send to a million. This method is far better compared to having to establish a one-to-one communication channel with every device.

One final reason to note, and this is not a feature itself, is that MQTT is actually rather easy to implement. There are some ready-made applications and brokers out there. My experience with it so far and what I will explain here isn’t exactly ready made but I did leverage some tools and a platform that makes things so much easier.

Now that covers the basics of why MQTT is useful, let's look into a little more detail on how it works.

How does it work and what components and concepts are involved?

I've mentioned a couple of the core components and concepts so far such as the broker, client, topics, publishing and subscribing. Hopefully from what's been covered so far you have a some idea on how these things are used. But how do each of these work exactly and how do they interact with one another? 

Publish and Subscribe

Perhaps the first thing to understand is how the whole publish/subscribe model works. The main idea is that devices and applications which serve as clients are decoupled from one another, that is, they don't even know that each other exists. This is easier understood when you contrast it to the traditional client-server model where the client communicates directly with and endpoint establishing communication specifically to something and directing targeted messages.

So how do the clients in this model communicate if the don't connect to each other? This is where the broker comes in, a third component that handles the connection and communications. The broker will filter all incoming messages and then distribute messages only to clients subscribed to the relevant topic. Additionally to clients not needing to connect to one another, a publisher and subscriber don't have to be connected at the time and operations on both don't need to be interrupted during publishing or receiving.

This model is advantageous for both message filtering and scaling which can be difficult challenges to tackle especially if not taking into consideration early in the architecture design process.

Client and Broker

Let's elaborate a little more on the client and broker components. An MQTT client is any device, from a microcontroller based "thing" to a fully fledged server, that runs an MQTT library and connects to a broker through a network. A client can be both a publisher or subscriber depending on which operation it is doing at a given time. The client implementation protocol is straight forward to use at least supposedly.

The counterpart to the client is the broker. The broker is the centerpiece of the publish/subscribe model. The broker receives all messages, filters them and then distributes them accordingly, to the agreed QoS level, and to the appropriate subscribed clients. The broker is also responsible for authentication and authorization of clients. Since the broker is the most exposed component it is important that security measures are put in place when integrating it into backend systems.

Topics

The broker manages communication between clients through the use of "topics". When a client wants to send a message somewhere it will be directed at a specific topic. The broker then delivers those directed messages to clients that are subscribed to that topic. So to understand clearly, when we're speaking of the act of publishing and subscribing, it's specifically directed to these topics. 

A topic is simply a UTF-8 string . It can consist of multiple levels and each topic level is separated by a forward slash (/). Topics are case sensitive and they don't have to be pre-registered at the broker. Topics are a great way to organize data flow as is but also as the scale increases.

Subscriptions

MQTT clients subscribe to topics to receive messages. The process goes like this, when a client wants to subscribe it first sends a SUBSCRIBE request to the broker. The request can include multiple topics. The broker then responds with a SUBACK acknowledgement. Subscriptions also contain an QoS setting that downgrades the setting set by the publisher (more on that in the QoS section).

A client can subscribe to multiple topics at once by using wildcards. There are two types of wildcard and they are different ways to address multiples of topics. A wildcard can only be used to subscribe to topics, not to publish messages. The two types of wildcards are: single-level and multi-level

A single level wildcard replaces one topic level and is represented by the plus symbol (+).

The above string for example would produce results that match all the strings in the specified topic level but where the wild card is is would include all available topic levels and brings results as:

• Building/firstfloor/office1/temperature
• Building/firstfloor/office2/temperature
• Building/firstfloor/office3/temperature
• Building/firstfloor/reception/temperature

A multilevel wild covers many topic levels and is represented by the hash symbol (#).

This wildcard would produce the following kind of results:

• Building/firstfloor/office1/temperature
• Building/firstfloor/office1/humidity
• Building/firstfloor/reception/temperature
• Building/firstfloor/reception/humidity

Quality of Service (QoS)

Getting started with MQTT, one of the more confusing things you may initially encounter is QoS levels. What are they? and how important are they? QoS or Quality of Service levels are there to ensure message delivery. It is an agreement between the sender and receiver that defines the guarantee of delivery for a given message. QoS allows the client to choose a level of service that matches how the application operates and according to network reliability. There are 3 service levels in MQTT:

• At most once (0)
• At least once (1)
• Exactly once (2)

Understanding these levels is helpful when deciding which levels are the best choice for any specific application. There are also two sides of message delivery to consider when deciding on QoS levels in MQTT.

1. Messages from the publisher to the broker
2. Messages from the broker to the subscribed receiver

A publisher would set a QoS level when sending to the broker while the receiver may set it's own QoS level that is equal or lower than the publishers when subscribing to topics to receive messages from the broker. 

With all that out the way lets break down the service levels:

QoS 0 - at most once

QoS 0 is the lowest level and is used to make sure that a message is delivered no more than once, otherwise guaranteeing a best-effort delivery. There is no guarantee of delivery. The best case scenario is that the message is simply received hence the "at most once". The receiver does not send a return acknowledgement and the sender does not store or queue messages and resend. Thus level 0 is aptly called "fire and forget". This service level is used when you don't need much reliability for each individual message. Good for use cases where, for example, data is sent frequently.

QoS 1 - at least once

QoS 1 guarantees that a message is delivered at least one time to the receiver. The sender stores the message until it gets a PUBACK packet from the receiver indicating acknowledgement. A message could possibly be sent multiple times. there is a packet identifier for each message to match the PUBACK packet to its corresponding PUBLISH packet. If the publisher does not receive the PUBACK packet in a reasonable amount of time then it resends the PUBLISH packet.

QoS 2 - exactly once

QoS 2 is the highest level of service in MQTT. This level guarantees that each message is received only once by intended receivers. QoS 2 is the most reliable but also slowest service level quality. It provides it guarantee through a four-part handshake mechanism that uses at least two request/response flows between the publisher and the sender.

Security

It's always important to take make sure that data is secure as its being transported. As IoT devices continue to proliferate, security becomes more of a concern. There are ways to implement security measures outside the mechanisms provided by MQTT itself and those may be more important. Here, let's look at how what security measures come with MQTT. There are 3 concepts that apply when communicating through this protocol that deal with security. Those concepts are identity, authentication and authorization.

The use of identity is simple, when a client initiates a request to a broker it includes a client ID. The broker then would determine if that ID is valid, making the client eligible to connect. It is optional to have a username and password requirement when requesting to connect to a broker.

Adding a username and password is one way of implementing authentication. The protocol also allows a device to authenticate with a X.509 certificate. X.509 is a digital certificate that uses a public key infrastructure (PKI) to verify that a public key belongs to a client. 

Authorization occurs along the two actions that can happen when a client is connected to a broker, publishing and subscribing. If Topics did not require authorization then any client could subscribe and publish to any topic. There are different types of authorization, which won't be discussed here but basically, these authorizations allow the broker to be configured with topic permissions. The broker could be set so that at runtime, the broker can determine topics that are allowed and what operations are allowed as well. If a client attempts an operation that is unauthorized the broker can perform a counter action or simply prevent a client from subscribing or publishing to whichever unauthorized topic it is attempting to reach.

Summary

So in summary, we've seen how MQTT uniquely works to address the current and expected challenges posed by IoT applications with it's flexible architecture. By way of a broker, clients can easily integrate into the communication network without having to establish a connection to each specific client that it may want to message. Topics allow for data to flow between clients in all kinds of directions without easily introducing complexity into the system as more clients are added. QoS service levels help to balance out resource usage and reliability. We also looked into some, but not all, of the security measures that can be implemented in order to assist in making a secure system.

There is of course more to know about each of the features mentioned but hopefully this information works as a good starter. We've covered the core concepts to not only provide a basic understanding of what MQTT is but also show a little bit about what it is good for and why. The next thing to do would be to use try it out in an actual solution. Explore and understand each component on a deeper level and see how to configure everything correctly for a given application.