Part I - Why I chose the ESP8266 as my wireless micro controller and how I use MQTT with it.

Most of my articles discuss a specific topic. This one will be a little different, Part 1 will explore why the ESP8266 using micropython and MQTT is an excellent choice for a wireless IoT sensor. Part 2 will be a 'how to 'article walking through the steps to setup and program the sensor and the MQTT broker. I'm planning the Part Tow - 'how to' for a few reasons; 1) while there are many, many articles about ESP8266 and MQTT, I haven't found one article that nicely puts it all together, 2) this is a convenient way to record my steps when I need to repeat them in the future. Since the ESP8266 boards are inexpensive (about $3 to $4 each), I will be repeating this process multiple times and, 3) A few things I learned were really buried in the documentation. Sharing here may save others time and aggravation.

The ESP8266 is a microcontroller board, that is available in many variations. The NodeMCU variation is the one I'm using and the one I recommend.

It may be helpful to first explain the differences between a microcontroller board (ESP8288, Arduino, Micro Bit, Quark) and a single board computer (raspberry Pi). Generally speaking, a microcontroller board does not have an operating system or support the peripherals you'd expect with a computer (monitor, keyboard, disk drive, etc.). A microcontroller often is embedded in another system (for example a microcontroller embedded in an automobile) and is programmed using a separate system or IDE (Integrated Development Environment). Because microcontrollers lack a conventional operating system and do not need to operate the peripherals mentioned earlier, they are physically much smaller, use substantially less power, are far less complex and area fraction of the cost of a computer. A single-board computer is exactly what is sounds like, a full computer on a single printed circuit board, it is capable of running an operating system, operating peripherals, and running multiple applications at the same time. It is important to mention both microcontrollers and single board computers may have analog and digital inputs, typically referred to General Purpose Input Output (GPIO), these can be in addition to other types of common I/O interfaces like SPI (Serial Peripheral Interface),I2C (Inter-Integrated Circuit) USB (universal serial bus).

So when do you use a microcontroller versus a single board computer? The following table provides some general guidelines for consideration. These are not hard fast rules. As an example while you can connect a display to a microcontroller using 4 wires connected to the SPI interface however, its not as simple as plugging in an HDMI cable to a single board computer.

Five reasons why I recommend the ESP8266 microcontroller board for wireless IoT application prototyping? 

1.    it has Wi-Fi built into the board. While there are small wireless addon boards (often referred to as shields) that you can integrate with a microcontroller board, the NRF24L01 is one example. These radios are much more complicated to setup, often requiring you to deal with control registers and radio channels. Having the Wi-Fi integrated right onto the board makes it far easier. 

2.    Many micro controller boards operate on 3.3 volts DC power, which if you are powering from batteries may create the need for a voltage regulator. The ESP8266 board has a USB connector and a built-in regulator, so if you’re connect the controller to your computer (to program) the board is powered from the same USB cable that is used to transfer data. Once programmed and in use if you choose to power the ESP8266 with a battery you could use a simple USB battery pack.

3.    The ESP8266 can be flashed with firmware that supports micro python. Micro python is a lighter weight version of Python, so if that is your language preference you’re all set.  

4.    The are several open source programming tools or Integrated Development Environments (IDE’s) that you can use with the ESP8266. If you keep the original firmware and wish to program in C++ the Arduino supports the ESP8266, you need to use the board manager to download/load the specifics for your board, but this is very easy. To program in micro python both the Thonny and Pycraft applications support both flashing the firmware and transferring programs.

5.    Cost, if you buy them three at a time their cost is approximately $3 a board.

6.    The ESP8266 has a deep sleep mode that uses very little power making it ideal for low powered or battery powered applications.

What is MQTT? Wireless sensors, especially in large scale, need a secure, reliable, scalable, and efficient means for transmitting data. One such solution is MQ Telemetry Transport or better known as MQTT. NOTE the is lots of debate about what MQ part of the protocol name stands for. Some say its stands for messaging Queue, although MQTT is NOT a traditional message queuing solution. Some say it is for the MQ series of products back when the protocol was created. This protocol was invented in 1999 by IBM as a bandwidth-efficient and lightweight protocol to transmit data over high latency networks for monitoring oil pipelines. The MQTT protocol uses traditional TCP/IP network transport, however, any network protocol that provides ordered, lossless, bi-directional connections can support MQTT. 

How does MQTT work? Similar to a client/server scenario, MQTT uses a broker/client arrangement, however, in a client/server environment the client typical communicates directly with an end point. With an MQTT solution there are two types of clients, subscribers and publishers and with MQTT the clients are decoupled from each other and everything goes through the broker. An IoT sensor that has data to send is a publisher, it publishes its data to the broker. A client that wants to read the data is a subscriber. The broker is responsible for receiving and forwarding the data. The messages (also known as payloads) can be any data format, one of the basic and core concepts of MQTT

To maintain order and keep messages organized MQTT uses topics.   A topic is a hierarchically-structured string that might be best explained with an example. Imagine a multi-story building at 1 Main Street. The topic for a temperature sensor on the first floor, with and an ID of temp_005 might look like the following: 1_Main_st\1st_floor\temp_005. When the temp sensor publishes its data (to the broker) it will use the topic 1_Main_st\1st_floor\temp_005 in its program code. For the program that subscribes or wants to read the data from temperature sensor, the program would subscribe to the topic 1_Main_st\1st_floor\temp_005. Its unlike that a program would want to subscribe to a single and very specific topic. For this reason, MQTT also uses wildcards, again we’ll use an example to explain. If I write a program to subscribe MQTT messages and I only want to get messages from devices on the first floor I can use the topic wild card #, so the topic I’ll subscribe to will be 1_Main_st\1st_floor\#

The preceding is a very basic primer on the essentials of MQTT. Some of the advanced features of MQTT include Quality of Service (QoS), message queueing,

So, what is good about MQTT? Now that we’ve explained a little bit more on how MQTT works let’s return to the topic of what are the benefits of MQTT. The code for MQTT subscribers and publishers is very small and efficient, making it an ideal choice for micro controller boards with limited memory and processing power. It can be very reliable on networks that have limited bandwidth such as cellular and satellite. It can be used to slowly grow a very large network of sensor, for example millions of sensors. As additional sensors are added multiple brokers may be added too with devices known as load balancers that make sure the messages are evenly distributed to the brokers. The publisher/subscriber model removes direct communication between the publisher of the message and the recipient/subscriber. The filtering activity of the broker makes it possible to control which client/subscriber receives which message. This decoupling has three dimensions: space (publisher and subscriber do not need to know each other, for example: no exchange of IP address and port), time (publisher and subscriber do not need to run at the same time), and synchronization( there is no synchronization between the publisher and subscriber, in fact the subscriber can be taken off line and that has no bearing of the publisher).

So, what is bad about MQTT? Because topics play a critical role of organizing and filtering messages, you need to carefully consider the structure of the topics beforehand. Because the publisher/subscriber clients are decoupled it is possible published messages are being read by no one. MQTT recently came under fire for security concerns. I think its worth pointing out that some of these security concerns are not specific to the MQTT protocol. Like any technology if you ignore the security features that are built into the technology (or in this case protocol), you create a recipe for disaster. However, there are some legitimate concerns. Some of the IoT devices lake the resources that may be needed for implementing strong security.

Look for the next article where I walkthrough setting up the board, programming, and connecting to the MQTT broker.