The role of the MQTT protocol in the development of the industrial Internet of things

We often host events to which we invite industrial automation experts. In 2016, Arlen Nipper, who is one of the creators of the MQTT protocol, came to us. We want to share his report in Russian translation.


Today I will talk about the Industrial Internet of Things (IIoT) and the MQTT data exchange protocol . In 1978, I studied electrical engineering at the University of Oklahoma. I wondered: why am I doing this, it's boring and not exciting. Then I had the opportunity to be an intern at Amoco Pipeline. The company has installed an automated data management and control system - SCADA (Supervisory Control & Data Acquisition). It included PLCs (programmable logic controllers) that transmitted data to the PDP-11 central computer via multi-channel telephone lines via Bell 202 modems. And in 2016 we use the same SCADA systems as we did over 35 years ago.

And the same SCADA systems, production automation systems, production control and management systems become an element of the IIoT infrastructure.

What is the IIoT infrastructure for? Users tend to gain more opportunities by spending less resources. This cannot be achieved without obtaining and using lower level production data.

First, information technologies were introduced everywhere. Then “clouds” appeared and everyone got access to the Internet. Now it remains only to use the "cloud" for SCADA systems.

Therefore, we started rewriting old closed source protocols. So on the market appeared Modbus, Allen-Bradley, DNP 3.0. And then there was a deregulation of the activities of telecommunications companies, including AT&T. Prior to this, production process control systems, SCADA systems, etc. worked on excellent conditions: AT&T received large subsidies and was ready to pull its telephone lines wherever we wanted. After deregulation, prices soared, and quality collapsed.

We began to use VSAT systems, but they worked slowly, with a large signal delay, and constant polling of devices was very expensive.

As a result of a joint project between Phillips 66 and IBM, in which I took part 19 years ago, the network protocol MQTT (MQ telemetry transport) appeared, which has been used for almost 20 years. Back in 1999, we had no idea about the Internet of things or the "cloud", but simply looked for ways to solve the problem. But we managed to create a protocol for critical real-time pipeline monitoring systems. Today, the MQTT protocol is one of the most used application protocols.

Actually, the Internet of things was conceived as a normal Internet of people, uniting users through web browsers, HTTP, etc., but which was supposed to combine “things”. Then we created the Industrial Internet of Things for production control systems, which has nothing to do with the ability to adjust the temperature of the thermostat through the phone.

To make the Industrial Internet of Things work, you need to:

1. “Unbind” devices from applications and connect with the infrastructure.

Suppose I installed a beautiful Advantech UNO computer, developed an excellent application for it, to which I connected the computer through a protocol. This means that I rigidly linked the capabilities of the computer with the capabilities of the application.

And even if I find a solution to a problem, tomorrow it may not work. For example, to take advantage of a much larger amount of data, I have to change the code.

That is why devices need to be attached not to the application, but to the infrastructure, and then embed applications in it. In this case, we will not be limited by the capabilities of applications.

2. Create a better solution at the level of operating technology (OT) than the existing one.

I’ve been developing this technology for nineteen years at IBM, but all implementation attempts were unsuccessful because we tried to push IIoT technology from IT to production.

But the IIoT solution must provide data from “things” and optimization at the operational level, regardless of whether the company is creating IIoT — a plant, a pharmaceutical company, a water supply and sanitation system, or oil and gas companies. And now I am trying to create an effective OT solution in B & B, because the Internet of things can only be created from the bottom up.

According to a 2016 study, MQTT is the most used protocol on the Internet of things (HTTP takes first place, but does not provide real-time control, and we do not take it into account).

There are many servers on the market that support the MQTT protocol.

^{MQTT Servers}

At the same time, the number of client MQTT technologies is limited.

^{MQTT Clients}

But in IIoT solutions you need to apply technologies known to yesterday's students of engineering and IT specialties. So, graduates in computer specialties, for example, 2016, most likely use a single-board Raspberry Pi that supports the MQTT protocol to turn on and off the light in the room. However, they may not know what OPC UA, Modbus, Allen-Bradley or DMP 3.0 protocols are. The openness and accessibility of such technologies will lead to the emergence of a large number of SRP solutions.

Accurate instructions for using MQTT in critical real-time production control systems do not exist. But we developed the Sparkplug MQTT specification, which defines some basic rules for creating real-time production control systems using the MQTT protocol:

• MQTT Topic Namespace,
• MQTT Payload Definition,
• MQTT State Management,
• High Availability / Redundancy / Scale.

This is an open source specification that is publicly available . In addition, we have developed a reference implementation of MQTT Client for message flow in C, Java, JavaScript, Python, and Node Read. Therefore, our partners in the Advantech ecosystem use a single specification.

So, we must “untie” devices from applications and offer a more advanced OT solution.

We strive to ensure that Advantech products interact with things in IoT:

• liquid level sensors in tanks,
• chromatographic gas analyzers
• corrosion protection and monitoring units
• flow rate sensors,
• Hydraulic motor speed sensors, etc.

The means of such interaction is the protocol for the delivery of data collected by all these “things”, which needs the infrastructure and the channels of interaction between its components.

Our MQTT infrastructure includes devices that track physical events and publish data about such events on secure message brokers. The Advantech product line allows you to not only create the Internet of things, but also use SmartFlex and eWorks routers, which publish data and provide an interface for monitoring them.

After we “untied” devices from applications, you can embed devices and Internet access modules into the infrastructure and sign them to data published in real time via the MQTT protocol. At this stage, it is necessary to demonstrate to the user, production manager or SCADA manager that our OT solution is better, faster, safer and easier to scale than conventional SCADA systems.

Previously, several SCADA systems could not be subscribed to the same topic. But if the systems are not tied to the application, we don’t even have to determine which SCADA system should receive information in the first or second turn - all systems, programs and devices can have access to the same production data in real time.

And at this stage, you can choose other interesting solutions and embed them in the infrastructure, for example, asset management applications, optimization of design and technical work, etc.

^{Advantech Unique Energy / Environment Solution Topology A}
streamlined data flow allows you to begin creating the Industrial Internet of Things with which we are already familiar, including big data, cloud computing, etc. You can use Microsoft Azure, IBM Bluemix or AWS IoT, as well as the well-known Hadoop and Big Data, Storm and Spark and various visualization and analytics tools. But it is impossible to be at this level if the devices are connected with applications and are not built into the necessary infrastructure.