Adapter MQTT

Fork me on GitHub

Description

MQTT (Message Queue Telemetry Transport) is a lightweight protocol used for communication between devices (M2M – machine-to-machine). It uses a model publisher-subscriber to send messages over TCP / IP protocol. The central part of the protocol is MQTT-server or broker who has access to the publisher and the subscriber. This protocol is very primitive: a short title, without integrity (that is why the transmission is implemented on top of TCP), does not impose any restrictions on the structure, coding or database schema. The only requirement for the data in each packet – they must be accompanied by the identifier information channel. This identifier specification called Topic Name.

The MQTT Protocol requires a data broker. This is the Central idea of the technology. All devices send data only to the broker and receive data also from him only. After receiving the packet, the broker sends it to all devices on the network according to their subscription . For the device to get something from the broker it must subscribe to a topic. Topics arise dynamically upon subscription or upon arrival of the package with this topic. By subscribing to a topic, you can give up. Thus topics are a convenient mechanism for organizing different kinds of relationships: one-to-many, many-to-one and many-to-many.

Important points:

  • the devices themselves establish communication with the broker, they may is behind a NAT and does not have static IP addresses,
  • you can use SSL to encrypt the traffic,
  • MQTT brokers allow you to connect to them through the WebSocket protocol on port 80,
  • different brokers may be interconnected by subscribing to messages from each other.

Information

 

Current version
Necessary conditions /
Developer Bluefox
Keywords
data transmission, sensors, cloud, publish/subscribe
Github icon_link Link
Platform Javascript/Node.js
License MIT

Installation

Installation is on the Driver tab page of the administration system. In the driver group Network find a line called MQTT Adapter, and press the button with the plus icon in the right side of the line.

 

You will see a pop-up window driver installation, after installation, it will automatically close.

If all goes well, on the Settings driver tab appears mqtt.0 installed instance of the driver.

Setting

As stated above, MQTT protocol implies a broker and clients. ioBroker server can act as a broker and a client. Setting to specify the operating mode – the type (server / broker or the customer / subscriber) Consider every option.

IoBroker working as MQTT-broker

Basic settings if you intend to use the server/broker is shown in the picture:

  • Use WebSockets – if there is a need to use WEB sockets for connection, you must install this option, with TCP-server will run in parallel with the WebSocket server,
  • Port – the port to connect on TCP (default is 1883), a WebSocket server (see option above) runs on port +1 (default: 1884),
  • SSL – this option is used if you want to encrypt all traffic (TCP or the WebSocket), thus it is necessary to specify certificates – simply select from a list of pre-set (specified in the system settings, see the systems management driver description),
  • authentication settings (username and password) – indicate, if necessary a specific user authentication, this setting is always used in conjunction with SSL-encryption option (not to transmit passwords in clear text over an insecure connection),
  • Mask private values – the template (or several comma-separated) to filter variables ioBroker, which will be sent to the client, you can use special characters to specify a group of messages (for example, memRSS, mqtt.0 – can be transmitted all the variables memory status of all drivers and all mqtt.0 driver instance variables),
  • To send only changes – sending data to the client will be made only in case of change of a variable (if the state simply update – the value is not changed, the customer message will not be sent) from the client will be accepted any message, even if the value has not changed,
  • To give private values at startup – for each successful client connection will be transferred to all known states (defined by the mask state) – in order to tell the client about the current state of the ioBroker,
  • Post status subscribes – immediately after the subscription will be sent to the customer value of the variable on which it is signed (at the first start or restart the client will receive the values of variables on which it is signed, can be used to initialize variables),
  • The prefix for all values – if the specified value, it will be added as a prefix to every sent topic, for example, if you specify iobroker/, then all topics sent along the following lines: iobroker/mqtt/0/connected,
  • Output log for each change – in the log file will display debugging information for each change,
  • To send not only commands, but also the state (ack=true) – if this option is not active, the client will only send variables/commands with ack=false, if the flag is set, then variables will be transferred regardless of the state of ack (false / true),
  • The maximum length of the name of a topic – the maximum number of characters for the description of the topic, including service.

As an example, consider the exchange of data between the client based on the arduino board and the broker is an instance of mqtt.0 driver system ioBroker.

  • – the client – the fee for developing arduino UNO + ethernet shield based on W5100 chip,
  • – to work with the ethernet board uses the standard library for working with MQTT library Pubsubclient,
  • – the AM2302 sensor (temperature and humidity) connected to pin_8 for the survey used library with DHTlib with DHTlib resource github.com,
  • – led led_green is connected to pin_9, control in discrete mode on/off
  • – broker – ioBroker system driver mqtt.

Format topics of data exchange:

  • example1/send_interval – client signed to change the transmission interval of the temperature readings and humidity (int value in seconds),
  • example1/temp – client publishes a specified temperature interval with DHT22 sensor (float type),
  • example1/hum – client publishes a specified humidity value intervals with DHT22 sensor (float type),
  • example1/led – the client is subscribed to the state change of the led (the text on/off or 0/1 or true/false).

Driver settings will be as follows:

Connecting via TCP (WebSocket is not necessary), default port 1883. The client within the local network, so to encrypt traffic and authenticate the user is not necessary. We will send only the changes since the client signed on the send interval indications and led state to obtain information about the update (without changing the value) to a variable makes no sense.
To publish the subscription – note this option, as when you first connect (or connected after disconnection) of the client, he must know the state of the variables on which it is signed (a current interval of sending and whether the LED is to be turned on). Setting to send variables ack = true or false is also worth noting, as a variable (which signed the client) can change any driver / script / VIS and any changes should be sent to the client.

The complete code for the arduino board will look like this:

The result of the part of the broker (temperature and humidity data is updated with the preset time period):

The result of the client-side (incoming data subscription output to the console for debugging):

IoBroker working as MQTT-client

For an instance MQTT driver earned as a client / subscriber – you need to choose the appropriate type of configuration. In this set of options will change slightly:

  • Connection settings – specifies the URL and port of the broker (if you want to encrypt traffic, indicated SSL) – settings to connect to the broker,
  • Authentication settings – user name and password, if the broker requires authentication (it is appropriate to use SSL to avoid transmitting the password in clear text),
  • Patterns – a mask for variables for which the customer subscribes (variables broker), the values are listed separated by commas, the # (pound) is used to indicate the set,
  • Mask private values – filter variables that should be published (client variables) whose values are listed separated by commas, for indicating a set use the symbol * (asterisk),
  • To send only changes – the client will publish only the variables that changed value (according to mask),
  • To give private values at startup – if this option is checked, the will be published all the States (according to mask) every time a connection is established, to declare the available variables and their values,
  • The prefix for all values – if the specified value, it will be added as a prefix to each published topic, for example, if you specify client1 /, then all topics will be published the following lines: client1/javascript/0/cubietruck,
  • Output log for each change – in the log file will display debugging information for each change,
  • To send not only the team, but also the state (ack = true) – if this option is not checked, the broker only sent variables / commands with ack = false, if the option to note that will be sent to all data, regardless of ack = true or ack = false,
  • The maximum length of a topic – the maximum number of characters for the description of the topic, including service.

Examples for setting the subscription mask variables (patterns). Consider topics:

  • “Sport”
  • “Sport/Tennis”
  • “Sport/Basketball”
  • “Sport/Swimming”
  • “Sport/Tennis/Finals”
  • “Sport/Basketball/Finals”
  • “Sport/Swimming/Finals”

If you want to subscribe to a certain set of topics, you can use the characters # (pound sign) or + (plus sign).

  • “Sport/Tennis/#” (subscription only “Sport/Tennis” and “Sport/Tennis/Finals”)
  • “Sport/Tennis/+” (subscription only “Sport/Tennis/Finals”, but not “Sport/Tennis”)

For JMS topics, if you want to subscribe to all topics “Finals”, you can use the characters # (pound sign) or + (plus sign)

  • “Sport/#/Finals”
  • “Sport/+/Finals”

For MQTT topics if you want to subscribe to all topics “Finals”, you can use the + (plus sign)

  • “Sport/+/Finals”

As an example, consider the exchange of data between the two systems ioBroker.

There is a working system ioBroker for BananaPi-Board (IP address 192.168.69.51), it launched MQTT- driver in the server/broker mode from the example above. To the server connects a client that publishes data from the sensor DHT22 – temperature and humidity, as well as signed variables of interval measurement transmission and the status led (enable/disable) – in the example above.

The second operating system ioBroker on the Board Cubietruck, it will run the MQTT driver in a client/subscriber mode. He signs up for the variables temperature and humidity of the broker (which, in turn, receives from another client) and will publish all the script variables – the state of the battery board (only the changes).

Client configuration will be similar to the following:

Connection type – the customer/subscriber indicates the IP address of the broker and the port (default 1883). Traffic encryption and authentication is not needed. Mask for the subscriptions (Patterns) – mqtt/0/example1/hum,mqtt/0/example1/temp – client is subscribed only on temperature and humidity (values separated by comma without spaces). Mask the data for publication – javascript.0.cubietruck.battery.* – publish all the script variables cubietruck in the group battery driver javascript.0.

To send only the changes – send state variables batteries (makes no sense to send if the value has not changed). To give private values at startup – when starting the driver, the client immediately will release all variables according to the mask – even if they are null or empty to create variables in the broker. To send data with ack=false, variables work battery updated driver javascript, so they are always ack=false.

The result of the work on the client side (temperature and humidity data of another customer – see the example above):

The result of the broker (status data of the battery client):

Application – MQTT gateway protocols – ModBus RTU

Driver MQTT can be used as a gateway for various protocols to connect new devices to the system ioBroker or any other. A universal basis for the development of such solutions are arduino boards. In a network many examples, libraries and best practices. A huge community is working with these controllers, and the system integrated a variety of devices/equipment/devices. For example, consider the common industrial protocol ModBus. In ioBroker system has a driver to work with it – version ModBus TCP (over ethernet). A set of sensors, controllers and actuators work physically on the RS-485 Network / 232 and ModBus RTU protocol. In order to integrate them can be applied MQTT Gateway – ModBus RTU based on arduino platform. Consider an example.

There is a temperature and humidity sensor (for the test on the basis of arduino pro mini board DHT22 Sensor), that outputs data via ModBUS RTU:

  • Port UART (you can use MAX485 chip to convert RS-485 interface) running at 9600 with options 8E1 (1 start bit, 8 data bits, 1 Even parity bit, 1 stop bit),
  • the address of the ModBus – 10,
  • temperature address 0 the value multiplied by 10 (the reader function 3),
  • humidity – address 1 value multiplied by 10 (read function 3),
  • PWM LED address 2 value 0…1023 to check the recording function (write function 6).

Connection scheme:

by Fritzing

Code for arduino pro mini controller produces the following:

To test the operation code and schema, you can connect to port serial board (for example, using a USB-UART Converter) and a special program to interview just made the temperature sensor and humidity with ModBus RTU interface. For the survey can be used, for example, qmodbus or any other program.

Settings:

  • port (choose from the list which port is connected to the serial Arduino boards);
  • speed and other parameters – 9600 8E1;
  • slave id: 10, read: function No. 3 read holding registers, starting address: 0, number of registers: 3,
  • slave id: 10, record: function No. 6 write single register start address: 2,

The answer in the program when reading should be approximately the following:

The answer in the program when recording:

Now configure the gateway itself and connect it to the iobroker

The gateway will be based on the platform arduino MEGA 2560 with ethernet shield – client MQTT, broker – an instance mqtt.0 ioBroker system driver. Choosing the MEGA 2560 due to the fact that on this Board more than one UART port, respectively, is zero Serial0 (pin_0 (RX) and pin_1 (TX)) or simply Serial – use to output debug messages, and Serial1 (pin_19 (RX) and pin_18 (TX)) – for slave via ModBus.

  • the client – the fee for developing arduino MEGA 2560 + ethernet shield based on W5100 chip;
  • to work with the ethernet board uses the standard library for working with MQTT library Pubsubclient;
  • for the survey on the modbus use library SimpleModbus version master;
  • survey on UART port (just connect the RX port master, TX port slave and respectively TX port master, RX port slave), transmission control port is not used (it is for RS-485);
  • port settings: speed 9600, 8Е1;
  • – the address of the slave device 10, a function of reading number 3 (read holding registers), recording function no. 6 (write single register);
  • – broker – ioBroker system driver mqtt.

Format topics of data exchange:

  • modbusgateway/send_interval – client signed to change the transmission interval of the temperature readings and humidity (int value in seconds),
  • modbusgateway/temp – client publishes with a a given interval the value of the temperature sensor DHT22 (type float),
  • modbusgateway/hum – the client publishes with a given interval the value of the humidity sensor DHT22 (type float),
  • modbusgateway/led – the client is subscribed to the state change of the led (PWM control value 0…1024).

СThe connection diagram will look something like this:

By Fritzing

For the test slave device energized from the master device. The Master in turn will work from the USB port, which is being debug (Serial0).

Driver settings will be as follows:

Connecting via TCP (WebSocket is not necessary), default port 1883. The client within the local network, so to encrypt traffic and authenticate the user is not necessary. We will send only the changes since the client signed on the send interval indications and led state to obtain information about the update (without changing the value) to a variable makes no sense. To publish the subscription – note this option, as when you first connect (or connected after disconnection) of the client, he must know the state of the variables on which it is signed (a current interval of sending and whether the LED is to be turned on). Setting to send variables ack = true or false is also worth noting, as a variable (which signed the client) can change any driver / script / VIS and any changes should be sent to the client.

The complete code for the arduino board will look like this:

This solution can be used as a prototype (example) ModBus network in your automation system. The data from the slave is transmitted with the desired spacing in the ioBroker.

MQTT client signed variables and redirects needed in slave-device on the ModBus network.

Application – connecting mobile clients

Recently MQTT protocol became very common due to the simplicity, economy of the traffic and the elaboration of good libraries for different platforms. There are many programs to work with MQTT on mobile devices, for example IoT MQTT Dashboard. With this program you can connect to the MQTT broker in a local network or the Internet. Consider an example, in the role of the broker will be the ioBroker system, to which using MQTT to connect the client – application IoT MQTT Dashboard.

In this example, we control the light controller MegaD-328, which is connected to the ioBroker with the driver MegaD. Controls relay (MegaD port P7) light in the lobby, a special script, which is signed by the state of the port – button P0 and MQTT-variable state mqtt.0.remotectrl.light.hall, which will publish the mobile client. This script toggles the state of the port that is bound to the switch (port P7), ie inverts it. It turns out that each time you press the button, electrically connected to port P0 (caught the true state) and every time you publish variable mqtt.0.remotectrl.light.hall value as true, the port P7 to turn on or off the light.

The text of the script will be like this:

Connect button and light bulbs to MegaD controller:

MQTT driver settings:

The mobile client can publish data to variable mqtt.0.remotectrl.light.hall and signs up for a real port status MegaD – megad.0.p7_P7. The configure publishing and subscriptions:

In total for one channel light control turn the control window (publish) and subscription window is a real condition light relay (for feedback):

Application – working with cloud servers

The example described above has several disadvantages. First, it is not always the mobile client may be on the same local network as the server ioBroker, and secondly, even if you implement port forwarding in the Internet and to protect the connection, not always the server itself ioBroker can accept incoming connection (located behind a NAT which has no access to settings). In the global network many different services that support MQTT – paid and free, for example sending weather data, geolocation, etc. Some services may act as MQTT protocol broker and can be used as a gateway (bridge) to output data from ioBroker the global network, or to obtain data in ioBroker.

As an example, consider the work of the bundles:

  • server / broker – service cloudmqtt.com (there is a free tariff),
  • customer/subscriber – the ioBroker system with access to the Internet, publishes data of temperature and humidity (see example above), publishes the real status of ports P7-P13 (relay driver MegaD megad.0 – light control), subscribing to properties of the remote light control (an instance of the driver mqtt mqtt.0),
  • customer/subscriber – the application of IoT MQTT Dashboard to work remotely – subscribe to sensor data of temperature and humidity, subscription to the real status of ports P7-P13 (relay driver MegaD megad.0), publication of variables of a remote control light (driver instance mqtt.0).

he result is the following structure:

Bundle driver mqtt.1 (broker) – Arduino UNO + Ethernet + DHT22 (client) as in the example above with a few modifications. Configuring an instance of the mqtt driver.1:

Code for the arduino platform:

The result of the work – mqtt.1 driver objects:

Now let’s set up publish/subscribe data to the cloud. For a start, register on the site cloudmqtt.com, select the desired rate, create instance, get settings:

For greater security it is better to create a separate user, assume that it will be user iobrokerwith the password 1234. Give user permission to read and write in any topic:

Next set the instance of the mqtt driver.0 to connect as a client/subscriber cloud broker and a list of publications/subscriptions:

  • connection type – the customer/subscriber,
  • connection settings – specify the URL issued in the control panel cloudmqtt.com the port will use 22809that works with SSL,
  • in the authentication options specify the user name and password,
  • patterns – our client ioBroker will be signed on all the topics that are in the cloud, so you specify here the number sign (#), you can use a mask to selectively subscribe,
  • mask of the eigenvalues client will publish to the server temperature/humidity and the status of all ports megaD (ports with relay P7-P13),this field separated by a comma specify the required variables:
    mqtt.1.example2.hum,mqtt.1.example2.temp,megad.0.p7_P7,megad.0.p8_P8,megad.0.p9_P9,megad.0.p10_P10,megad.0.p11_P11,megad.0.p12_P12,megad.0.p13_P13,
  • to send only changes – put a tick, will publish only the changes,
  • to give your own values at the start – just specify to create topics,
  • to send not only commands, but also the state (ack=true) – it should be noted that setting both the temperature/humidity updated driver mqtt (ack=true).

Settings saved, make sure that the connection is established (on the control panel cloudmqtt.com watch the log server). After some time, data will appear (in the panel link WebsocketUI):

In the end, it remains only to configure a mobile client, for example IoT MQTT Dashboard. Create a new connection:

Create topics for publication (for example, lighting of the hall – port P7 MegaD):

then create subscription for the topics (temperature, humidity, hall light on port P7 MegaD):

In the end, your dashboard might look something like this:

After the creation of the publications on a mobile device, in the driver instance mqtt.0 system ioBroker should appear variable light control that should be used in the script for lighting control (see example above):

Congratulations! Now you can control the system ioBroker and receive data via a cloud service!