Get best of the week

Xiaomi Gateway 2 can not be soldered



Hi Habr! In previous series, I:

  • I bought devices from Xiaomi for a smart home and, through a soldering iron, made them work in an exciting way without native servers via the home assistant ( link to post )
  • I wrapped the web interface from the home assistant in electron ( link to the post ) with support for notifications, menus, dotbar, etc. ( code here )

But as more and more devices appeared, a strange thing began to open up ...

For example, that Aqara Wireless Relay 2ch (LLKZMK11LM) did not support xiaomi_aqara integration, and when the Issues were launched on GitHub, the code owners replied that it was impossible to support this relay without updating the device firmware. “Something is wrong here,” I thought, and went to understand the code, how it happened. And so I got carried away that I washed down the integration first to work with the relay via xiaomi_miio ( link ), and then I implemented support for the cube and Xiaomi button for this integration ( link ), and now I will tell you why and how.

Developer API


Once upon a time, Xiaomi releasing a gateway (lumi.gateway.v3) was equipped with its developer api and even documentation for it. The Developer API is the thing that communicates over UDP on ports 4321 and 9898. The community really liked this move and it responded with a bunch of repositories that integrate the device into everything that is possible. Working with this developer api formed the basis of the home assistant xiaomi_aqara integration ( link ) and everything was great so far ...

While Xiaomi did not override and they did not hammer out developer api support, identifiers of new devices stopped appearing in it. The desired relay is in the list of devices connected via gateway to zigbee, but there the relay appears with an empty model identifier, without which it is impossible to do anything. Then, more recently, Xiaomi rolled out a gateway firmware update after installing which it was impossible to turn on developer api without a soldering iron (if it was not turned on before the update), link .

Once in the heart of this Santa Barbara, I burst into tears and went to order the German ZigBee ConBee2 dongle on Amazon to work with zigbee devices without a gateway. But while the Germans in the conditions of the apocalypse were preparing to send my order, I, tired of waiting, began to dig deeper ...

More native MIIO protocol


Miio is a newer protocol for working with Xiaomi devices connected to the network via wi-fi. The native Mi Home application communicates via this protocol with wi-fi sockets, and with IR Remote, and vacuum cleaners, and with everyone else, and even with my xiaomi gateway v2. This is also a UDP protocol, but this time it is not port 4321 that is involved, but port 54321 (I consider a great method for selecting ports). “Goofy,” I thought: “So she also knows another protocol, but should I try to connect the relay using this new protocol?”

The miio protocol encrypts all messages with a token, but, fortunately, the methods for extracting the token and encryption / decryption have already been opened and work with them has been implemented in several libraries (link once , link two) I am not the first to integrate the new device using the miio protocol, so the path was known and I followed it.

The way is this:

  • We get security token devices by extracting them from the logs ( link ) or from a modified application ( link )
  • We remove traffic between the mobile application and the device
  • We decrypt UDP packets with security token received at the beginning
  • Learning code to create similar packages

I took traffic directly from my wi-fi router, fortunately there is a custom asuswrt-merlin firmware for it, in which tcpdump can be installed using Entware. The utility is fine, but it does not understand dumps from tcpdump, but it understands the json-dump generated by Wireshark. To override the binary dump from tcpdump to json, I used the tshark utility. After that I saw which commands the mi home application controls the relay. I generated PR with the addition of relay support ( link ).

I canceled the order for ConBee2, because the software method is both faster and more fun ...

Why do we need a blacksmith?


And so it means that I, with a dash bare, find myself in a situation where everything seems to work, but now in my home assistant my Xiamo Gateway 2 is represented by two entities, one under the developer api protocol (xiaomi_aqara integration) and another one under the miio puncture ( xiaomi_miio integration ) . And I do not like to solder, I wanted to simplify my life and myself. It was necessary to implement the missing part of the devices in xiaomi_miio.

For the most part, after creating the software strapping, this is a mechanical job:

  1. enabled tcpdump
  2. created an automation action in the Mi Home application that I want to track
  3. initiated this automation
  4. decrypted the dump
  5. copy-paste from decryption to code
  6. send commands from the console
  7. file finishing to shine

But hands itch, and adventure calls ...

Fake miio device


An autopsy showed that there is no callback mechanism in the miio protocol. Sensors can be pulled once every 20 seconds, but for the correct operation of buttons and cubes it was a bad idea to interrogate the device 10 times per second. In this paradox, I saw an interesting activity, but I did not really want to deal with the mechanical encoding of the captured packets.

A study of another library working with the miio protocol ( link ) surprised me to the core. The authors have assembled a harvester that communicates with the device via the miio protocol, but uses developer-api to intercept events from cubes and buttons. This approach thoroughly strained me, it leads to the fact that part of the functionality starts working immediately, but for part it is necessary to solder the device ...

Armed again with tcpdump and, having modified the decoder with the decryption function from all the devices in one run, I began to listen to everything that flies over the network to UDP port 54321. And I see a rather interesting picture. When I create automation like “If you turned the zigbee cube, turn on the wi-fi socket”, json is sent from the application to the gateway using the miio protocol inside the data field whose json is encoded in a string; “DSL” - thought Stirlitz, for some reason, from time immemorial, everyone has been packing this way ... Details are here .

Viewing the contents of this line showed that the reaction to the action is not transmitted somewhere to the clouds, directly in this json program appeared the ip address and encryption token of the wi-fi socket (this paragraph contradicts the paragraph from the first series, I was wrong there; (gate he communicates with friendly devices himself - without the help of clouds.) Yes, and the script itself resembles automation from the home assistant, there are trigger parameters, there is ip where to send a reaction. That is, no pub / sub protocol could be found in the dumps.

But the idea was born of everything to catch all necessary events from buttons and cubes:

  1. We emulate a miio device in the application, respond to PING applications and requests from gateway
  2. We write a script generator for the gateway to generate one per pair ~ (device, event)
  3. We send scripts so that when an event occurs, a miio request comes to the emulator

It took a couple of days to realize this and bring it into a divine form.

At the moment, the python-miio library can work with relays and with a cube and with a button, link .

It remains to attach this update to the home assistant and it will be possible to work with Xiaomi gateway 2 without soldering and developer api.

If anyone is ready to test on their devices and / or add support for new ones - join :)

Still on sale is the glorious Xiaomi 1C vacuum cleaner here and such a shortcoming ( link ) for its integration into the home assistant. I think to take ... Do you need it?

More articles:


All Articles