👂 🍲 👩🏻‍🤝‍👨🏽 Remote control and drones 👨🏾‍🔧 🐮 🧓

In my first articledescribes the development of unmanned technologies in railway transport and refers to the presence of a remote control. Is remote monitoring and control necessary for unmanned trains? Our analysis unequivocally says that it is not permissible to launch autonomous trains without remote monitoring and control. The number of scenarios where you can not do without a remote operator-operator is huge. For example, for electric trains in each car there is an intercom with a button. But who will answer the passengers if the train is unmanned? Who will manage the actions of passengers through the intercom in the event of smoke or other contingency? Who will decide if there are obstacles such as ribbons, bags,that wound around the nearest pillar and triggered vision sensors? For us, the answer to these questions is the presence of the operator-operator, who must intervene in the management of an unmanned train in the event of any emergency. The question immediately arises: “Why do we need an unmanned train if an operator-operator appears instead of a driver?” The answer lies in the fact that the operator-operator controls not one, but several trains at the same time, and his intervention in control should be extremely rare.if the operator-operator appears instead of the driver? ” The answer lies in the fact that the operator-operator controls not one, but several trains at the same time, and his intervention in control should be extremely rare.if the operator-operator appears instead of the driver? ” The answer lies in the fact that the operator-operator controls not one, but several trains at the same time, and his intervention in control should be extremely rare.

World experience

Not only we have gone the similar way. In 2018, Alstom began testing remote control trains.

In 2019, SNCF tested a prototype remote control.

Figure SNCF Remote Control Prototype Figure

Below is a video test of the remote control technology of SNCF peers.

History of Remote Control Development

Our history of the development of remote control began in January 2017, when we were given the task of demonstrating a prototype of remote control of a shunting locomotive at the Luzhskaya station from Moscow in one week.

In order to have time to create a prototype, a game joystick was purchased, on which button labels were pasted, as shown in the photo.

Drawing. The first prototype of a remote control.

The following tasks were solved in one week:

installation of video cameras on a locomotive and organization of a channel for transmitting video data to a control computer
connecting the joystick to the control computer and writing a control program
organization of a channel for control commands and data from a locomotive

This became possible due to the fact that our locomotive was already equipped with an SDU-ML system (remote control system for a shunting locomotive), which allows us to control within the visibility range using a portable remote control, so we used the same protocol and radio channel, providing only relay commands from a control post to Moscow via optical communication channels with the help of specialists from the central control station of Russian Railways. As a radio channel for video transmission, we used LTE at a carrier frequency of 450 MHz from Tele-2.

The demonstration showed the technical feasibility of remote control and the next task was to develop a new control panel and add new functionality.

I would also like to note that this work was carried out on the basis of the successful interaction of several organizations:

NIIAS JSC - lead contractor and integrator
PKB CT - development of a part of the remote control, technical consultations
VNIKTI JSC - development of an onboard locomotive control system
CSS OJSC Russian Railways - organization of communication channels
Tele-2 - providing the LTE-450 MHz communication channel
Russian Railways OJSC - coordination of work.

Almost every two weeks we released a new prototype of the remote control

Figure. The first prototypes of remote controls

As you can see the first prototypes of remote controls to speed up work were done from plywood, later on we switched to 3D printing.

Drawing. Remote control of a shunting locomotive

For the convenience of remote control, the driver displays additional information about the current and permissible speeds, pressure, a given route and the distance to the end of the route, superimposed on top of the video image. On additional screens you can see the driver’s cab, station diagram.

Drawing. Remote control and control installed at the station Luzhskaya

For the created unmanned electric trains "Swallow" in 2018, a remote control and control was created, as shown in

Figure. Remote control for the electric train Swallow.

Its main difference from the remote control in the driver’s cab is the replacement of all mechanical switches with buttons with a status light. This is necessary when switching control from one electric train to another and understanding the condition of the equipment.

In 2019, successful tests of remote control were conducted.

In this work, along with JSC Russian Railways and JSC NIIAS, a large part was attended by:

NPO SAUT LLC - developer of an on-board electric train control system;
Ural Locomotives LLC - manufacturer of electric trains.

The following video shows the tests of the remote control train Swallow.

Technical issues

The most important role in the implementation of remote monitoring and control is played by reliable high-speed radio communications.

In order to transfer heterogeneous traffic in an LTE network in accordance with 3GPP TS 23.203, several classes of service are provided, which are to varying degrees critical to quality parameters, such as network delay, percentage of IP packet loss, uneven network delays:

“QCI-7” (Voice, Video (Live Streaming)) - transmission of a streaming video image critical to delays and delay fluctuations;
“QCI-69” (Mission Critical delay sensitive) - transmission of critical information critical to packet loss;
"Normal" (Normal - NL) - transmission of diagnostic information.

Quality of service (QoS) parameters must be implemented in a virtual private network.
The quality of transmitted traffic is ensured by marking it with QoS labels.

The release of 3GPP TS 23.203 should provide the following features:

for QCI-7, the delay is not more than 100 ms and the probability of packet loss is not more than 10 ^ -3;
for QCI-69, the delay is not more than 60 ms and the probability of packet loss is not more than 10 ^ −6;
for Normal, the delay is not more than 300 ms and the probability of packet loss is not more than 10 ^ −2.

However, in any case, there is a limitation on the bandwidth of the radio channel and the task arises of finding a middle ground among the following parameters:

resolution of the transmitted image;
frame rate;
transmission delay.

When transmitting video, it is extremely important to minimize the delay and ensure a smooth display of the video stream to the operator-operator. However, smoothness is achieved by buffering, which leads to an increase in delay.

The delay itself is formed from the time the frame was captured by the camera, the video stream is compressed by the codec, the information is transmitted over the air, the video stream is decoded, and it is displayed on the monitor to the operator.

Reducing the amount of information transmitted is due to codecs (the most famous H.264, H.265), but encoding / decoding increases the overall data transfer delay.

A mandatory aspect of remote control is the measurement and control of video data delay, which is achieved by synchronizing the time on board due to the time stamp from the satellite navigator and synchronizing the time of the remote control using the PTP IEEE 1588v2 protocol. If the first threshold of the video transmission delay is exceeded, a warning is issued to the driver; when the second threshold is exceeded, the locomotive stops automatically.
An important question is also how many unmanned locomotives can be controlled by a single operator-operator? A preliminary assessment was made on the basis of the calculation of the probabilities of occurrence of certain emergency situations, as well as the time of the operator-operator necessary to eliminate them. At the same time, over time, an unmanned locomotive will be able to resolve more and more situations on its own, which will reduce the load on the operator-operator and increase the number of vehicles that he can control.

Perception of remote control technology by drivers

When remotely controlling a locomotive / electric train, the driver has less information. He does not feel vibrations, accelerations of the rolling stock, slopes, but focuses mainly on the video image. Moreover, the assessment of the range of the video image also differs from visibility from the locomotive. For understanding by the operator-operator of the distance, we applied a special marking on the image corresponding to the range. At the first stage, remote control is more difficult, but with experience come skills. In addition, it is important to understand that the remote control mode is intended only for emergency situations.

The use of remote control in other areas

The most widely used remote control is in the field of unmanned aerial vehicles. From land transport, remote control has been developed in mining enterprises, since often being in the work area poses a threat to life and health. The figures below show the remote controls for controlling equipment operating in quarries.

According to this link , you can become more familiar with the use of mining truck remote control.

In my opinion, the use of remote monitoring and control technology is also necessary for other unmanned vehicles being developed, such as agricultural machinery, taxis.

Conclusion

Of course, there are still a large number of both technical and regulatory issues that need to be addressed to improve this technology.

For example, to completely eliminate the delay in transmitting information, it is possible to predict the image (rebuild the image) several hundred milliseconds in advance based on the optical stream. There are ideas about transferring vibration of slopes to the operator’s operator’s chair, but because of the high cost of this approach, we have decided so far not to implement this solution. In the part of encoding / decoding a video stream, the possibility of using neural networks is being investigated.

One thing is certain, that in parallel with the development of unmanned technologies, communication technologies, remote control will also develop. It is possible that in the near future we will have centers for remote control and monitoring, where machine operators control the movement of electric trains throughout the network at a distance of thousands of kilometers. A similar picture will be observed in the mining industry, where operators will supervise the work of dump trucks, bulldozers and other equipment from cozy offices at a great distance.

I would also like to note that NIIAS JSC and Russian Railways JSC were the first in the world to develop technology for remote control of railway transport and, according to our estimates, we are currently at the leading positions in this field.