Video conferencing is now a market and new technologies. Longrid, part two

We publish the second part of the review about the video conferencing market. What developments have appeared over the past year, how they penetrate our lives and become familiar.Above is a screenshot of the SRI International video, it can be viewed closer to the end of the article.

Part 1 :
- VKS market - world -wide
- Hardware vs video communications software
- Huddle rooms - aquariums
- Who will: mergers and acquisitions
- Not a single video
- Competition or integration?
- Compression and data transfer

Part 2:
- Smart conferences
- Unusual cases. Robot control and policing

Smart conferences

The field of videoconferencing is quite mobile in terms of introducing new technologies, many developments appear every year. Substantially expand the possibilities of machine learning and artificial intelligence.

The closest to reality and in demand was speech-to-text technology. The machine recognizes a clear articulate speech quite successfully, but it is not very live with voice analysis yet. However, video communication simplifies the procedure by sequential replicas on different channels, and many vendors have already announced services based on speech recognition.

In addition to live captioning, which is convenient for people who are hard of hearing or in public places, businesses also need meeting management tools. Tons of videos are inconvenient to revise, someone has to keep a protocol, record agreements, turn them into plans. So far, a person helps to mark and sort the decrypted text, but this is already much more convenient than writing it down in a notebook. If necessary, post factum search in decrypted texts and created labels is much easier. Integration with planners and various project management services significantly increases the effectiveness of video communications. For example, Microsoft, BlueJeans are working in this direction. Cisco bought Voicea for this purpose.

Of the popular features, it is worth noting the replacement of the background. Behind the speaker you can place any image. This opportunity appeared at various manufacturers, including the Russian TrueConf, for a long time. Previously, a chromakey (green banner or wall) behind the speaker was needed for its implementation. Now there are already solutions that can do without it - for example, Zoom. Literally on the eve of the release of the material, the replacement of the background was announced in Microsoft Teams.

Microsoft also knows how to make people transparent. In August 2019, the Intelligent Capture feature appeared in Teams Rooms. In addition to the main camera, which is intended for shooting people, an additional content camera is also used, the task of which is to broadcast the image of a regular marker board on which the speaker can write or draw something. If the speaker is carried away and blocks the writing, the system will make it translucent and restore the image from the content camera.


Intelligent Capture, Microsoft

Agora has made an emotion recognition algorithm. The system based on a cloud server processes video data, selects faces on them and informs the user about what emotions the interlocutor is now demonstrating. With an indication of the degree of accuracy of the determination. So far, the solution works only for one-on-one communication, but in the future it is planned to implement this for multi-user conferences. The product is based on deep learning, in particular, the Keras and TensorFlow libraries are used.


Agora Emotion Recognition

A fundamentally new field of application for video conferencing systems was opened by technology that understands sign language. The GnoSys app was created by Evalk from the Netherlands. The service recognizes all popular sign languages. It is enough to put the phone or tablet in front of you during a video call session or regular conversation. GnoSys will translate from sign language and play your speech to the person you’re talking to, opposite or on the other side of the screen. Evalk development information appeared in February 2019. Then the partner of the project was the Indian Association of the hearing impaired - National Deaf Association. Thanks to her assistance, developers gained access to a huge array of data on sign languages, dialects and nuances of use, and active testing was underway in India.

Now the issue of leakage of confidential information from negotiations is becoming very relevant. Zoom in early 2019 announced the introduction of an ultrasound signature. Each video is equipped with a special ultrasound code that allows you to track the source of information leakage in the event that the recording goes to the Internet.

Virtual and augmented reality also penetrate the field of video conferencing. Microsoft suggests using the new HoloLens 2 glasses with its Teams cloud collaboration service.


HoloLens 2, Microsoft

Belgian startup Mimesys went even further. The company has developed a virtual presence technology that allows you to create a human model (avatar) and place it in a common workspace, which can be monitored using virtual reality glasses. Mimesys was acquired by Magic Leap, a world-famous manufacturer of VR glasses. Industry experts firmly link the prospects for the development of virtual and augmented reality technologies with the development of 5G mobile networks, since only they can provide the necessary speed and reliability so that such services become available to a wide range of customers.


Collaboration on a project in virtual reality, photo Mimesys

Unusual cases. Robot control and policing

In conclusion, a little about how the scope of video communication is expanding. The most obvious is the remote control of mechanisms in the danger zone and uncomfortable environments, saving people from dangerous or routine work. In the news field over the past year, management topics have been encountered, for example, telepresence robots in space , housework robots , BELAZ in a coal mine . Solutions for the penitentiary and law enforcement systems are being developed.

So recently there was information about a new development of the SRI International research institute (USA), where the problem of police safety is quite acute. According to statistics, about 4.5 thousand attacks are carried out by aggressive drivers on guard of law and order every year. About one in one hundred of these cases ends in the death of a policeman.

Development is an integrated system that is mounted on a patrol car. It is equipped with high-resolution cameras, a display, speakers, and microphones. There is also a breathalyzer, a scanner for authenticating documents and a printer for issuing fines. Since the complex monitor is touch-sensitive, it can be used to carry out special tests to assess the general condition and adequacy of the driver. When the police crew stops the intruder, the device extends towards the vehicle being tested and blocks its movement until all test procedures are completed using a special studded bar at the wheel level. The system is already undergoing final tests.

Robotic car inspection system, SRI International

Prison has become another medium for the use of VKS. Several U.S. prisons in Missouri, Indiana, and Mississippi have replaced ordinary prisoners with ordinary short dates for communication through a video terminal.


Communication through the videoconferencing terminal in one of the US prisons, photo by Natasha Haverty, nhpr.org

Prisons in this way not only increase safety, but also reduce costs. Indeed, in order to deliver a prisoner to the meeting room and back, it is necessary to provide a whole range of security measures along the route and during communication. Since visits to US prisons are allowed once a week, for large facilities with a large contingent, this process is provided almost continuously. If you replace personal dates with video, then there will be less potential problems, and the staff of escorts can be reduced.

Human rights activists and prisoners themselves say that in the current version, the video communication system significantly loses to personal communication and is in no way its equivalent, even despite the increased talk time. Relatives do not have to go to jail, communication can be done from home, but in this case the cost of communication is much more expensive - from a few tens of cents to ten US dollars per minute, depending on the region. Communicate through local terminals in the prison for free.

Prisons, which have tried to introduce similar communication systems, are very pleased with the result and do not plan to abandon this practice. Independent sources note that the administration may be interested in introducing the technology because of the commission from the operators of the videoconferencing system, who install their decisions there. In all cases, we are talking about special closed systems, the quality of which, according to American journalists, is inferior to popular services like Skype.

---

The videoconferencing market will continue to grow. This is especially evident now, in the midst of an epidemic. Access to the clouds opened up opportunities that have not yet been fully realized, and new technologies are already approaching. Videoconferencing is getting smarter, integrating into the common business space and continuing to improve.

We thank Igor Kirillov for the preparation of the material and the editors of B + K for its updating.