Get best of the week

The secretary robot, bot informant, bot therapist and other projects of the online hackathon VirusHack

More than 70 teams and over 300 people from 64 cities took part in the work on the Megapolis Moscow track, which was held as part of one of the country's largest online hackathons, VirusHack.

image

The event was organized by the Moscow Innovation Agency and Rostelecom in partnership with the Russian Association of Electronic Communications (RAEC).

As a result of the coordinated and hard work of the teams of programmers and experts, five of the best innovative projects were selected: a bot informant for the ICQ New messenger, a bot therapist, a robot secretary for negotiations, an automated cash register with the function of voiceless contactless payment for purchases, and also a service to help people with disabilities in accessing audiovisual content.

Messenger bot-messenger ICQ New


Best of all, the EGD BAG team handled the task from ICQ New (Mail.ru Group). Its participants developed an information bot for the ICQ New messenger, which will inform users about the spread of coronavirus infection.

According to the geotag, people can receive information about new and old cases of infection of citizens, find out the addresses of the nearest medical institutions and laboratories for taking tests on COVID-19 and the addresses of the nearest pharmacies and shops. Also in the bot is a simplified SMS-message generator for receiving an electronic pass.

To adjust the computational flows when writing the bot, standard Java language tools were used. To greatly simplify the work of the bot, an API library from ICQ was chosen. The task of simplifying the deployment of the bot in the production environment was also solved: knowing that the standard in corporate development is now Docker, the EGD BAG team prepared a Docker image.

The result is a product that is easy to upgrade and scalable. The bot’s functionality will be relevant even after a pandemic: the search capabilities for laboratories are easily replaced by the search for fitness centers, restaurants and other urban facilities.

The bot is available now at: github.com/airaketa/egdbag-bot

Contactless payment


The Buckwheat42 team was better than the others in the task of X5 Retail Group in developing additional voice input functions for contactless payment for purchases in Pyaterochka supermarkets.

The project was developed in Python. The prototype is based on open-source technology for translating speech into text (speech-to-text) and a module for processing and analyzing the received text (Natural Language Understanding). Of the available libraries for converting voice to text, kaldi was chosen, since it works quickly and provides a relatively high recognition quality not only of Russian, but also of a number of other languages.

According to the conditions of the task, to facilitate the integration of the solution into the current cash desk software, the buyer's voice requests are transmitted to messages based on the current cash desk API. For ease of deployment and testing, the prototype was assembled using Docker technology.
For each operation, this module identifies the intentions (intentions) of the user, retrieves the pronounced names of the products, as well as barcodes, loyalty card numbers, coupons and other related information.

The function works without access to the Internet or external voice conversion services.

Negotiating Robot


Import this team has successfully completed the task of creating a secretary robot from SberCloud. This program is combined with any services for online conversations, including the well-known Zoom, Skype and Teams, and is also suitable for offline conversations.

The secretary robot can recognize speech, distinguish the voices of interlocutors, guess the mood of a person by the tonality of the spoken words. He is able to notice when people plan meetings and calls, and save these tasks in a calendar or CRM system. The secretary robot translates human speech into text messages, and after negotiations saves these messages in a doc file.

The Import this development team has done the technical part of the secretary robot (backend) in the Python programming language, and the external part (frontend) on the Vue.js and Electron software platforms.

For recognition of text and voices, the Google Cloud Speech API was temporarily used, but in the future it will be replaced by a more suitable model. To analyze the emotions of the interlocutor through the tonality of the text, the Dostoevsky library was used. To distinguish from the text the arrangements for calls and meetings, the NLTK natural language processing library was used.

Access to audiovisual content for people with disabilities


Uma.Tech company directed the socially significant task “Equal Opportunities”, where the participating teams were offered to develop a solution that simplifies for people with disabilities to interact with the PREMIER platform and with the MATCH PREMIER website or perceive the content posted on them.

As part of the task, the teams came up with ways to simplify access to sites and media for people with hearing, vision, fine motor and concentration problems. The complexity of the hackathon for the teams consisted in the problem of the exact statement of the problem - the spectrum of possible options is too wide - and in the traditional limited time for development. Therefore, as part of the hackathon, the jury accepted both valid prototypes and well-developed concepts.

The winner was the XGBeasts team, which developed a neurocomputer interface for interacting with web pages. The principle of the service is to process data coming from the neural interface located on the user's head. This data is a description of a person’s intentions to press a specific button on a computer screen. Using artificial intelligence algorithms, this data is recognized, and then the program presses the button that the user wants to click.

Therapist chatbot


Better than the others, the task of Mobile Medical Technologies LLC to create a bot therapist was performed by the Dev labs team. Its programmers created three solutions at once:

  • a chat bot that can quickly determine the likelihood of a COVID-19 disease and give a recommendation (contact a doctor or call an ambulance);
  • an online assistant who will give the necessary information or reassure the patient at the right time;
  • a service that will calculate the most accurate epidemiological situation within a radius of 500 meters from a person.

To implement the Web API, the development stack of React, Redux, Typescript was chosen. The technical part was implemented in Python Flask. The classification of voice intents is implemented using the DeepPavlov deep learning library. To implement the functionality of answering frequently asked questions about COVID-19, AzureQnAMaker was deployed. Using TelegramBot, a convenient service for monitoring the spread of a pandemic has been implemented.

The Moscow Innovation Agency said that in the near future, the services developed within the Megapolis Moscow track will be finalized with the partners of the tracks and piloted in their infrastructure.

More articles:


All Articles