Get best of the week

Smoking Detection Complex by photo or video based on Intel NUC

Intel NUC8i5BEK

In this post, we will talk about how we solved the problem of determining the fact of smoking through object video analytics on Intel NUC. At the input, video streams from surveillance cameras that are decoded are cut into frames on the computer, and then each frame (taking into account the frame rate divider) is given to a neural network that detects the presence of a fact of smoking and returns the probability of an event.

Now let us consider in more detail the problems, the main differences from the solutions existing on the market, as well as the obtained indicators for speed and number of threads.

General issues


In Russia, on June 1, 2014, new norms of the Anti-Tobacco Law came into force. The law against tobacco regulates relations arising in the field of protecting the health of citizens from the effects of surrounding tobacco smoke and the consequences of tobacco consumption: smoking is prohibited on long-distance trains, on passenger platforms, in hostels and hotels, retail premises, markets, cafes, bars, restaurants .

In order to combat smoking, many states have introduced a law banning smoking in public places. In all offices and theaters, “smoking rooms” were eliminated, and smoking rooms were removed in catering establishments.

The introduction of the prohibitions described above involves monitoring compliance with established rules and regulations. To date, a variety of dust sensors and gas analyzers (e.g. CO2) are used for this purpose. The general principle of operation of these devices is as follows:

The general principle of operation of various dust collection sensors and gas analyzers

The sensor detects changes in the environment, and the control microcontroller creates a reaction event according to a predetermined algorithm.

An alternative to dust collection sensors and gas analyzers can be object video analytics using neural networks, where the input is a photo or video stream from a surveillance camera, and the output is the probability of the presence of tobacco smoking or other compositions in the frame or set of frames.

General block diagram of a hardware-software complex


There are several options for implementing the complex:

  1. Separate system in compact design for installation on site
  2. Centralized system with data transmission and processing in the data center with the ability to use existing video surveillance systems
  3. A hybrid option, when part of the data is processed in the immediate vicinity of the data source, and part is processed in the data center with centralized storage of the results of both systems

Let's consider them in more detail:

General block diagram of a hardware-software complex for smoking detection

Composition of the complex when used in the immediate vicinity of the data source:

  • IP camera / camera with direct connection or a set of cameras (used as a data source).
  • Switch (when connecting more than one data source).
  • Executive device, Intel NUC8i5BEK calculator.

With the low cost of the hardware and software complex, a lot of significant and significant security tasks are solved, such as:

  • Monitoring compliance with fire safety rules with high accuracy and with photo-recording of the fact of the offense (including data on the time, date, place of the offense)
  • Identification of facts of offenses at hazardous industries and companies whose activities are associated with the use of flammable and fuels and lubricants
  • Monitoring compliance with internal regime at sensitive facilities

A valid use case is a server architecture, in which data from cameras is transmitted to the data center for further processing:

Server architecture, in which data from cameras are transmitted to the data center for further processing

When scaling and using this scheme, as a device for centralized inference, it is assumed to use the same Intel NUC8i5BEK, but in a different form factor (server 1U) :

Server for executing neural networks based on 8 Intel NUC8i5BEK

Description of the principle of detection of the fact of smoking


To detect the fact of smoking in the photo (frames of the incoming video stream), the neural network topology SSD Mobilenet v2 from the Open Model Zoo is used. The network is pre-trained on the COCO dataset and further trained on Tensorflow. Next, the model is converted through Intel OpenVINO for further operation on the CPU / GPU in order to optimize the cost of FPS. Model performance after conversion:



Total on a single Intel NUC8i5BEK with a frame divider value of 5 (25 FPS / 5 = 5 FPS at the input), up to 40 streams can be processed without taking into account the cost of decoding. When using VAAPI hardware decoding and the latest intel-media-driver, decoding costs will be minimal.

One of the advantages of the Intel OpenVINO framework is the ability to transfer networks between different devices, for example, the same model with minimal modification can be run on CPU, GPU, FPGA, VPU and other devices.

For the sake of the experiment, a model for detecting the fact of smoking was launched on the Intel Neural Compute Stick 2 based on Myriad X. Results:



Smoking Detection Launched on Intel Neural Compute Stick 2 Based on Myriad X

Based on industrial PCs with motherboards from AAEON or other manufacturers with integrated MyriadX chips, you can already get and use industrial solutions.



To demonstrate the operation of the neural network, the Telegram bot was implemented - https://t.me/smokers_recognition_bot . The input is the image, and the output is the probability of the fact of smoking on it. We try, watch, experiment ...

The interference is performed on the Intel NUC8i5BEK GPU.



Solution Benefits


The following advantages can be noted:

  • Availability of processing data from multiple sources in one place
  • Possibility of detecting the fact of smoking at a distance limited only by the focal length of the camera, data source, for example, 5, 50 or 100 m (such indicators cannot be obtained with classical sensors and / or devices)
  • Possibility of detecting smoking not only classic cigarettes, but also other devices (for example, vapes or smoking mixtures)
  • The ability to save the fact of the offense (photo and event metadata, such as date, time, location) when smoking in the wrong places
  • Possibility of retrofitting existing chambers with a function of detecting the fact of smoking and reactions to this event
  • Availability of integration with existing monitoring systems and video surveillance systems, for example, Zabbix, Telegraf, Hikvision NVR, etc.

Application Areas


Consider some objects and problems for the application of the described hardware-software complex for smoking detection in the video stream:

  • Corridors of business centers and other buildings and structures, stairwells
  • Schools and kindergartens (due to the inefficiency of smoke detectors and other existing solutions in open space and in blown zones)
  • Gas station (due to the inefficiency of smoke sensors and other existing solutions in the open space and in the blown zones)
  • Metro (due to the large area, ceiling heights and the ability to connect multiple cameras into a single system)
  • (- , )
  • , (- , , )
  • ( )
  • , ( )
  • ( )

,



More articles:


All Articles