How to use CCTV cameras not only to monitor intruders

The practice of installing CCTV cameras in cities under the auspices of countering terrorism and preventing crime has spread globally and is gaining momentum. For example, at the beginning of 2019 in Moscow there were more than 167 thousand security cameras, although in China or the United Kingdom this figure could be considered ridiculous (for comparison, in London in 2018 there were 642 thousand, there are no up-to-date data on Beijing in the public domain, but for over 176 million cameras are now operating throughout China).

Such a number of lenses around us involuntarily evokes thoughts of 1984. It becomes especially alarming when you recall that the development of artificial intelligence has brought video surveillance to a new level. After all, deep learning allows AI to accurately distinguish one object from another in the image. In addition, artificial intelligence does not need to sleep, it is not distracted and does not miss anything.



For example, developed by Fujitsu GREENAGES Citywide Surveillance software allows you to simultaneously distinguish and track several types of objects on the camera image at once: transport, people and objects. The AI ​​used in it is able to take into account the time of appearance of objects in the frame, the number of objects of different types in the image, type, brand, model and color of transport, information on the license plate, recognize the faces and clothes of people, etc.

But not only surveillance of suspicious elements is limited to the use of video cameras. The ability of artificial intelligence to analyze images opens up new prospects for their use. Say for marketing research.

If, through the analysis of images from video cameras, a certain purchasing behavior is linked, for example, with any age group, then in the future it will be possible to more accurately satisfy the needs of customers. In addition, with the help of deep learning, it is possible to count the number of people in the image and track their movements. In retail, this will help to better understand which customers visit the store, to determine the routes of their movement inside the trading floor, which can be used to optimize retail space.

But the widespread use of AI for analyzing camera images is currently hindered by two main problems.


Fig. 1. The process of recognizing AI images from the camera

To begin with, let's look at the process of recognition of the image from the camera by artificial intelligence (Fig. 1). First, the camera captures the image, and then sends it to VMS (Video Management System, full-featured software for managing the video surveillance system), where the records are stored, they can be viewed, etc. After the video data is transferred to a computer equipped with a graphics processor capable of high-speed processing. At this stage, artificial intelligence is used to analyze images, the results are then displayed in the form of analytical data, and then visualized. In the future, as shown in the figure, the results of this analysis can be used directly in business applications.

In order for this scheme to work effectively, it is necessary to solve the following problems: provide sufficient data for training AI and make it possible to quickly process a large amount of data.

Let's start with the first one. The use of in-depth training, say, to analyze the image of a car allows the AI ​​not only to understand that the object is a machine, but also to determine the manufacturer and model of the vehicle. However, in order to create a more accurate model for training, a large amount of training data is required. Images of cars should be taken at different angles and in different lighting conditions. Thus, millions of images may be required to identify vehicles from different manufacturers and configurations.

Fujitsu solved the problem of getting enough data for AI training using simulation technology. Modeling is able to change the shape of the shadows in the image and other parameters, which ultimately increases the amount of data for training.

Fujitsu handles the massive amount of data that cameras transmit (especially if they shoot in high resolution) with the Fujitsu Technical Computing (TC) Cloud, a high-performance computing solution that uses GPUs for machine learning. GPUs are better for learning AI. Thanks to the architecture of its core, it is easier for them to cope with a large number of simple tasks of the same type and the process of learning AI on GPUs is much faster.

Even in solving the problem, boundary calculations can help to share the task of processing data between cameras and VMS, which ultimately reduces the load.

It is still important to ensure the continuous transfer of video data throughout the process of their collection and analysis. To do this, Fujitsu uses the best practices from the time when the company worked on systems for television broadcast stations.

For example, one of the solutions for exchanging real-time video (Real-time On-site Video Sharing) allows for secure video transmission even through not the most stable mobile data lines. In addition to real-time compression and video transmission, it has its own video transmission control technology, which ensures stable transmission even in conditions of poor reception and in low bandwidth conditions. Using this technology, for example, it is possible, in real time, to send images from cameras installed in vehicles, including from cameras with a shooting angle of 360 °, which transmit much more information.

So, where else can you use the data analysis of images from video cameras. In fig. 2 and fig. Figure 3 shows the potential uses of GREENAGES Citywide Surveillance at airports and shopping centers.


Fig. 2. Possible applications GREENAGES Citywide Surveillance at the airport in


Fig. 3. Possibilities of applying GREENAGES Citywide Surveillance in a shopping center

The results of the analysis of camera images are used in three main areas: first of all, ensuring security, then marketing research and, finally, improving the level of customer service.

As we already found out, using AI-based analysis, you can recognize car models and license plates. So you can track which car models and at what time they visit particular gas stations, determine their number, and also link data with the behavior of customers in shops and cafes at gas stations, which allows station operators to increase profits from the sale of related products.


Fig. 4. Analysis of images from cameras at a gas station and an adjacent cafe

In fig. 4 you can see an example of such an analysis. It was found that in the late evening the most cars visit the gas station. At the same time, a large number of cars did not affect the increase in cafe visits during refueling. According to the results of the study, it was proposed to diversify the dinner menu and additionally advertise the cafe itself for gas station visitors in order to increase attendance.

We can also use these technologies on food courts in shopping centers. For example, it is easy to teach artificial intelligence to distinguish a person who sits from a person who is standing (Fig. 5). Consequently, we are able to calculate how long visitors sit at tables, determine how many seats are occupied, etc.

As mobile ordering systems are spreading more and more on food courts (when a visitor makes an order in advance through his smartphone), in the event of queues, the AI ​​warns about this and the order application displays a notification for customers offering alternative options. Moreover, if the layout of the food court changes, the AI ​​will determine the new position of the seats automatically. The determination of peak congestion helps to think in advance of the optimal configuration of seats depending on the number of visitors in a given time period.


Fig. 5. Analysis of images from cameras on the food court of the shopping center

As we can see, the analysis of images from AI-based cameras has already reached the stage when it can be used not only to track suspicious persons or objects, but also for marketing research and projects. In the future, such an analysis will be used to solve a variety of problems, to make our life safer and more comfortable, or, for example, to help customers make purchases, optimize corporate governance practices, etc. Still, Big Brother is not so scary as we used to think about him.

Source: https://habr.com/ru/post/undefined/