Get best of the week

Using Flowmon Networks to Monitor the Performance of Distributed Applications and Databases



Article prepared by Dmitriy Andrichenko | Sales Executive, Russia & CIS | Flowmon Networks

Welcome to the page of our new article on solving the problems of monitoring the performance of distributed network applications and databases. This article is a continuation of a series of publications devoted to Flowmon Networks solutions and, in particular, a continuation of the review “ Network Monitoring and Detection of Abnormal Network Activity ” using signatureless technologies.
So, let’s start, but in the beginning we’ll say a few words about Flowmon Networks and the issue.

For those who are too lazy to read, a webinar on Flowmon Networks solutions will soon be held .

Flowmon Networks, as


Flowmon Networks is a European IT manufacturer highlighted in Gartner's squares and reports, specializing in the development of innovative solutions for network monitoring, information security, DDoS protection, as well as the topic of our today's article - monitoring the performance of network applications and databases.

The company is headquartered in Brno, Czech Republic. For the final customer, this has one key advantage - the ability to work with companies on the sanction list. Read more about Flowmon Networks here or here .

But what is innovative about Flowmon solutions, you ask? After all, none of the above areas is new in the market. Firewalls or intrusion detection systems have long and successfully existed, and the monitoring topic is not new in itself. Everything is true, but, as usual, "the devil is in the details."

Consider, for example, the topic of network information security. What comes to mind first? Firewall or perhaps IDS / IPS? Maybe even NG Firewall. That's right, this is a good proven classic, but which has two significant drawbacks:

  • limited signature approach to identifying security issues,
  • point protection only at the boundary level of network segments.

We are talking about the application of heuristic analysis technology and machine learning. Artificial Intelligence, in other words. The advantages are obvious - there are no fixed signatures that protect against zero-day attacks only if they are updated and relevant.
A signatureless analysis allows you to record atypical application-level attacks, protocol format deviations from the RFC and many other problems that cause a lot of headache for administrators every day.

In addition, the second main advantage is not just the point-by-point traffic control at the “junction” of segments or perimeter, solved by standard means of protection, but the total control and “transparency” of each network connection in the network.

We do not propose replacing existing defenses, we say that in a modern world with constantly evolving technologies and potential threats, a standard set of defenses is no longer enough. We wrote about this earlier, right here.

A similar situation is with the tasks of monitoring the functioning and performance of network applications, as well as databases. I believe that everyone is familiar with a situation where users complain about the functioning of a business application, but the problem is not solved. Network administrators claim that everything is in order with the LAN and refer to problems in the application itself. Application administrators check the server, event logs, DBMS and it turns out that everything works for them too. As a result, the problem is not diagnosed, at all levels “everything is in order”, administrators “nod” at each other, and nothing works for the end user. What to do is not clear. It happened? That's what we’ll talk about today.

Solution Architecture


To correctly understand the approaches and technologies used by Flowmon Networks to solve the problems of monitoring the performance of distributed applications and databases, it should be noted that the entire analysis is based on information about network traffic that is sent to the system. One of the advantages of this approach is the lack of agent software on workstations and servers . Of course, you won’t be able to measure the performance of the Solitaire Solitaire, but it’s quite possible to identify the SQL query that “hung” the database, or the button after which the application hangs.

In the last articleWe have already examined the Flowmon Networks product portfolio and the process of installing the system on the VMware EXSi virtual environment, so we won’t repeat it. The only difference in our case will be the method of receiving traffic. Since none of the Flow protocols transmits information about the contents of the packets that we need to analyze the functioning of level 7 protocols using the ISO OSI model, we will use the mirrored SPAN (Switched Port Analyzer) port on the switch to collect data.

In this case, the solution architecture will look something like this:



The switch (s) mirrors the required traffic to a dedicated server (Flowmon Probe), which is responsible for processing it and converting it to IPFIX-rich format, which is then transmitted to the central node (Flowmon Collector) for storage, correlation and analysis. Instead of the SPAN port, by the way, you can use a TAP traffic splitter:



The advantages of this deployment option are:

  • independence from the model and manufacturer of network equipment (Cisco, Juniper, any),
  • lack of additional load on existing network equipment,
  • preservation of the existing logical architecture of the company’s network.

In fact, each component of the system can be either a dedicated hardware server or a virtual machine. In the second case, the Flowmon Collector will include an integrated Flowmon Probe, but the performance will naturally be lower.

The central node (Flowmon Collector) is built on the principle of modular architecture and is configured individually for each Customer’s tasks:



Flowmon Collector consists of a system core (Network Visibility Troubleshooting), which includes all the functionality required by network administrators to monitor traffic in the LAN with details for each specific network connection, as well as a number of additional and separately licensed modules:

  • Anomaly Detection Security (ADS) module - detection of abnormal network activity, including zero-day attacks, based on heuristic analysis of traffic and a typical network profile;
  • Application Performance Monitoring (APM) module - monitoring the performance of network applications without installing “agents” and affecting target systems;
  • Traffic Recorder module (TR) - recording network traffic fragments by a set of predefined rules or by a trigger from the ADS module, for further troubleshooting and / or investigation of information security incidents;
  • DDoS Protection (DDoS) module - protection of the network perimeter from volumetric DoS / DDoS denial of service attacks.

In this article, we will look at how everything works live using the example of 2 modules - Network Visibility Troubleshooting and Application Performance Monitoring.

Solution Installation


We already wrote on the topic of deploying a virtual machine ; everything is done quite quickly and simply from the OVF template. We will not repeat ourselves, we recall only the requirements for system resources:



On the Flowmon Collector side, the key difference between monitoring SPAN traffic from NetFlow monitoring will be the method of receiving data. If we previously used the Management interface for NetFlow with our IP configuration, then to receive SPAN traffic, we need the Monitoring Interface, which is actually an L2 interface, associated with the hypervisor with a dedicated physical port on the server chassis.



In other words, the Monitoring Interface is the Flowmon Probe built into the Flowmon Collector.

The next step is to verify that the dedicated port is correctly configured and ready to receive traffic on the Flowmon Collector.



In our case, the UDP / 2055 port is occupied under IPFIX / NetFlow from the network equipment, so for traffic with Flowmon Probe we will take the UDP / 3000 port. Separating traffic by port from different sources is not necessary, but it is more convenient and simpler in terms of monitoring and Troubleshoot.

Next, we configure the export of traffic from Flowmon Probe to Flowmon Collector. To do this, in the Configuration Center -> Monitoring Ports section, check the current settings. Mainly you need to make sure that monitoring of required applications of level 7 ISO OSI is enabled, because by default it is turned off.



Ideally, include only those protocols that you want to control, but you can simply turn on everything.

We save the settings and again go to the main screen of the Configuration Center, you need to make sure that the traffic from Flowmon Probe is correctly sent to the Flowmon Collector.



Also check in the Flowmon Monitoring Center -> Sources section.



We see that traffic began to flow, the system is working. You can go directly to configuring the Application Performance Monitoring (APM) module.

Application Performance Monitoring Module (APM)


We will deal with what exactly and exactly how we will control. What parameters does Flowmon APM control?

  • analysis of problematic HTTP and SQL queries, application server and database response error codes,
  • delays and packet losses that occur during client-server interaction, as well as in the interaction of application servers with each other and with database servers,
  • information on each transaction (number, size, time, IP address, session ID, username ...), as well as a list of problematic transactions with SLA violations,
  • application response time (max, min, average, percentage ...) and transmission time at the transport level,
  • number of simultaneous user sessions, ...



What protocols does Flowmon APM support?

  • HTTP 1.1, HTTP 2.0, SSL and TLS,
  • SQL (including MSSQL, Oracle, PostgreSQL, MySQL, MariaDB),
  • Email (including SMTP, IMAP, POP3),
  • VoIP SIP,
  • DHCP, DNS, SMB (including v1, v2, v3), AS, NBAR2,
  • SCADA / IoT (including IEC 60870-5-104).

As a result, for each monitored application or database, the system calculates the value of the APM Index metric, which varies from 0 to 100 and depends on the current state of the service. The higher the metric value, the better.



A customizable interface based on widgets and dashboards allows the administrator to customize the system individually for himself and control exactly those APM Index metrics that he needs. In the example below, the system controls the Internet portal (WebEshop) and its database (MySQL_DB).



In this example, performance analytics is divided into three blocks:

1. The overall performance of the application and database over the last day.



For completeness, the application performance index is displayed along with the database performance index. This is convenient enough for troubleshooting and understanding on which specific area the problem arises.

For example, in our case, the database performance index is fine, it is 96.839 out of 100. But there are obvious problems with the WebEshop application, its index is only 63.761 out of 100.

You can immediately notice the reason for this rating - a high response time to user requests. The average time is 21.148 seconds and the maximum is as much as 151.797 seconds. If you are an administrator of an online application, then you understand that few users will wait until the page loads 2.5 minutes ... Well, if this happens once, and if the user needs to go 2-3-4 ... pages? This is already a problem.

2. APM index for the last day.

With this section, everything is quite simple and clear. It displays the ratio of the number of queries from the total APM index of the application or database.



Each element of the dashboard is interactive and clickable. Everything obeys the principle of drill-down, when choosing an interesting area on the chart, you can “fail” one level below to get more detailed information.



Choosing the time interval when the problem was fixed, the administrator will quickly find answers to the questions:

  • What SQL queries were executed at this moment?
  • What and how many users worked with the system?
  • How did the system respond to user requests?
  • What was the reaction time and system delay?
  • How do application problems correlate with database interaction?
  • How does the system work with a given SLA?
  • and much more…

3. The five slowest queries in the last day.

A modern HTTP portal or WEB application is a complex and complex program. Like any other application, it consists of different pages and modules that were not always written by one programmer. Very often, a modern site is a CMS engine on which dozens of third-party modules are installed that extend the basic functionality. Sometimes these modules work well, and sometimes not very well. It is not always possible to quickly understand where the problem occurs and it takes more than an hour or a day to troubleshoot.

With Flowmon APM, everything becomes transparent.



If you are interested in more detail - click on the icon "magnifier" and get the details. For example, for an HTTP application:



Or for a database:



Of course, everything is exported to CSV, fields and columns are customizable, filters can be saved.

The widgets discussed are an example of standard default settings. If necessary, the system can be customized for individual tasks - create your own dashboards and display them on the main screen. As an example, database response error codes:



Or HTTP error codes:



Also, we want to draw your attention to an important point - the proactive monitoring functionality. The system not only “listens” and analyzes traffic in a passive mode, but also independently emulates the interaction of a “virtual” user with the system. This approach is called Synthetic Users and allows you to automatically check the status of the application and detect a problem at the moment when it just starts to happen, and not after the first complaints of users. For this, for example, scheduled scripts are configured to check application availability, functionality and response time.

What is the result?


This example is a clear demonstration of the capabilities of the system and the Application Performance Monitoring (APM) module, in particular. I can’t say that working with Flowmon APM makes the trouble shooting process a pleasure, but it’s for sure that this process is simplified and much faster.

Have questions or want to test the system? We will help, contact us .

We summarize what conclusions we can draw about Flowmon in the bottom line:

  • Flowmon - premium level solution for corporate Customers;
  • due to its versatility and compatibility, data collection is available from any source: network equipment (Cisco, Juniper, HPE, Huawei ...) or proprietary probes (Flowmon Probe);
  • , ;
  • ;
  • «» – ;
  • Flowmon – , 100 /;
  • Flowmon – ;
  • / .

We also want to invite you to our webinar dedicated to the solutions of the Flowmon Networks vendor . For pre-registration, please register here.

That's all for now, thanks for your interest!

More articles:


All Articles