👨🏽‍🚒 👌🏾 🤔 Integration of Netflow and SIEM Monitoring Solutions 🕜 🍘 🍮

SIEMs have long become a de facto standard in analyzing security events and detecting incidents (although there is some movement towards abandoning SIEM and replacing them with solutions for managing logs with an add-on from machine learning technologies), but the effectiveness of this solution depends on what data sources it works with. Still, usually SIEM specialists primarily work with logs, leaving aside such an important source of information as Netflow, which allows you to see something that often does not get into the logs or gets, but too late. This raises a number of questions. Why does a modern SIEM solution need Netflow support? What can SIEM get from Netflow analysis? Which option to integrate SIEM with Netflow if there are manufacturers,who embed Netflow support directly into their solutions, and there are those who prefer to work with various Netflow collectors. What are the features of working with SIEM Netflow? We’ll talk about this.

Netflow for threat detection

In a previous articleI have already described the possibilities of using Netflow for cybersecurity. Let me remind you that, unlike the logs of network equipment or raw traffic, Netflow is a protocol that allows you to analyze traffic by metadata collected from network sessions. These are not only addresses and destination and source ports, but also message types and codes for ICMP, IP service types, IP protocol type, network interfaces, session duration, start and end times, etc. If the logs are usually generated on terminal devices (for example, servers, workstations and DBMS) or means of protection, then what to do in a situation where the former do not generate security events or you cannot put security means on them,and the latter are installed only on the perimeter and do not see what is happening inside the corporate or departmental network? Another thing is network equipment - switches and routers (hardware or virtual), wireless access points and controllers. They are installed inside and the interaction of users, devices, applications always passes through them. It is impossible to pass them! By transmitting data about this interaction to Netflow (even Cisco ASA or Firepower generates it), we get a valuable source of information for not only IT, but also cybersecurity. Where equipment does not understand the flow protocol, we can use special exporters, software or hardware solutions that generate flow records for the traffic passed through them (for Cisco, the Netflow Generation Appliance orwhich generate stream records for the traffic passed through them (with Cisco, the Netflow Generation Appliance orwhich generate stream records for the traffic passed through them (with Cisco, the Netflow Generation Appliance orwireless access points and controllers. They are installed inside and the interaction of users, devices, applications always passes through them. It is impossible to pass them! By transmitting data about this interaction to Netflow (even Cisco ASA or Firepower generates it), we get a valuable source of information for not only IT, but also cybersecurity. Where equipment does not understand the flow protocol, we can use special exporters, software or hardware solutions that generate flow records for the traffic passed through them (for Cisco, the Netflow Generation Appliance orwireless access points and controllers. They are installed inside and the interaction of users, devices, applications always passes through them. It is impossible to pass them! By transmitting data about this interaction to Netflow (even Cisco ASA or Firepower generates it), we get a valuable source of information for not only IT, but also cybersecurity. Where equipment does not understand the flow protocol, we can use special exporters, software or hardware solutions that generate stream records for the traffic passed through them (for Cisco, the Netflow Generation Appliance orBy transmitting data about this interaction to Netflow (even Cisco ASA or Firepower generates it), we get a valuable source of information for not only IT, but also cybersecurity. Where equipment does not understand the flow protocol, we can use special exporters, software or hardware solutions that generate flow records for the traffic passed through them (for Cisco, the Netflow Generation Appliance orBy transmitting data about this interaction to Netflow (even Cisco ASA or Firepower generates it), we get a valuable source of information for not only IT, but also cybersecurity. Where equipment does not understand the flow protocol, we can use special exporters, software or hardware solutions that generate flow records for the traffic passed through them (for Cisco, the Netflow Generation Appliance orwhich generate stream records for the traffic passed through them (with Cisco, the Netflow Generation Appliance orwhich generate stream records for the traffic passed through them (with Cisco, the Netflow Generation Appliance orStealthwatch Flow Sensor ). Algorithms for detecting anomalies or signatures superimposed on Netflow streams, as well as machine learning methods, allow you to identify deviations from the reference behavior of network traffic, which will characterize information security threats or problems with the IT infrastructure.

It is worth remembering that Netflow cannot say what is transmitted inside network traffic (although there are already technologies that allow you to recognize the applications used and malicious code by Netflow), it was developed for other tasks. But he can tell who “spoke” and with whom, how and for how long. Despite the existence of different versions of flow protocols, most of them allow you to collect the following data:

Source and Destination IP Address
source and destination ports
protocol
type of service
source interface
start and end timestamps
the amount of information transmitted.

In some versions, for example, in IPFIX or Netflow v9, you can extract some of the information from the data body, which allows the Netflow analyzer (stand-alone or built-in to SIEM) to identify running applications. All this information is often enough to detect anomalies or even threats in network traffic.

Netflow and SIEM: what is the strength, brother?

What if the target system does not generate logs or they are disabled by intruders? What to do when there are simply no security features on the target system, because they load the processor and slow down the system? What to do when an employee brings his personal device that is not wired to SIEM? We still have the only source of information - network traffic, which in any case is generated by the target device. What can help identify a “clean” Netflow that can be analyzed without SIEM using Network Traffic Analysis (NTA) solutions? Here is a short list of such threats and anomalies:

malicious code, ransomware and crypto miners
interaction with command servers
network scan
denial of service attacks
data leakage
hacking ssh or rdp
torrents and other P2P applications
,
..

And what do the logs that your SIEM collects give you? The ability to monitor user activity on nodes, access to resources, entry and exit to and from systems, alarms from network equipment and security tools, and much more. The combination of logs and Netflow coming to SIEM and analyzed there allows you to quickly identify new sessions and new traffic from devices that have just been attacked, including those that bypass your perimeter. By combining these data types, you can also identify incorrectly configured network security features. Netflow complements the traditional SIEM capabilities with the ability to detect new types of traffic, its bursts, interaction with external and internal resources, data leaks, malicious code distribution, interaction with team servers,encapsulation of malicious load into allowed protocols, etc.

The combination of traditionally processed SIEM data with Netflow allows not only to see more various security events, but also to see them faster, reducing the so-called TTD (Time-to-Detect) parameter and, as a result, response time. It is clear that SIEM and Netflow analysis can work independently of each other and do their job well, but it is the combination of these two solutions that allows you to achieve a synergistic effect.

Changing the pattern of traffic behavior may indicate suspicious or abnormal behavior. For example, exceeding the threshold for data uploaded to the Internet may indicate a leak. A change in the size of DNS queries and responses compared with the RFC provided can characterize the fact of the operation of malicious code (according to Cisco statistics, 92% of malware uses DNS to receive commands, download data, or download updates). Remember the story with Equifax? Then the attackers were able to access the Web-portal, and then to the internal database servers, slowly uploading large amounts of data to the outside. Separately, all these events were not of great interest and only gathered together and enriched with Netflow data, they allowed to identify information security incidents. This is another case for Netflow analysis systems,which can detect non-standard behavior of both the Web portal from which the query to the database is frequented, and databases that suddenly give a lot of data to the portal. I'm alreadyDescribed this scenario on Habr earlier. SIEM, receiving Netflow data, can get an additional context for events generated by IPS, ITU, proxies, antiviruses, EDR, etc., in order to more effectively respond to the incident.

Another example. Detection of unknown attacks for which intrusion detection systems do not have signatures - due to the correlation of network anomalies and events from hosts, as well as triggering of triggers. For example, scanning 100 nodes in 3 minutes can characterize the operation of malicious code that expands its bridgehead. An increase in the response time from the server can characterize a DDoS attack against it, and, for example, a mismatch in the amount of information received and sent via the DNS protocol can indicate a data leak. This is a case we already looked at earlier when I talked about finding a DNSpionage campaign. At the same time, you can note that in one case we analyzed only Netflow, and in the other , we also collected data from terminal devices, sandboxes, a DNS gateway, etc.

Another area where integrating Netflow into SIEM can help is situational awareness. Correlation of flows with logs, context, geolocation information, user activity, reputation data. For example, in a previous note, I cited a case with the detection of internal traffic with nodes from Iran or North Korea. If you do not have contacts with these countries, then perhaps this is a sign of malicious code stealing information, or pro-government hackers targeting your company. A network anomaly analysis system can do some of this work by identifying the corresponding anomaly. But if you also want to understand which user was involved in this incident, you will need additional context in the form of user names from Active Directory.Netflow itself does not operate on this information - so we need to enrich it with information from AD. If you useCisco Stealthwatch , for this you just need to integrate it with Cisco ISE , which will link IP and MAC addresses with specific users and device profiles. What if Cisco ISE isn’t deployed on the network? The task of enriching Netflow with user information is the responsibility of SIEM.

Another example. I have a Web site running on my computer that is used for a number of tasks. But how many ordinary users can boast the same? Suddenly you capture traffic inherent in the operation of a Web server from a user computer? Or in the segment where users working on a Windows PC are located, you suddenly see the traffic inherent in the Linux OS. Perhaps this “handy” user decided to raise a virtual machine with Linux, thereby violating the security policy? And you can also identify utilities for remote access in this way (for example, RAT), cryptominers, cloud or peer-to-peer applications, etc.

If you are a proponent of a single-vendor approach, building a security system based on Cisco solutions will allow you to reduce your dependence on SIEM — all Cisco solutions, including firewalls, email protection, a sandbox, a PC protection solution ( Cisco AMP for Endpoints ) and etc. can exchange data, context, security events, commands among themselves directly. But if your infrastructure has inherited many different solutions from different manufacturers, then SIEM will be the link that will help to build a complete solution from the zoo of different products. In any case, the analysis of network traffic collected from the existing network infrastructure, combined with the data that SIEM usually collects, expands the ~~horizons.~~the ability of the IS service to see more and respond faster.

Benefits of using Netflow in SIEM:

correlation of network activity with other information on information security collected from the level of applications and PC / servers
the ability to monitor where there are high risks of violation of privacy laws (for example, GDPR) and you can analyze only the headers or metadata of network traffic, and not its contents
detection of anomalies characterizing the first stages of the development of an incident or targeted attacks
collection and storage of various data characterizing the incident, and providing them as part of IS investigations or interaction with law enforcement agencies.

Silver bullet?

But don't think that Netflow is the silver bullet everyone has been looking for for so long. It also has disadvantages. For example, its processing can load the processor and memory of outdated or incorrectly selected network equipment and this can adversely affect its performance and network bandwidth. To work effectively with Netflow, you may need its hardware support or the use of the so-called exporters, who by passing traffic through themselves will translate it into Netflow (those who have encountered the introduction of IDS / COB in switched networks, used so-called tapes or splitters for a similar task ) I have already given two examples of such external solutions - Cisco Stealthwatch Flow Sensorand the Cisco Netflow Generation Appliance. Although, given the recent modernization of the network, it can be assumed that your switches and routers already support one or another version of Netflow and you will not need any additional exporters.

Other Netflow features worth knowing include:

false positives that can be associated with incorrect or insufficient training of the analysis system, as well as with a change in IT operations, about which the analysis system has not yet been “informed”
decreased performance and impact on SIEM storage (we'll talk about this even further)
metadata collected through Netflow does not always allow a full investigation, which may require raw traffic in the PCAP format.

SIEM Integration Options with Netflow

Which of the SIEMs on the market today work with network traffic? I must say that almost everything, but in different ways and often this requires a separate paid license, which, in turn, is also divided into options. I would highlight three options for analyzing Netflow in SIEM:

built-in support for Netflow in SIEM
own exporters / sensors for generating Netflow and transmitting to SIEM
SIEM integration with external solutions of the Network Traffic Analysis (NTA) class.

SIEM with integrated Netflow support

For example, Microfocus ArcSight, which is quite popular in the post-Soviet SIEM space, has built-in support for Netflow. This feature allows SIEM to correlate network flows with other security events on the fly or to enrich them with data from Threat Intelligence sources. However, this option has its drawbacks, namely:

-, , . ( , « », )?
-, Netflow SIEM, Netflow . ? SIEM , , , ? - «» Netflow ?

-, . ? VPN- ( ).
-, Netflow SIEM FPS ?
, , flow- (. ). Netflow, SIEM. Netflow — v5, , v9, IPv6, MPLS . Flexible Netflow ( Netflow v9), IPFIX, Netflow v10, sFlow, , , , NetStream, Jflow .. SIEM?

If you are only facing the choice of SIEM, then include in the list of parameters under consideration also the type of Netflow, which is generated by your equipment. If you have already purchased SIEM, then you do not have much choice. In this case, you should consider the following options:

a separate IS network anomaly analyzer (for example, Cisco Stealthwatch), which will conduct the entire analysis on its own, and give its results to SIEM
Separate Netflow collector, which will be able to submit to SIEM summary analytics on network flows, and SIEM will already analyze this data.

How much to hang in grams, that is, stored in bytes?

By the way, before we move on to the next option for integrating Netflow with SIEM, it's time to touch on the question, how much Netflow data will we receive in our security event analysis system? There are a lot of examples and statistics on EPS (events per second) for security features or regular logs. There is not much data on FPS (flow per second). Netflow's average volume is directly proportional to the number of unique TCP / UDP sockets created by client devices and servers on your network, which can vary greatly from case to case. And the inclusion of sampling (that is, selective transfer of Netflow data) also affects the total amount of data.

So how many FPS can we generate? Of course, this depends a lot on the situation, but I would say that for a regular workstation this number is, on average, 1.5 FPS, and 6 FPS at peak load. In other words, if your network has 10 thousand nodes and the average FPS for each of them is 4, then the network generates about 40 thousand flows per second. Why so much? As I wrote above, it depends on how many unique connections your applications or your network generate. Today, there are a lot of “chatty” programs running on users' computers, which either actively load content from the Internet, such as browsers, or constantly check for updates, such as antiviruses. Here is a sample list of software and services that are actively increasing the number of FPS on the network:

Adobe, antiviruses, Java
Skype
email clients
Netbios
browsers
feed-oriented apps (Twitter, news, Telegram, etc.).

A more accurate answer will tell you the analysis of Netflow in the desired network segments, which is done with just one command on a network device (depending on the manufacturer).

The length of one Netflow v5 record is 48 bytes. For the 9th version of Netflow, such an exact figure does not exist, since this version allows you to describe what you will include in the record and therefore its length can vary greatly. But if, very roughly, we take 100 bytes for the average length of a stream record (and each network packet can generate 20-30 streams), then we can estimate how much data will be generated and transferred to SIEM. At the same time, the SIEM storage volume for this data may be larger (this will depend on the storage format, indexing, compression, backup, etc.). By the way, when calculating the number of FPS, remember that in the framework of a DDoS attack, the concept of “average FPS” does not work, since each connection, each TCP SYN packet will be a separate stream, and with a powerful DDoS attack, the number of FPS in your peak will be very large.

I mentioned above that in the case of transferring Netflow to the central SIEM, you will have to “drive” it through the Internet. Do not think that the generation of Netflow will create a huge load on the network and reduce its bandwidth. According to our research, since only heading information and additional telemetry are transmitted in Netflow, and not the entire data body, the load will increase by about 1-2% to the interface from which network telemetry is exported (in fact, using sampling and modern protocol versions Netflow this value can be even less by an order of magnitude and vary at the level of 0.1%).

Collect cannot be analyzed

But let's say you still decided to get a raw Netflow on your SIEM. This scenario has another nuance. It is very important to understand that the availability of Netflow support in SIEM is not enough; It is extremely important to be able to handle this Netflow from a security point of view, that is, have rules for analyzing and correlating Netflow streams built-in and constantly updated for new types of attacks. Let's say Netflow gives us this picture:

We see a surge in the SSH protocol. In fact, we are now seeing the same picture with attacks on the RDP protocol. This is password guessing. But this can be revealed only on the condition that we have a corresponding rule, which from a number of Netflow streams will collect one event “Password Matching”. Then we can say that SIEM has built-in Netflow support and can analyze it from a security point of view. Therefore, choosing this path, you should ask the seller what can analyze SIEM in Netflow “out of the box” and, if nothing, how laborious is the process of describing your own Netflow processors and do you have any specialists who will do this? We are well aware that writing a connector to Netflow is not so difficult, unlike the rules for processing it and identifying anomalies in it that require constant work.This is about copying someone else’s engine for your IDS (Snort, Zeek or Suricata), but not being able to write signatures for newly discovered exploits and attacks on an ongoing basis. In the example above, the system itself should recognize a surge in traffic on SSH and say itself that these are “password guessing” attacks on SSH (either Telnet, or RDP, or FTP). It might look like this (for exampleCisco Stealthwatch Enterprise ):

And then you can investigate this incident at a deeper level, using the capabilities provided by either SIEM or a separate Netflow analysis tool. Without the ability to "understand" Netflow in terms of information security, the presence of built-in support for Netflow is a dubious advantage for SIEM.

SIEM with its own Netflow exporter

Another player on the SIEM market, LogRhythm, in turn, offers an additional exporter of NetMon streams, which can be useful in a distributed infrastructure, as well as in a network whose equipment does not support Netflow and requires a separate Netflow generator for network traffic passed through it. In fact, in this embodiment, the SIEM manufacturer takes on the functions of network vendors, developing a solution for generating Netflow and reducing the load on SIEM, which eliminates the need to process and store raw Netflow. The situation is similar to the support of SSL Offload on many new generation firewalls. Yes, it exists there, but with an intensive exchange of HTTPS traffic, the additional load on NGFW leads to a significant decrease in its throughput.Therefore, in heavily loaded architectures, a separate device is usually allocated for this task, which takes on the task of decrypting SSL traffic and then returning it to NGFW. The same thing happens with the SIEM Netflow processing in this scenario.

It is clear that this scenario also has a drawback - an increase in the final price of the solution, since in addition to paying licenses for the number of analyzed FPS, you will also need to pay for additional sensors that will pass traffic through itself and generate Netflow. In addition, you will need to make changes to the network architecture, but this will have to be done anyway, so I would call it not a drawback, but a feature of this scenario. If your network equipment does not know how to generate Netflow, and you want to analyze network anomalies, then the only option to do this is to use separate sensors. The only question is what will be cheaper - buy sensors from the SIEM manufacturer or use sensors from network manufacturers (for example, Cisco Netflow Generation Appliance) or developers of network anomaly analysis tools (for example,Cisco Stealthwatch Enterprise Flow Sensor). In this option, it is also worthwhile to find out if SIEM is able to analyze Netflow from the point of view of information security, or does it just have connectors in the form of exporters / sensors taken out (usually it can)?

SIEM, NTA

The third option, integration with NTA class solutions, is quite obvious, since, in fact, NTA is the same security event generator as NGFW, antivirus, security scanner, IPS, etc. This scenario, however, is interesting in that you combine the two security analytics tools you have, but you can work with them separately. NTA allows you to conduct an in-depth analysis of network traffic, detect malicious code, DDoS attacks, information leaks, monitor remote users ... At the same time, a good network traffic analysis tool also allows you to use separate sensors in those segments where network equipment does not support Netflow or its inclusion leads to an increase in the load on network equipment. NTA in this scenario allows you to aggregate, process, analyze Netflow (in its different versions),and on SIEM give only alarms upon the fact of detection of one or another malicious activity. Obviously, this option also makes sense to use when you or your IT staff already have an NTA class solution for network troubleshooting and it can also be used for network security tasks. Or when, on the contrary, you want to share the expenses for the NTA solution with your networkers, who will use it for their tasks, and you for yours.who will use it for their tasks, and you for yours.who will use it for their tasks, and you for yours.

The disadvantage of this option is the additional cost of an NTA class solution, as well as the need for double training of specialists in the basics of working with two different solutions. But on the other hand, a separate solution for analyzing network traffic will allow for deeper incident investigations than with a single SIEM with built-in Netflow support, and more flexible applications than with the SIEM manufacturer who has separate Netflow sensors. But it’s worth remembering that when we talk about a separate solution of the NTA class, I mean the security solution, and not just a tool for analyzing network traffic or monitoring network performance. For example, the SolarWinds NTA already mentioned earlier analyzes network traffic well in order to support IT tasks, but it does very poorly for information security purposes.The same goes for 5View from InfoVista or Visual TruView from Fluke. And the same one, for example,Cisco Stealthwatch Enterprise can be used by both in the company.

What to look for when choosing?

When choosing an NTA solution for security purposes, as well as analyzing exporters / sensors provided by SIEM or SIEM vendors with Netflow support, I would recommend paying attention to the following criteria:

Types of detectable malicious activity. You take a tool for monitoring information security; it is logical to assume that it should be able to identify various anomalies and threats related to information security “out of the box”. Moreover, this parameter is divided into three parts - built-in algorithms for detecting various types of information security threats, writing custom handlers / rules and supporting external sources of Threat Intelligence to enrich the analyzed flows with data about the threats.
Netflow. , , .
. , 1 ( FPS)? 60 FPS, NTA 40 , , , , , 80 FPS.
. , flat, … , . , . . , (, , Argus, Fluke, Plixer, Riverbed SolarWinds), , . , . , ; , . , , Cisco Stealthwatch.
. , , , , . — NTA. , Cisco nfdump OSU FlowTools Lancope, .
. , - . :
- /
- flow cache ( cash flow :-)
- . MAC-, VLAN, MPLS, TCP, , , , , ( IPFIX ).
. Netflow SIEM, SIEM NTA . , , (, ) REST API, syslog ..

If it was a question of what to choose regardless of SIEM, then I would also advise to look towards the threat hunting opportunities that the chosen solution provides. But since the topic of the note was chosen a little different, then we will not pay attention to this aspect now.

Price issue

If you look at these three options from the point of view of their cost, then the third option will turn out to be the most expensive from the point of view of capital expenditures (subject to the same capabilities for Netflow analysis in all three scenarios). It is understandable. In addition to the cost of SIEM, you will need a separate solution for analyzing network traffic, which will consist at least of a control system and the required number of collectors collecting Netflow from various exporters / sensors. On the other hand, no matter how much you expand the coverage of your network with new exporters / sensors, it will not affect the cost of SIEM and infrastructure for it, since it will work with already processed alarms, and not with “raw” Netflow streams, which (signals) will be several orders of magnitude less.

The first option looks the most attractive from the point of view of its price, since we do not have to pay extra for the training of analysts and NTA administrators, or for additional exporters / sensors; know, transfer Netflow streams to SIEM and all. However, the cost of infrastructure for SIEM will increase significantly, as you will have to store raw Netflow streams, which will require expanding your existing storage. The second option is intermediate between the two extremes, both in terms of Netflow analysis capabilities and in terms of cost. In any case, in the first two options, it is worth checking with the SIEM manufacturer the total cost of ownership of the solution based on the challenges, as well as the current and expected FPS and, as a result, changes in the cost of licenses for SIEM and storage for it. Take for exampleLogRhythm and its subsystem for Netflow analysis. There are at least three options for its implementation and, as a consequence, pricing. The youngest option, Freemium can only send alarms to SIEM, the bandwidth cannot exceed 1 GB / s, the storage capacity is only 1 GB (this is about one day of Netflow storage without sampling), the index storage period is not more than 3 days, and there is no correlation with additional data sources, and support works only online and through community. In the next version, NetMon, the indicators are better (bandwidth up to 10 GB / s for all exporters, storage is unlimited, the index is stored for up to a month, but there is no correlation with other sources either). And only in the premium version of NetworkXDR you have no restrictions, but it stands as “a kilometer of Moscow roads”, that is, not cheap.

In one of the projects, we were faced with the fact that with a daily volume of network telemetry of 1 TB and sending it to SIEM with built-in Netflow support, the total cost of the solution was about 600 thousand US dollars (even before the next jump in the course). At the same time, part of this data remained unprocessed due to the lack of appropriate rules in the SIEM and duplicate. The use of a separate NTA solution (in our case it was Cisco Stealthwatch Enterprise ) led to a 80% reduction in the volume of data transferred to SIEM, and the cost of the solution dropped to 99 thousand US dollars. The math may differ from project to project, but we noticed that the more Netflow we need to process, the more expensive the SIEM infrastructure will be to process it. Why is that?

Let's look at one example. When a user connects to the server, from the point of view of the classical analysis of information security events, we deal, conditionally, with only one event that we “remove” from the server, sending it, for example, via syslog to SIEM (in fact, there will be two events - an attempt connection and its result). If this event is decomposed into network components, then we will see that there will be an order of magnitude more threads generated. For example, the average number of “hop” (hop) from the client to the server is 5-6 in the average network. Each switch or router through which the request passes from the client to the server side generates its Netflow entries describing the traffic passing through. Moreover, this is done for each direction of the session (request and response) separately. It turns out that there,where only 1-2 events were generated at the application level, Netflow flows required at least 10 times more (actually even more, since one network packet generates about 20-30 Netflow flows). And not only will we have to pay for all these dozens of threads (although the event is still one) and allocate space for their storage. So, SIEM will also have to process them, remove duplicates, combine multidirectional flows within one session and only then correlate this data with other events. Therefore, the total cost of a seemingly obvious solution may be higher than in the case of a separate Netflow exporter or a standalone network anomaly analysis solution integrated with SIEM.so one network packet generates about 20-30 Netflow streams). And not only will we have to pay for all these dozens of threads (although the event is still one) and allocate space for their storage. So, SIEM will also have to process them, remove duplicates, combine multidirectional flows within one session and only then correlate this data with other events. Therefore, the total cost of a seemingly obvious solution may be higher than in the case of a separate Netflow exporter or a standalone network anomaly analysis solution integrated with SIEM.so one network packet generates about 20-30 Netflow streams). And not only will we have to pay for all these dozens of threads (although the event is still one) and allocate space for their storage. So, SIEM will also have to process them, remove duplicates, combine multidirectional flows within one session and only then correlate this data with other events. Therefore, the total cost of a seemingly obvious solution may be higher than in the case of a separate Netflow exporter or a standalone network anomaly analysis solution integrated with SIEM.to combine multidirectional flows within one session and only then correlate this data with other events. Therefore, the total cost of a seemingly obvious solution may be higher than in the case of a separate Netflow exporter or a standalone network anomaly analysis solution integrated with SIEM.to combine multidirectional flows within one session and only then correlate this data with other events. Therefore, the total cost of a seemingly obvious solution may be higher than in the case of a separate Netflow exporter or a standalone network anomaly analysis solution integrated with SIEM.

And such external Netflow analyzers can be used in one of two scenarios. The first is the transfer to SIEM of optimized telemetry, cleared of repetitions (deduplicated), combining multidirectional streams. Based on our experience (and Stealthwatch Enterprise can work in this mode), I can say that this gives a six-fold reduction in the volume of telemetry transmitted to SIEM, which, in this scenario, should still be able to analyze Netflow from a security point of view.

The second scenario assumes that all processing is carried out on a solution of the NTA class, and only alarms received as a result of Netflow processing are received in SIEM. This option reduces even more data sent to SIEM, reducing the cost of its licenses and infrastructure. Yes, and SIEM is no longer required to be able to analyze raw Netflow, which expands the possibilities for choosing tools for analyzing information security events.

Are there any alternatives?

Having examined the options for using Netflow in SIEM to detect more security events than without Netflow, let's look at possible alternatives.

Network Diagnostic Tools

You can try using the network diagnostic tools used to assess network bandwidth, the availability of internal nodes and segments, peak loads, etc. For example, SolarWinds NetFlow Traffic Analyzer. These solutions are not designed to detect IS anomalies and threats, but they can be used to transfer flow information to SIEM, which may be able to analyze Netflow. This will load SIEM, as I described above, and you have to decide whether you are ready for this? True, it is worthwhile to clarify first whether the network analyzer can send Netflow data to external systems. Sometimes they can give out only summary statistics or generate alarms, which is clearly not enough for IS tasks.

Intrusion Detection Systems and NGFW

Network IPS and NGFW are another alternative. They can peer inside network traffic and can detect many threats through a deep network inspection mechanism ... but on the perimeter. Still, usually NGFW and IPS are placed on the border of the corporate network and the Internet and see only what passes through them. Installing these devices, “iron” or virtual, in every place you are interested in will be too expensive, and in some cases technically impossible. Speaking about the border or perimeter, it is worth remembering that the interface between the corporate data center and user segments is also the border. And the junction between the corporate and industrial network, too. But in any case, installing additional IPS or NGFW sensors can hit your wallet painfully, while Netflow will be collected from already purchased and implemented network equipment.

Network Incident Investigation Tools

Network Forensics Tool (NFT) class solutions allow you to store, process and analyze network traffic in order to investigate incidents. But there is a significant difference between this type of solution and NTA - NFT works with a full copy of traffic, that is, usually with PCAP files, and NTA with records about it. And if Netflow analysis solutions work almost in real time, then NFT - with a significant delay. In addition, any solution that works with PCAP (or in another way to capture and save a copy of all network traffic) will run into the problem of a place to store all the collected data.

Imagine that you have a port on a 1 Gb / s network device. With a packet length of 64 bytes, this port will be able to pass through 1953125 packets per second, and with a length of 1500 bytes - 83,333 packets. That is, depending on the length of the packet (and this depends on the applications on the network), we will have from about 80 thousand to 2 million packets per second, which we will need to save. On a day, such a port will allow 86400 Gbit / s or almost 11 TB. Almost 4 PB will run over the year, and this is only for one port, of which we can have thousands and tens of thousands in our network. Even selective storage of traffic will not greatly facilitate our lives. Therefore, NFT class solutions are needed, but they are not able to replace Netflow analyzers. These are tools that solve different tasks - monitoring and investigation of incidents.Typically, these solutions work in pairs - Netflow allows us to identify incidents, and NFT already collects detailed data on them in the form of capturing all network traffic.

" But there is SIEM with NFT, for example, IBM QRadar Incident Forensics or RSA Security Analytics, which allow you to work with a full copy of network traffic and all Netflow meta-data will automatically be available in SIEM"Yes, there is! In addition, the advantage of this solution is the possibility of reconstructing all network sessions of interest and visualizing them, which can facilitate the investigation of incidents. This SIEM allows you to take the place of the attacker and see everything that he sees. But this advantage is hidden and a serious drawback that I mentioned above is that storing a full copy of network traffic requires a large, no, not so huge storage that can cost as a separate SIEM, or even more (more than storing even just raw Netflow ) .You may need to choose which specific sessions to save in order to save space, and this may lead to some traffic being lost. In addition, in case of compliance with the legal requirements for storing data related to incidents,You will have to break them or pay extra money for storage. Another feature of this solution is the need to save traffic that has already been decrypted in SIEM, which means you will need to review the architecture of your monitoring system in order to be able to decrypt traffic and submit it to SIEM. And do not forget that such solutions are still focused on conducting investigations, which requires qualified personnel, and not on detecting anomalies and threats using ready-made algorithms.that such decisions are nevertheless focused on conducting investigations, which requires qualified personnel, and not on detecting anomalies and threats using ready-made algorithms.that such decisions are nevertheless focused on conducting investigations, which requires qualified personnel, and not on detecting anomalies and threats using ready-made algorithms.

In this scenario, I would still look at the initial analysis of network traffic using Netflow, and only then, for the events and incidents of interest to me, I would enable network traffic recording. This will save and not spend resources on the storage of "everything."

Conclusion

So, to summarize briefly. Netflow is a valuable source of data for monitoring the security of corporate and departmental networks, often the only one that can be collected and based on which you can make decisions about the presence or absence of threats in your environment. In principle, Netflow can also be analyzed independently using NTA class solutions, which, having a large database of rules and algorithms for detecting malicious and abnormal activity, are able to quickly detect incidents and respond to them. Integration of Netflow with the data collected by SIEM gives us much more. SIEM begins to see what it has not seen before and see it much earlier than we can be harmed. At the same time, we do not need to make strong changes to the existing monitoring infrastructure, since we already have network equipment,- you just need to redirect Netflow to SIEM, directly or through intermediate solutions. Enabling Netflow also allows me to achieve a small but quick victory - almost all of our solution pilotsCisco Stealthwatch Enterprise ends with the fact that we detect certain violations of IS policies that were not previously seen by the IS service. Netflow allowed them to be seen, and its integration with SIEM, to get a synergistic effect from the applied network monitoring tools and the activity system.

Integration of Netflow and SIEM Monitoring Solutions