Get best of the week

Hidden Threat - Vulnerability Analysis Using News Graph

When you are faced with a new vulnerability, what is the first thought? Of course, respond as quickly as possible. However, speed is just one of the conditions for an effective fight against information security threats. When it comes to corporate security, it is equally important to accurately determine what you should respond to first. An underestimated threat can cause serious loss or loss of goodwill. But if the number of vulnerabilities is constantly growing, how to quickly assess their significance and not to miss important details?


Vulnerability dynamics by CVSS group (source - vulners.com)

For ranking vulnerabilities by various criteria, the CVSS Score (Common Vulnerability Scoring System) scale is traditionally used, ranking vulnerabilities by various criteria, from exploitation complexity to the harm done and other parameters.

It would seem, why come up with something else - but the CVSS Score has one weak point - it is based on expert estimates that are not supported by real statistics.. It would be much more efficient to offer experts cases that were already selected according to certain quantitative criteria and make decisions based on verified data - but where to get this data and what to do next? It sounds like an unusual and interesting task for a datacenter - and it was this challenge that inspired me and the Vulners team for a new concept for assessing and classifying vulnerabilities based on a graph of related information.

Why graphs? In the case of social networks and the media, graph methods have long and successfully been used for various purposes: from analyzing the distribution of content in the news stream, to notes on the influence of TOP authors on readers' opinions and social network clustering by interests. Any vulnerability can be represented as a graph containing data - news about changes in software or hardware and the effects caused by them.

About data


I did not have to manually collect news about each update, all the necessary texts were found in the vulners.com open vulnerability database. Visually, the data is as follows:



Each vulnerability, in addition to its name, publication date and description, has a family (NVD, scanner, exploit, etc.) that has already been assigned to it (CVD rating (CVSS is used hereinafter) v2), as well as links to related news.

If you present these connections schematically in the form of a graph, one vulnerability will look like this: the orange circle indicates the source or parent publication, the black circles indicate the news that you can click on while on the parent page, and the gray circles represent the related news, accessed by You can only go through all the publications indicated by black circles. Each color of the circles is a new level of the graph of related information, from zero - the original vulnerability, to the first, second, and so on.



Of course, when viewing one news item, we know only the zero and the first level, therefore, to get all the data, we used the method of traversing the graph in depth, which allowed us to unravel the tangle of news from the beginning to the most recent connected nodes (hereinafter referred to as the graph node). At this stage, optimization problems got out - graph assembly for a long period took a long time and had to be conjured with both a script and a data structure. By the way, I decided to pack the final data into parquet for further work with them using spark sql, which greatly facilitated the initial analysis.

What does graph data look like? Visualization will help us better understand their nature. Figure 4 shows a graph of the known but not very dangerous Heartbleed vulnerability (only 5 out of 10 points on the cvss scale).



Looking at this magnificent “bouquet” of related news and exploits, where the red dot is the original vulnerability, we understand that Heartbleed was significantly underestimated.

Based on this example, we can conclude that systematicity, duration, and other vulnerability parameters are estimated fairly well using graph metrics. Below are a couple of examples of research metrics that served as the basis for an alternative classification:

  • the number of nodes in the graph - is responsible for the "breadth" of the vulnerability, how large a trace it left in various systems,
  • the number of subgraphs (large clusters of news) - is responsible for the granularity of the problem or the presence of large problem areas within the vulnerability,
  • the number of related exploits and patches - speaks about the explosiveness of the news and how many times it had to be "treated",
  • the number of unique types and families of news in the graph is about systematicity, that is, the number of subsystems affected by the impact of the vulnerability,
  • the duration from the first publication to the first exploit, the time from the first publication to the last related news - about the temporal nature of the vulnerability, whether it stretches with a large “tail” of consequences or quickly develops and fades.

Of course, these are not all metrics; under the hood of the research, there are now about 30 indicators that complement the basic set of CVSS criteria, including the average increase between the levels of the news vulnerability graph, the percentage of exploits at the first level of the graph, and much more.

Open the gray zone


And now a little data science and statistics - after all, hypotheses need to be confirmed on data, aren't they?

For the experiment with an alternative scale and new metrics, news published in January 2019 were selected. This is 2403 newsletters and about 150 thousand lines in the news column. All source vulnerabilities were divided into three groups according to CVSS Score:

  • High - from 8 points inclusive.
  • Medium - from 6 inclusive to 8 points.
  • Low - less than 6 points.

First, let's see how the CVSS score correlates with the number of related news in the graph, the number of news types and the number of exploits:







In the ideal picture, we should have seen a clear separation of metrics into three clusters, but this did not happen, which indicated the possible presence of a gray zone, which CVSS Score does not define - this is our goal.

The next logical step was the clustering of vulnerabilities into homogeneous groups and the construction of a new scale.

For the first iteration, a simple metric classifier and k-means were chosen and a new matrix of estimates was obtained: the initial points (Medium, Low, High) are found on the Y axis, along the X, where 2 are the largest in the new vulnerability metrics, 1 are the new vulnerabilities, 0 are the smallest.



A zone marked with an oval (Vulnerability Class 2 with an initial low & medium rating) —Potentially underestimated vulnerabilities. Separation into new classes also looks clearer, which is what we were aiming for:







However, simply trusting models is a bad idea, especially when it comes to unsupervised clustering, where the correct answer is not known in principle, and you can only rely on the separation metrics of the obtained classes .

And this is where we need expert knowledge - because for a good test and interpretation of the results, knowledge of the subject area is necessary. Therefore, it is advisable to check the model pointwise, for example, by pulling out a couple of vulnerabilities for detailed analysis.

Below are a few bright samples from the gray zone that have a low CVSS score, but a high graph score - which means potentially requiring a different priority for working with them. Here's what they look like in a graphical representation:

CVE-2019-0555 (CVSS score 4.4, graph class 2 high)





SMB_NT_MS19_JAN_DOTNET.NASL (CVSS score 5.0, graph class 2 high)





CVE-2019-1653 (CVSS score 5.0, graph class 2 - high)





RHSA-2019: 0130 (CVSS score 5.0, graph class 2 - high)





It seems that the concept was confirmed by statistics and spot checks, so in the near future we want to refine and automate the collection of graph metrics, and - perhaps - the classifier itself. Of course, there is still a lot of work to do - from collecting a huge number of new graphs for months not covered by the study, but this only adds enthusiasm, as does the essence of the task. As a data scientist, I can say that the work on this study was an incredibly inspiring experience, both in terms of topic and complexity - even preparatory engineering work with loosely structured data was very interesting.

Finally


How to take a step from expert assessments to real numbers and evaluate the invaluable?

After the study, it became clear that first of all, a critical approach is needed not only to any metric or data, but also to the process as a whole, because the world is too dynamic and changes faster than methodologies and documentation. Always evaluated in one way - why not try to shift the angle of view? As our example shows, even the most unusual hypotheses can be confirmed.

An important role is played by the availability of data for datasaentists - it allows you to quickly check the most daring hypotheses and better understand the essence of your subject area in all its manifestations. Therefore, if you are not yet collecting or deleting “unnecessary” data, think about it, maybe there are a lot of discoveries lurking there. This case suggests that data driven and information security complement each other perfectly.

More articles:


All Articles