Our World Watch service works on behalf of the customer to collect, analyze, prioritize, contextualize and summarize the essential threat and vulnerability data customers need to make informed decisions. The Orange Cyberdefense Computer Emergency Response Team (CERT) analyses information from public and partner sources that is enriched with our proprietary threat intelligence. The information is triaged to determine accuracy, significance, relevance, and risk level before being shared to our clients. 

Our World Watch Advisories covered 8 of the 10 Ransomware Index Top 10 CVEs during the period October 2021 to September 2022.  

The Log4J vulnerability, CVE-2021-44228, was mentioned in 6 World Watch Advisories or updates. At the time, the flaw was considered extremely serious. Security professionals were convinced that this flaw, if left unpatched, could result in devastating breaches. 

The vulnerabilities not covered by our World Watch Advisories are CVE-2020-5902 - a remote code execution vulnerability in the Traffic Management User Interface (TMUI) of F5 BIG-IP - and a remote code execution vulnerability in the Microsoft VBScript Engine, tracked as CVE-2018-8174. We opted not to publish advisories containing explicit references to these CVEs. 

When comparing vulnerabilities mentioned in the World Watch Advisories with the full KEV Catalog we see a slightly different story, but with familiar elements. Bear in mind that the KEV Catalog has over 860 entries at the time of writing, so we are sampling across a much larger set.  

Our CERT team warned customers of 95 of the 860+ vulnerabilities that appear in the KEV Catalog, the first ten of which are shown above. Log4J was discussed the most, as already mentioned.  

The second highest number of mentions is a tie between a remote code execution vulnerability impacting Microsoft MSHTML (CVE-2021-40444), a vulnerability in Atlassian Confluence (CVE-2022-26134), and a vulnerability impacting the Microsoft Windows Support Diagnostics Tool (CVE-2022-30190). 

Six of the 10 vulnerabilities mentioned here overlap with the Ransomware Index Top 10. These are CVE-2021-44228, CVE-2022-26134, CVE-2017-11882, CVE-2021-31207, CVE-2021-34473, andCVE-2021-34523.

Now let’s consider the occurrence of the Ransomware Index Top 10 CVEs in the findings recorded in our VOC scanning data for October 2021 to September 2022.  

On the assumption that our vulnerability scanners have a reasonable chance of detecting these vulnerabilities, we’re asking whether we ever identify these issues on client estates. 

We only see half of the potential Ransomware Index Top 10 vulnerabilities reported in the sample of client assets we examined.  

Ransomware Index Top 10 Vulns reported on client assets in a subset of OCD VOC Scanning Data 

• The one vulnerability that was absent in our World Watch Advisories (CVE-2018-8174) was in fact reported on client estates (but never exploited by our penetration testing teams). 
• The three most prominent CVEs here are all related to Microsoft desktop application products and are thus not directly exploitable over the network: CVE-2017-0199 and CVE-2017-11882 impact Microsoft Office 2007-2016; and CVE-2018-8174, impacts the VBScript Engine. We assume that these three vulnerabilities are exploited through phishing emails with malicious attachments or links to malicious Office documents.  
• The Log4J vulnerability, CVE-2021-44228 is present to no surprise and was discovered at 17% of sampled clients.  
• Our VOC team also identified vulnerable Atlassian Confluence servers, CVE-2022-26134. 

Percentage of Assets impacted by vulnerabilities listed in Ransomware Index Top 10 

Looking at the number of impacted assets, we see that the Microsoft vulnerabilities CVE-2017-11882, CVE-2018-8174, and CVE-2017-0199 impacts the most assets of all the vulnerabilities listed in the Ransomware Top 10. The Log4J vulnerability, CVE-2021-44228, still looms close to the forefront. Comparably the Atlassian Confluence Server vulnerability impacts a handful of assets compared to the first four present on assets scanned.  

When cross-referencing the VOC scanning data with the 860+ entries in the full KEV Catalog, we see a rather different picture: Microsoft vulnerabilities dominate the first 20 records, as pictured below. We also observe a large chunk of Cisco IOS vulnerabilities relating to an SNMP flaw in the assets we scanned.  

The table lists the descriptions of the relevant vulnerabilities related to the VOC/KEV Catalog chart. 

First 20 Vulns found when comparing vulns in VOC with the KEV Catalog

As we can see the top 20 vulnerabilities discovered by the VOC in terms of the KEV is dominated by two enterprise vendors namely Microsoft and Cisco. Microsoft products are very popular, and any vulnerability in a core Microsoft product will quickly result in many reported vulnerabilities.  

The chart below also includes the Log4J vulnerability, CVE-2021-44228, tacked on to show its relative impact. There are at least 10 other KEV vulnerabilities we reported at VOC clients more often than Log4J. 

Top 10 KEV CVEs with Log4J Vuln impacting percentage of VOC clients

A more useful perspective, perhaps, is to consider: At how many clients did we see vulnerabilities in the KEV catalog reported. We can search a sample of vulnerability scan reports for a significant subset of our clients to determine how frequently vulnerabilities in the KEV are reported on client assets. We note that this is a limited sample biased by the obvious fact that these clients have implemented robust, professional vulnerability management programs, and would thus not be fully representative of the entire cyberspace. 

This perspective is reflected below: 

Distribution of KEV CVE across our scanning client base

The chart above depicts what proportion of the KEV catalog has been reported across our client base between October 2021 and September 2022. The Y-axis reflects a proportion of KEV, while the X-axis reflects the proportion of the clients we sampled. 

The resulting distribution is illustrated above. An examination of this data reveals the following: 

1. An astonishing 52% (450 CVEs) of the vulnerabilities listed in the KEV were not reported at any of the clients sampled. It’s hard to understand why this would be, except that these vulnerabilities exist in technologies that are not very widely deployed and very specific to U.S. Government FCEB agencies. 
2. Not a single vulnerability in the KEV impacted more than 44% of the clients sampled. 
3. One CVE, CVE-2021-40438, impacted assets associated with 44% of clients. The vulnerability is found in Apache HTTP Server. 
4. Seven CVEs of the KEV list were reported at 17% of clients - 6 of these 7 CVEs are sequential (CVE-2017-0143 to CVE-2017-0148) and are related to SMBv1 within Windows operating systems, including ‘EternalBlue’ (CVE-2017-0144). The final CVE follows a similar pattern, in that it is another Windows vulnerability which received significant attention from the community – CVE-2019-0708 ‘BlueKeep’. These are vulnerabilities dating back to 2017 and 2019 respectively. Many of the affected hosts would eventually have been patched by our clients, of course, but on average these 7 CVE persisted on hosts for 451 days! In our 2023 ‘Security Navigator’ report we note that the average age of a vulnerability on our client estates is ‘only’ about 215 days. Considering the ubiquity and severity of these issues, this is a very concerning figure indeed. 
5. Interestingly, 14% (121 CVEs) were reported at 15% of clients, depicted by the ‘bump’ in the graph featured above.  These 121 vulnerabilities mostly impact common Windows components, which accounts for the large proportion of our clients that are impacted by them. The oldest vulnerabilities were in this set was allocated to Oracle Java dating back to 2011! Several browser vulnerabilities were also present, and spread between Google Chrome, Mozilla Firefox, and Microsoft Internet Explorer. 
6. A total of 159 vulnerabilities from the KEV each impacted less than 10% of our client base. This seems like an important observation, as it suggests that businesses can be severely impacted by an exploitable vulnerability in a technology that is not widely deployed, or an uncommon vulnerability that has not been patched. This serves as a reminder that security managers need to take the severity and exploitability of a vulnerability into account, not just the frequency with which it occurs. 

Are our penetration testing teams exploiting these vulnerabilities?

When we compare the Top 10 vulnerabilities from the Ransomware Index with the vulnerabilities reported by our Penetration Testing teams (represented by a sample of 1,400 test reports we analyzed), the only vulnerability in common between the two is the Log4J vulnerability (CVE-2021-44228), and that was only reported once.

The three Microsoft vulnerabilities; (CVE-2018-8174, CVE-2017-0199, and CVE-2017-11882), for example, are potentially suited to phishing attacks, which is often considered out of scope of ‘traditional’ penetration testing engagements.

KEV vulnerabilities exploited during penetration tests

Comparing the vulnerabilities in the penetration testing dataset with the much larger KEV Catalog, we observe a small number of vulnerabilities shared across the datasets:

These vulnerabilities are therefore being listed in the KEV catalogue and being exploited by our analysts during penetration tests. Here’s what we see:

• The Remote Desktop Protocol flaw, CVE-2019-0708 aka BlueKeep, in Microsoft Terminal Service is a serious vulnerability that has reliable proof-of-concept exploits readily available. Penetration testing team found this flaw is sometimes accompanied by Eternal Blue and Eternal Synergy (CVE-2017-0144/CVE-2017-0143), as well as the two Microsoft Active Directory vulnerabilities CVE-2021-42278 and CVE-2021-42287. 
• The presence of the Microsoft Windows SMB protocol flaw in the form of Eternal Blue and Eternal Synergy highlights the very real risk posed by these wide-reaching flaws with reliable exploits.

Once again, we can determine how frequently vulnerabilities in the KEV are reported at our clients, this time in penetration testing reports. We note again that, because these clients have robust, professional vulnerability management programs, our sample is biased and not necessarily representative of the entire Internet.

The chart above is based on an examination of a sample of 1,400 test Penetration Test reports that we analyzed. Each colored series reflected in the legend reflects a proportion of our sample client base, while the size of the segment in the pie reflects the proportion of the KEV observed.

The following very small numbers emerge:

• 99.2% of the CVEs in the KEV did not feature in even one of the Penetration Test reports we examined. This is not entirely surprising due to the observer bias we emphasized above, but also because of the way we expect our testers to work, in other words do not focus purely on finding CVEs.
• The 0.8% of the KVE that we did reference in our Penetration Test reports amounts to just 7 CVEs.
• One of these CVEs, CVE-2019-0708 - A remote code execution vulnerability exists in Remote Desktop Services (BlueKeep), was referenced at about 2% of the sampled clients and was also listed under vulnerability scanning section earlier. CVE-2017-0144 - Windows SMB Remote Code Execution Vulnerability (EternalBlue) – was referenced at about 1% of sampled clients and the other 5 CVEs from the KEV were each referenced at only about 0.4% of sampled clients.

This is rather surprising, as one would think the list presents prime vulnerability candidates for penetration testing teams to exploit. Exposed services such as Atlassian Confluence Server, Microsoft Exchange, Fortinet FortiOS devices, and F5 TMUI would be easy pickings for our testers. It appears that we are simply not encountering vulnerable versions of these platforms within the scope of the tests we sampled, or that exploiting these vulnerabilities is being precluded by the test scope.

At the 44Con security conference in London in 2011, a security researcher called Haroon Meer delivered a presentation titled ‘Penetration Testing considered harmful’.

Penetration Testing considered harmful?

The presentation asks a lot of searching questions of Penetration Testing service companies. One of Meer’s assertions is that Penetration Testers develop their own kill chains on what they know and what works for them, but not necessarily based on what real-world adversaries are doing in the wild.

Could it be that the low prevalence of KEV and the ‘ransom top-10’ CVEs in our penetration testing sample set support that notion? We do not think so. Here’s why…

Our first consideration is that we don’t really understand the rationale for vulnerabilities being listed in these catalogues to begin with. The logic behind both lists seems to be that be that these vulnerabilities have been exploited in the wild … somewhere. It clearly makes sense to patch them if you have them, but that’s not to suggest that are commonplace, or that they represent the sole or premier risk to any given business.

We note that the vulnerabilities in the two catalogues occur very infrequently at our customer base to begin with, as the low incidents of these vulnerabilities in our scanning data attests. We do not understand the processes that result in vulnerabilities being added to these lists well enough to understand why this is, but we believe this is more a function of the make-up of the lists, then the security of our customers or the behavior of our testers.

Next, we must understand the specific role penetration testing plays in the security assessment domain. Pure vulnerability scanning services aim to cover as much ground as possible by verifying the state of all assets in terms of known vulnerabilities. Penetration Testing or Ethical Hacking aims to demonstrate the exploitation of vulnerabilities that allow the hypothetical attacker to move deeper into an environment or cross control boundaries. In other words, it is a matter of breadth vs depth.

The toolbox of ethical hackers consists of several tools, of which a vulnerability scanner is one. The role of the vulnerability scanner is to automate the environment reconnaissance or discovery process. The analyst can then perform several actions in parallel to cover even more ground.

Finding a flaw with a vulnerability scanner may not be enough to convince a client that there is real risk. High and critical vulnerabilities may carry enough weight for consideration, but without demonstrating that these could be exploited bind the argument to speculation. The most compelling evidence of the impact of a vulnerability or sets of vulnerabilities is chaining or combining these into multiple exploitation phases. This demonstrates how several medium or even low impact vulnerabilities exploited in tandem can have a material impact. Penetration testing is not about CVE hunting, but about demonstrating the real impact of weaknesses in systems or applications.

We should, however, recognize that some ethical hacking projects require that analysts work on live client environments. In many cases, this means the analyst needs to be careful not to disrupt or destabilize the client’s environment. This constraint means that analysts must use exploits that are reliable, work as expected, and leave the target unharmed. A truly malicious threat actor may not be constrained by the limitation and may therefore exploit vulnerabilities a penetration tester might not. This seems unlikely to be a big factor, however, since most malicious threat actors prefer to remain undetected and would avoid disruptive action that might draw attention.  

Another potential mismatch to consider is that penetration tests often proceed from some assumption of some successful compromise. For example, it may be assumed that an attacker has breached the perimeter via a phishing attack so that testing can proceed from there. This makes sense for a business that wishes to test the effectiveness of its internal controls and detections on the assumption that the perimeter will eventually be breached. This kind of ‘scenario-based’ testing may result in well-known exploits never being deployed in testing.

Goal oriented penetration testing a form of testing that is specifically designed to emulate the behavior of a specific adversary and would likely include vectors like phishing. This kind of testing is relatively infrequent, however, therefore vulnerabilities associated with phishing attacks are less frequently reported on.

Finally, many attacks do not require exploitation of a vulnerability. Some attacks can target misconfigurations or abuse legitimate system behavior. One such example involves stealing password hashes or authentication tokens and manipulating these to demonstrate risk in environments by gaining elevated privileges or impersonating someone or some service. 

The techniques deployed and vulnerabilities leveraged by the penetration tester are therefore a function of the goals and constraints set by the clients. Where vulnerabilities that happen to be listed in one of the catalogs is encountered, and supports those goals, it may be used. But as these vulnerabilities occur infrequently to begin with, this convergence simply does not happen very often.

Attackers think in graphs

We end this post with a contentious thought.

A pervasive security truism was first captured by John Lambert in 2015 on his blog “Defenders think in lists. Attackers think in graphs. As long as this is true, attackers win”. 

John’s point suggests that attackers succeed in compromising networks and systems because they understand and exploit the complex trust and dependency relationships that exists across diverse systems in a company, as well as the people that use them. In a classic example, an attacker will compromise a humble workstation that’s integrated with the organization’s Active Directory, then use a local exploit to dump hashed user login credentials and decrypt them. Those clear text credentials can then be used to exploit another computer, dump credentials, etc. Rinse and repeat until the attacker achieves her goals.

In such a scenario, it really does not matter where the attacker starts her attack, and any vulnerability in any system may become a key factor in achieving a full compromise.

This observation has important implications for our very first argument – that security managers need to prioritize patching based on the ‘seriousness’ of the vulnerability and the ‘value’ of the system. It remains clear that we need to find means of prioritizing vulnerabilities and patches, and exploitability would be an obvious variable to consider. Placing too much emphasis on the relative value of the asset, may be an example of ‘list thinking’, however, and potentially not a viable strategy in the face of the kind of ‘graph’ thinking that attackers are likely to deploy.