Are Threat Intelligence Services Actually Delivering Value for Your Money?
The businessman and author Peter Drucker famously said that trying to predict the future is like ‘driving down a country road at night with no lights while looking out the back window’. As difficult as the road may be, there is immense benefit to be gained by those who are able to illuminate the path ahead.
In cyber security, a whole industry has emerged in trying to predict when and where cyber threats may emerge. At the forefront of this trend are threat intelligence services, which have commoditised the supply of intelligence feeds purporting to identify imminent threats based on lists of malicious IP addresses that have appeared in other networks – the ‘back window’ that Drucker references.
Companies pay large sums to access these lists, hoping the addresses will provide valuable indicators that will help them take defensive action. With healthy scepticism, we set out to answer the question of whether these lists really have predictive power or whether companies paying subscriptions – often six-figures sums each year – are still driving in the dark on cyber threats.
IP lists – predictive or deceptive?
The idea behind malicious IP lists sounds really quite logical. If an IP address has behaved maliciously in one network, then it can be assumed that this same address will be active in multiple networks. After all, cyber criminals play a numbers game: the more corporate networks they target, the increased probability that they’ll find a vulnerability they can exploit.
Threat Intelligence vendors sell their services on this basis. They record malicious IP across a wide range of networks and consense them into a live watch list of IP addresses of which companies should be wary: if that IP shows up in your own threat detection logs, then they’re likely there with malicious intent. By giving companies access to the feed, they’re giving them the ability to take proactive measures to block the addresses and remove the threat.
Sound too good to be true? We thought it might be, but we wanted to test it empirically. Over the course of one month, we investigated 118,000 unique ‘suspicious’ events, involving 12,750 unique IP address indicators, seeking to establish the probability that an IP marked as suspicious would reappear as an IP indicator. The correlation, we found, was weak, with only 0.01% of suspicious IP addresses confirmed as malicious.
The inference we drew from this data is that malicious addresses have a short half-life. By the time the company processes the list against their own security logs, the cyber criminal will already have changed address. For a business to make these indicators actionable would therefore require a lot of resource-intensive data mining, with very low probability of picking up on threats.
A sticky solution
Based on our data, we can therefore conclude that the ROI of buying malicious IP address lists is dubious at best. Threat detection is a game of probabilities, and building your defences on infinitesimal odds is unlikely to move the needle in your favour. No matter how skilled your team is, or how strong your IT infrastructure, working with weak data isn’t a great use of resources.
So, in that case, what can enterprises do to get better value from their resources? This question prompted us to analyse an alternative option: honeypotting. To some, the logic behind honeypotting may seem counterintuitive. When seeking to defend a corporate network, our default goal is to ‘keep the attackers out’. Honeypotting turns this objective on its head and actually invites the attackers in. Of course, the network we’ve attracted them to is not the genuine network – it’s a simulation. Once they’ve entered the sandboxed environment we can track their activity and create profiles that can be used to prevent attacks on the main network.
To conduct our test, we set up and compared the relative value of a honeypot network versus a sensor network. What we found was that 37% of IP addresses observed on the honeypot were only observed on our sensor, while only 3% that were observed on our sensor network were observed on our honeypot network.
What these findings seems to suggest is that the honeypot networks a more useful predictor than an experimental feed.
Further, we found that around 1 in 7 of the profiles identified in the honeypot then showed up in the production infrastructure. From a threat detection standpoint, having a method with around 14% accuracy more firmly puts the balance of probabilities in your favour. Interesting, those IP addresses detected demonstrated absolutely no benign intent and practically no false positives.
The concept behind Threat Intelligence makes sense in theory, but it’s based on a fundamentally flawed premise: what an attacker does in network will be replicated in other environments. As tempting as it is to believe the predictive value of this assumption, in practice it’s really not the case. Cyber criminals typically go to great lengths to avoid detection, so activity in each network must be understood in context. In short, there’s no cut ‘n’ paste approach that will work across all networks.
Based on our research, we recommend that businesses think carefully about their threat intelligence approach. Simply purchasing a feed-based solution may seem like a ‘set and forget’ measure, but in reality it’s going to take a lot of manual work for you to find any value in the data.
Based on our research, we recommend that your organisation conducts internal cost analysis before adopting a threat intelligence solution, and gives due consideration to alternative approaches, such as honeypotting, that may yield better results. You may find you don’t need to spend over the odds to shine a light on the road ahead.
If your business needs help with threat intelligence services, get in touch with us here at Orange Cyberdefense. We aim to understand your business infrastructure to recommend the best solutions for you.