AV-TEST places Cisco Umbrella first in security efficacy – again!

AV-TEST places Cisco Umbrella first in secure web gateway to protect remote workers

With so many employees working remotely during the global pandemic, businesses are having an even harder time keeping their users, devices, networks, applications and data safe. According to Ponemon Institute, organizations who believed they were effective at mitigating risks, vulnerabilities and attacks across the enterprise declined from 71 percent before the pandemic to 44 percent after the pandemic.¹ The “new normal” increases the importance of moving to a cloud-delivered security model with high efficacy to stop threats from exploiting the risky behavior of remote workers. Lots of vendors claim to block and detect threats, but only one vendor stands out as the industry leader in threat detection for the second year in a row!

AV-TEST places Cisco Umbrella, the heart of Cisco’s SASE architecture, first in security efficacy in a recent test. Cisco Umbrella is a cloud-native security service that simplifies network security by helping you secure internet access and control cloud application usage across your network, branch offices, and roaming users. Umbrella unifies DNS-layer security, secure web gateway, firewall, and cloud access security broker (CASB) functionality. Umbrella integrated with Cisco AnyConnect provides secure endpoint access to the network so employees can work from any device, at any time, in any location. 

Umbrella received top marks across the board, with a whopping 96.39% total detection rate, crushing the competition.

AV-TEST evaluated Cisco Umbrella’s secure web gateway (enhanced with DNS security) and DNS-layer protection functionality. Umbrella received top marks across the board, with a whopping 96.39% total detection rate, crushing the competition. Umbrella also demonstrated a significantly lower false positive rate than other products, helping employees to stay productive while making security analysts more efficient and less likely to miss real threats. And, while we don’t like to brag, this data is too good to keep quiet, especially since this is the second year in a row that AV-TEST has found that Umbrella outperforms competitive offerings.

Umbrella places first in 2020 cloud security efficacy test 

In September and October 2020, AV-TEST performed a review of Cisco Umbrella’s secure web gateway and DNS-layer security functionality, alongside comparable offerings from Akamai, Infoblox, Palo Alto Networks, Netskope, and Zscaler. The test was commissioned by Cisco to determine how well vendors protected remote and roaming workers against malware, phishing sites, and malicious websites. AV-TEST also carried out a false positive test against known clean popular websites and downloads from Alexa’s top list.

AV-TEST is an independent research institute for IT security based in Germany. For more than 15 years, cybersecurity experts from Magdeburg have guaranteed quality-assuring comparison and individual tests of virtually all internationally relevant IT security products. 

About the test

In order to ensure a fair review, research participants did not supply any samples (such as URLs or metadata) and did not influence or have any prior knowledge of the samples tested. All testing methodology engaged was solely AV-TEST’s.  All products were configured to provide the highest level of protection, utilizing all security-related features available at the time. The test focused on the detection rate of links pointing directly to portal executable (PE) malware (e.g., EXE files), links pointing to other forms of malicious files (e.g., HTML, JavaScript) as well as phishing URLs. The test included a total of 3,572 malware samples.  

Secure Web Gateway Test

First, the lab test assessed each vendor’s secure web gateway functionality, specifically the ability to protect roaming and remote workers. Given that the global pandemic has accelerated the move of edge security controls to a cloud-delivered model, each vendors’ secure web gateway functionality was configured with the protection of their roaming agents on the devices tested.

A secure web gateway is based on a full web proxy that sees and inspects all web connections. Unlike DNS-layer protection which only analyzes domain names and IP addresses, a web proxy sees all files and the full URLs enabling more granular inspection and control.  For secure web gateway testing, the products achieved the following blocking and false positive rates (ordered by best detection rate): 

DNS-Layer Protection Test

DNS-layer protection uses the internet’s infrastructure to block malicious and unwanted domains, IP addresses, and cloud applications before a connection is ever established as part of recursive DNS resolution. DNS-layer protection stops malware earlier and prevents callbacks to attackers if infected machines connect to your network. DNS-layer protection with selective cloud proxy redirects only risky domain requests for deeper inspection of their web content, and does so transparently through the DNS response. For the DNS-layer protection testing, the products achieved the following blocking and false positive rates (ordered by best detection rate): 

Note: Netskope, Palo Alto Networks and Zscaler do not have comparable DNS-layer protection offerings that add security to the recursive DNS process and policies that can be configured with a secure web gateway (or parallel offering).

Key Takeaways

Cisco Umbrella protects roaming and remote works best

In both test scenarios, Cisco Umbrella outperformed offerings from other vendors. In the secure web gateway test, Cisco Umbrella’s secure web gateway functionality (layered with DNS security) performed best and demonstrated a higher threat detection and lower false positive rate than other solutions. In the DNS-layer protection test, Cisco Umbrella functionality clearly outperformed competitors in malware and phishing protection as well as in false positive avoidance. 

In some cases, DNS-layer protection is sufficient as it’s fast to deploy and provides a good base layer of security. Since many potential attacks can be blocked efficiently at the DNS-layer before a connection is even established, securing at this initial stage is vital to securing your business. When a connection is blocked at the DNS-layer, the attack stops there which reduces the security burden on your security teams and security tools. Other cases require a secure web gateway for a deeper set of controls and a stronger level of protection. Sending all traffic to a full proxy gateway significantly improves overall security posture. As seen in the results, combining secure web gateway with DNS-layer security provides the highest level of security efficacy.

Efficacy matters 

The shift toward workforce distribution has accelerated the need to protect users anywhere – at home, on the go, on vacation, at a campus office, and on any device. A recent Cisco survey found that 61 percent of organizations globally have experienced a jump of 25 percent or more in cyber threats or alerts since the start of the pandemic.² It only takes one malicious threat to compromise your business. If your security is not effective your business is not protected. Don’t settle for second-rate security. Cisco Umbrella, the gold standard in security efficacy, can help you ensure a worry-free, secure and effective defense with low latency. 

For more information on specific configurations and detailed test results, click to read the full report by AV-TEST.

¹ Cybersecurity in the Remote Work Era: A Global Risk Report Ponemon Institute, October 2020
 ² Future of Secure Remote Work Report, Cisco