NSS Labs' Defense In Depth Testing Reveals Weaknesses In Traditional Layered IT Security Strategies; No Clear And Present Winners

Wednesday, July 10, 2013

Joe Franscella


At the end of May, NSS Labs (@NSSLabs) released its Correlation of Detection Failures report. In the accompanying news release, Are Security Professionals Overconfident in “Defense in Depth?”, the company said it was the culmination of a comprehensive in-lab test revealing that popular layered technology combinations are failing to protect organizations against exploits. Wrote NSS:

"In the past 18 months, NSS Labs tested the security effectiveness of typical defense technologies such as next generation firewall (NGFW), intrusion prevention systems (IPS), and end point protection suites (EPP -- also known as antivirus/malware detection) and found that there are significant correlations in their failure to block against known exploits."

As an FYI, NSS defines exploit as:

"An exploit is a piece of software, a piece of data, or a sequence of commands that takes advantage of a security vulnerability in software or hardware, in order to cause the target system to behave in an unintended or unanticipated manner. Exploits allow an attacker to gain control or to escalate privileges on the targeted system, or to render the target unusable through a denial-­‐of-­‐service (DoS) attack."

The news almost frames the research as an indictment of the entire IT security industry, especially when you consider that the lab reported that it:

"... looked at 606 unique combinations of security product pairs (IPS + NGFW, IPS + IPS, etc.) and only 19 combinations (3 percent) were able to successfully detect ALL exploits used in testing."

Interpretation: Only 19 out of 606 combinations are effective (3%) at providing some level of IT security. To borrow from pop-culture vernacular: Seriously? Only 19 combinations work? Say it in reverse and it is even more impactful, "587 combinations are failing."

The report’s key findings section was extensive, including bullet points such as:

--Security performance varies considerably between individual security products, or between combinations of security products. A comparison of the combined block performance of 606 unique pairs of security products revealed the performances to be wide ranging.

--The significant correlation of failures to detect exploits over a wide range of security devices particularly impacts the layered security approach, since the enterprise is inclined to overestimate the security effect of combining multiple protection technologies.

--There are some exploits targeting relevant software that are able to bypass detection by the majority of security devices or combinations of security devices.

--While it is helpful to adopt a layered approach to security, the real key to effective protection against threats lies in an organization’s choice of protection technologies to be combined.

The recommendations section was brief, with a compelling argument for SIEM usage:

--Organizations should assume they are already breached. Prevention should be paired with both breach detection and security information and event management (SIEM) to enable the prompt detection of successful security breaches.

I think it is worth mentioning that the testing appears neutral, with NSS stating:

--This analyst brief was produced as part of NSS Labs’ independent testing information services. Leading products were tested at no cost to the vendor, and NSS Labs received no vendor funding to produce this analyst brief.

However, the company’s client roster isn’t visible, so it is hard to say whether or not any of the major SIEM providers are customers.

Near as I can tell from a quick read of the accompanying report, NSS tested the security performance of 37 unique products from 24 different vendors, too many to list in this post but they of course included some of the most easily recognized and popular brands out there, @Barracuda, @Fortinet, @Sourcefire and @Palo Alto Networks just to name a few.

While the release and report make it clear on what isn't effective, it's not -- and it may be revealed as I review it further -- clear on which combinations provide the best defense against exploits tested.

Related Reading: Layered Security Approach Still Fails to Block Exploits

Possibly Related Articles:
Firewalls IDS/IDP Network Access Control Network->General Enterprise Security
Post Rating I Like this!
The views expressed in this post are the opinions of the Infosec Island member that posted this content. Infosec Island is not responsible for the content or messaging of this post.

Unauthorized reproduction of this article (in part or in whole) is prohibited without the express written permission of Infosec Island and the Infosec Island member that posted this content--this includes using our RSS feed for any purpose other than personal use.