To Pen Test or Not to Pen Test, That is the Question...

Sunday, October 16, 2011

Andrew Weidenhamer


Time and time again I am challenged by clients and security professionals alike on what is the real benefit of penetration testing.

Though this seems like an age old debate with many famous hackers and security professionals weighing in, I am not entirely sure I understand the argument undermining the importance/benefits of penetration testing.

Below are arguments from both black and white hat security professionals:

White Hat – “Pen testing can show one of two things: your security sucks or your security is better than your pen tester”

Black Hat– “The very concept of "penetration testing" is fundamentally flawed. The problem with it is that the penetration tester has a limited set of targets they're allowed to attack, while a real attacker can attack anything in order to gain access to the site/box. So if a site on a shared host is being tested, just because is "secure" that does NOT in any way mean that the server is secure, because could easily be vulnerable to all sorts of simple attacks. The time constraint is another problem. A professional pentester with a week or two to spend on a client's network may or may not get into everything. A real dedicated hacker making the slog who spends a month of eight hour days WILL get into anything they target. You're lucky if it even takes him that long, really.”

Though I understand the point of the above arguments, I believe the logic behind these statements is fundamentally flawed. The point they are trying to make is that an organization is never going to be entirely secure and that an attacker with dedicated time and resources WILL in all cases break in, so performing a pen test is only validating something already known. I see penetration testing differently. Penetration testing is not designed to make an organization 100 percent secure but to make them MORE secure (assuming identified vulnerabilities are remediated) and MORE aware what they were before the penetration assessment was performed. It also is a good means to test current logical and physical security controls. From my experience, many security professionals responsible for an organization’s security do not understand the full ramifications of vulnerabilities and thus can become complacent in fixing them.

Case Study:

A vulnerability scanner returns one vulnerability on Company A’s external presence. The vulnerability identified was Cross Site Scripting or SQL Injection. A report is issued to Company A showing a “High” risk rating based on this vulnerability.

Company A may not understand how a single vulnerability can translate into a “High” risk rating and thus chooses to ignore or at least delay remediating this vulnerability until time and resources become available. Does this mean Company A’s security is bad? I would say if that if Company A had an external presence of 50 servers and 10 applications, and only one vulnerability was identified, the answer would be no.

Company A may very well have good security, but as with everything else in life, mistakes happen. Now let’s assume a penetration assessment was performed on Company A’s external presence instead. Not only would the penetration assessment identify this vulnerability, it would attempt to exploit it.

Let’s go on to say that this vulnerability is in fact exploitable and allows for full system compromise. What is a client going to react to more? A report stating they have one critical vulnerability stating what could happen or a report stating they have one critical vulnerability and, oh yeah, by the way we compromised your entire domain controller?

A client who just had their entire domain controller compromised is going to be more inclined to fix the vulnerability in a timely manner than one reading a report stating what could happen if not fixed.

How can one argue that this penetration assessment was not beneficial to Company A? It effectively made Company A more aware as to the dangers associated with a critical vulnerability, which in turn made them take a proactive approach to fixing the problem almost instantaneously, thus reducing their overall risk rating.

This is one simple example. I can go on and on about the benefits of penetration testing. Security is about managing and reducing risk to an acceptable level. A penetration assessment isn’t intended to reduce an organizations risk to zero percent, but then again neither is any security assessment.

Any time an organization connects a device to a network it assumes a certain amount of risk. It’s understood that zero-day vulnerabilities will always surface and cannot be prevented.

So sure, a dedicated attacker could decide to spend 6 months developing an exploit for an unknown vulnerability, however this is going to take a more sophisticated attacker which makes this a less likely scenario.

A penetration assessment is simply used as a means to identify vulnerabilities and provide proof of concept examples on exploiting these vulnerabilities. By doing so, it effectively better explains ratings associated with vulnerabilities which in turn produce much more conscious/aware security professionals.

A much more aware security department will be able to better help reduce the overall risk for an organization.

Possibly Related Articles:
Information Security
Penetration Testing Network Security Ethical Hacking Remediation Pentesting Case Study
Post Rating I Like this!
Craig S Wright @Lance, I have been doing a review that I will be publishing on this topic (a followup from a paper being published in Dec).

Basically, 93% of high level issues were rectified in 12 months. What is interesting is when.

The analysis showed that over 60% of the fixes were done as the following test/audit came due. That is, the fixes for the last year test were driven by the upcoming test.
Ray Pesek I use them to determine whether we could detect an attack in a timely manner. We look at the logs and other indicators and make adjustments to the various devices and sensors and then have a re-rest. A good pen test is when we have no indications it took place in the downstream logs.

I may not be able to remediate something as fast as I want for some reason or another but if I can raise our awareness I'm at least a bit better off. And I can usually make these kinds of changes while the original test is in progress.
Michael Farnum I was going to ask you to define the term "penetration test" until I saw you say "Not only would the penetration assessment identify this vulnerability, it would attempt to exploit it." That is the key. Unless you try to actually exploit, I have a problem calling what is essentially a vulnerability test a penetration test.

Good article and something people need to listen to. Not only does this identify weaknesses, it is acceptable in the business world.

Acting as a real attacker with a slow and go methodology is wonderful, IF IF IF you can get the business to pay that kind of bill.
Andrew Weidenhamer Although a Penetration Assessment can be a good way to test logging and monitoring controls, to Michael's point, there is typically an amount of time that is associated with a 3rd party vendor's penetration assessments. An attacker, has unlimited amount of time and would be able to go as slow and low as they wanted. Even if you were able to detect a real attack and then block that attack, a real dedicated attacker would simply try again, simply being more stealthy the next time.
Ray Pesek Yep, the operative words being "... try again". We need to latch on to every opportunity to detect someone malicious regardless of how and why they do it (pen test or real). Once we start ignoring or writing off minor anomalies, we're setting ourselves up for a breach. If you can protect yourself from the "high and fast" attacks, hopefully you'll pick up _something_ occurring because of a low and slow.

Fortunately "high and fast" works so often that there aren't that many good low and slowers. :-)
The views expressed in this post are the opinions of the Infosec Island member that posted this content. Infosec Island is not responsible for the content or messaging of this post.

Unauthorized reproduction of this article (in part or in whole) is prohibited without the express written permission of Infosec Island and the Infosec Island member that posted this content--this includes using our RSS feed for any purpose other than personal use.