Pen Test vs. Vulnerability Scan: You know the difference, but do they?

Wednesday, November 28, 2012

Stacey Holleran


Small business owners often don't have the luxury of an on-site technical expert, let alone someone who is versed in network security. So when they are told they need a “network penetration test” to comply with the Payment Card Industry Data Security Standard (PCI DSS), many small business owners will contact one or more of the growing number of companies offering inexpensive testing services.

While the PCI DSS clearly states that a penetration test is required, the methods by which a penetration test can be performed are somewhat left to interpretation. Because of this, small business owners are often presented with the option of a low-cost, "automated" penetration test. Information security professionals know that there is no such thing as an automated penetration test; however, the small merchant doesn't know this and will most likely abandon the manual penetration test in favor of the less costly alternative.

Those of us working to help small merchants secure cardholder data and comply with the PCI DSS often encounter the scenario described above. If you run up against this problem, it can be helpful to have a handy list of talking points. Jarred White, CISSP, PCI-QSA and Penetration Tester, recently posted some excellent talking points on PCI Compliance Guide, and I thought I’d share them with you here


What Constitutes a Penetration Test?


  • Penetration Tests are Always Manual: A penetration test is characterized by a person at a computer behaving as a hacker would, running a series of manual, simulated attacks against your information systems. Sure, there are automated elements to penetration testing (after all, hackers are smart; they leverage automated scripts and tools to quickly and efficiently gather data), but the test is orchestrated and driven by a real human trying to break into your network and its applications. This is important because information discovered during the various phases of testing must be intelligently fed back into the testing methodology—something that computers aren’t very good at doing.


  • Penetration Tests Leverage Professional Experience: While automated scripts and scanners are great at efficiently identifying “low-hanging fruit,” one noticeable and important trait they lack is experience-led logic. An experienced penetration tester can quickly identify the systems, services and configurations that present possible vectors for attacks, while automated vulnerability scanners rely on a pre-compiled list of signatures, or fingerprints, in order to detect vulnerabilities and vectors of attack. In addition, an automated scan is looking for very specific things, while a human being is free to apply creativity, look at the big picture and consider past experiences and findings that may lead to the detection of issues that a scanner won’t find.


  • Penetration Tests Have Unique Methodology:Penetration tests simulate the very real dangers of an intelligent human being actively attacking your systems and trying to bypass your countermeasures. A vulnerability scan or other automated “attack” against a network is not the same as a penetration test because it cannot adhere to the same methodology. Therefore, many PCI Qualified Security Assessors (QSAs) will not recognize automated penetration tests (a.k.a. vulnerability scans in disguise) as valid for compliance with PCI DSS Requirement 11.3.


  • Penetration Tests Yield Specialized Reports: The report resulting from your test should be written by a human being, not auto-generated through a computer program. The professional penetration tester’s goal is to give your organization the detailed information it needs to successfully secure all in-scope business systems. Therefore, the written report is tailored according to your organization’s unique risks and includes the testing methodologies used as well as any issues discovered, assessment of each issue’s level of risk and recommendations for addressing those issues.


Stacey Holleran is a Sr. Manager at ControlScan, a provider of Payment Card Industry (PCI) compliance and data security services headquartered in Atlanta, Georgia.


Possibly Related Articles:
PCI DSS Network->General
Information Security
Compliance Enterprise Security Penetration Testing pentest
Post Rating I Like this!
Marc Quibell Good article
Stéphane Bergeron Totally agree with you.
Mary Addo Nice write up
Tom Coats Nicely summarized
Ian Tibble Yes, good article and thanks. I think we're getting closer to stage 1 of the "awakening" of infosec, whereby there is growing realisation of the ineffectiveness of the unauthenticated scanning method. The next will be the ineffectiveness of any kind of remote unauthenticated scanning, regardless whether it's manual or automated.
The views expressed in this post are the opinions of the Infosec Island member that posted this content. Infosec Island is not responsible for the content or messaging of this post.

Unauthorized reproduction of this article (in part or in whole) is prohibited without the express written permission of Infosec Island and the Infosec Island member that posted this content--this includes using our RSS feed for any purpose other than personal use.