OSSTMM v3 From A Client's Perspective

Monday, December 13, 2010

Rod MacPherson


As a new contributor to ISECOM's efforts I was granted a sneak peek of the upcoming OSSTMM 3.

I should start with a little background on myself. I work for a small municipal government in Ontario, Canada. We do take credit cards at our recreation facilities, so I've been tasked with co-coordinating our PCI-DSS effort.

I am not a professional penetration tester or an auditor, I'm a "Network Analyst", which in my case means I manage and maintain the firewalls, switches, Internet DMZ servers, security products, and help manage our VM environment and SAN.

But the building and testing of information systems defenses is a really big interest of mine, and over the past several years my role in the Town has moved more and more into the security realm. Operational Security is the heart of my job, and as OSSTMM is all about testing OPSEC, it is of great interest to me.

My involvement with ISECOM is that I'm trying to get involved with the development of the "Smarter, Safer, Better" awareness program. I'm on the mailing list, but at this stage I have not yet contributed. Anyway, I think that's enough background for you readers to feel you have a grasp of where my biases and limitations in knowledge might be... lets get on with the meat of the article.

OSSTMM 3 weighs in at nearly twice the number of pages as the previous version, but there is good reason, and readers will appreciate the new additions. Compared with the older version 2.2 OSSTMM, the writing and layout have been improved greatly. It now more clearly expresses the ideas that make OSSTMM work.  OSSTMM 3 is much more readable, and entertainingly written than previous versions.

It also should be relevant longer than methodology documents like NIST-800-115 which make reference to specific tools that may be outdated over time. OSSTMM doesn't specify tools to use, but rather looks at concepts such as channels, vectors and attack surface in more generic ways that will still be applicable as technology progresses,  and in fact, don't necessarily have to be applied to technology.  As the way we do things changes, the security of the system will still be measurable using OSSTMM metrics and we will still be able to compare those results to past results.

One new addition that I found particularly useful is the new Chapter 1, which gives definitions for terms used throughout the rest of the document. Like most science and technology fields, different groups use different terms to express the same ideas, and often the same terms to express different ideas, so starting off by clearly defining the key terms is important.

While OSSTMM 2 did reference an online glossary for definitions, I feel that bringing well written explanations of key terms, especially those that we all know are used in different ways at times, was a big step forward in improving understanding between ISECOM and new readers of this material.

Further definitions, just as in the previous version, are scattered throughout the document, placed in chapters where they are referenced. However, all of the important stuff that you need to understand before getting into the process of testing and measuring security, is up front in Chapter 1.

With both the new definitions chapter, and a more easy to read style of writing, I found it much easier to read through OSSTMM 3.

Now, what makes OSSTMM interesting to me?

I have never worked as a professional Penetration Tester (although I see my career potentially headed in that direction), nor have I ever hired one, but I'm about to. I'm currently co-lead on the Town's PCI-DSS compliance project.

In this capacity I will soon be required to choose which contractor we will hire to fulfil the PCI-DSS 11.3 pentesting requirement. I've personally got a preference for someone who uses the OSSTMM (preferably 3.0 for ease of comparing results to future tests) because there are several nice advantages for the Town as a client.

The first big advantage is the Rules of Engagement.

I can't think of anything I dislike about the Rules of Engagement. I would be in heaven if every vendor I dealt with held to even half the Rules. I could especially do with not being fed FUD, or a list of past clients who's engagement had little in common with what I'm looking to hire the vendor for.

Another is simply the fact that the methodology is well documented which is a requirement of PCI.

Yet another advantage is repeatability. If we are going to test, I'd like to be able to have another, independent, consultant be able to repeat the test at a later date to see if our state of security has changed at all. OSSTMM's risk assessment values (ravs) allow us to do that reliably.

OSSTMM testing doesn't make any risk analysis. I like that because it is not up to a contractor to do that, it's not even up to me to do that, that's a job for our senior management. What the test should do is provide unbiased hard data that they can use to make risk decisions. We aren't looking for an opinion on what we should be doing (the PCI council is already giving us enough opinion on how we should run things) what we need from a pentest is measured results, facts and hard data.

I like that the OSSTMM asks for the analyst to provide a transparent report including things like unknowns, untested targets, limitations verified during the test, false positives, etc.

Even if I don't get my wish and get to bring in a contractor that uses the OSSTMM for testing, I feel that the OSSTMM is useful to me, as a client, in preparation for the testing.

 I like that I can take the OSSTMM and meet up with my team and use it to define our assets, engagement zone, scope, vectors, and the test type we want before even meeting with the vendors. This helps to define exactly what we want so that there is an opportunity to compare potential vendors fairly.

The new "What You Need to Know" chapter (Chapter 1) will make it much easier to take a group of both computer people and finance people with a wide variety of past experience, probably none of them having done this before, and come away from the meeting having made some progress, and at least having everyone working from the same set of definitions.

In the realm of government, even at the local level, buying a product or hiring a consultant/service provider often has to be done through tenders or RFPs where it is important that we define clearly, exactly what we want before opening it up to bids from the vendors who want to provide that product or service. Having something like the OSSTMM to help define our needs explicitly is a huge benefit.

Possibly Related Articles:
Methodologies Penetration Testing OSSTMM ISECOM Standards
Post Rating I Like this!
The views expressed in this post are the opinions of the Infosec Island member that posted this content. Infosec Island is not responsible for the content or messaging of this post.

Unauthorized reproduction of this article (in part or in whole) is prohibited without the express written permission of Infosec Island and the Infosec Island member that posted this content--this includes using our RSS feed for any purpose other than personal use.