Why QA Doesn't Do Security Testing

Thursday, January 06, 2011

Rafal Los

0a8cae998f9c51e3b3c0ccbaddf521aa

This post is inspired from a conversation recently had with one of the folks who shapes our security testing tools here at HP Application Security - thanks Sam for dislodging this nugget from the back of my brain.

QA teams have some interesting ideas when it comes to answer this question: "Are you doing any application security testing currently?"... and depending on who you ask it's possible you will receive a variety of different answers.

I am, of course, taking the assumption that you've accepted that security testing is as much a part of the QA testing cycle as oxygen is to breathing.

"Testing for security" can mean many things to many different people, and I wanted to take a moment to debunk some of the myths that I've heard spread over the last 2-3 years.  

I can't help but feel like Information Security organizations are at least partially responsible for the existence of some of these myths since we've done little or nothing to dispel them amongst our QA brethren.

  • Security Testing is already being done - Just because you're checking for the existence of the password requirement, or making sure pages aren't accessible without authentication doesn't actually mean you're doing security testing.  In reality, this is just a small part of the overall security testing that applications require, but it's the things that QA teams have become accustomed to doing, readily and quickly.  Most security testing that I've observed QA organizations performing is centered around session management, and account authentication (and in some rare cases authorization).  Very few QA organizations actually check for the types of issues that come from manipulating application logic, sending in garbage or malicious inputs, or attempting to break in to the application with malicious intent.
  • Security Testing is too hard [complicated] - Security testing can be quite difficult, if there is no expressive and well-established framework or process utilizing tools and documented, repeatable methodology.  It's easy to say something is too hard and simply not do it, that's one of the most common excuses in the world!  The way out here is for the security knowledgeable (the InfoSec or AppSec team) to establish clearly defined testing protocols, with easy to follow processes utilizing templated tools and "cheat sheets" where appropriate, and leveraging as much existing technology and process as possible.  The reason many QA teams complain about security testing is that some security teams try to force a whole new suite of testing tools and methodologies on the QA analysts - that's a sure-fire way to meet opposition.  In order to be successful it's critical to first understand the existing processes, tools and methodologies the QA teams use today and then adapt security testing protocols, tools and methods to fit within those existing processes and be minimally invasive or disruptive. 
  • Security Testing is not QA's job [that's security's job!] - Absolutely untrue.  Security is everyone's job, more explicitly - the QA organization is responsible for software quality - of which security is a component.  We here at HP preach about the three pillars of software quality: Does it perform? Does it function? Is it secure? ... is any one of those three questions is left unanswered the QA team is failing.  The other big argument to be made here is that QA gets the application much earlier than security teams, for testing.  and finding defects earlier is always a bonus.  Speaking of bonuses... catching software security defects, assigning them to a developer as a bug, and tracking them through fix,re-test and ultimately closed-fixed states is a thing that QA teams own almost exclusively.  I can't name many security organizations that can produce the types of KPIs that QA organizations can - KPIs that are absolutely critical to proving conclusive success or failure of a software security assurance program.  So yes, security testing is your job, QA analysts!
  • QA can't be effective at security testing [they're not experts] - A valid argument... if you completely disregard the fact that the application security testing tools market is maturing faster than our national debt.  The secret here is that QA analysts don't have to be security "hacking" experts if they have the right tools in place.  QA analysts don't have to be expert hackers, in fact, we'd prefer them not to be.  QA analysts just need to be able to use tools and follow process effectively to identify basic application security flaws.  The name of the game here is leveraging the scale of the QA teams, which is typically 30:1 all the way to 50:1 when compared against application security analysts.  If the testing and automation can be driven through QA analysts rather than taking up precious few cycles that the highly skilled application hackers have - this is a tremendous win because the security experts can spend their time validating and digging deeper into the applications, rather than doing simple testing.  The security teams just need to make sure that the tools and processes they set up for their QA analyst counterparts are templated and uncomplicated - so that the QA resources can effectively focus on what they do best - testing.
  • QA analysts don't understand Security Testing - Security testing is not altogether too different from functional testing, except in the details.  Testing is basically set up, execution, and interpretation.  If the set-up and interpretation is done by application security analysts the part that's left can be templated such that the level of understanding is minimal.  Let's be clear here - we're not expecting a QA analyst to be able to cobble together a complicated script to evade an anti-cross-site scripting library ...but we should reasonably expect that the analyst can either effectively use a tool, or follow a well-documented process that has varying tests and permutations allowing the analyst to think for themselves and flag questionable results for review by the security experts.  In reality, many QA analysts start to become very interested in hacking and security testing once they've been given the opportunity to learn and experience.  It's the lack of exposure that's currently hindering adoption and fueling this final myth.

So there you have it, 5 of the most common myths behind why QA analysts haven't been doing security testing in the QA cycle in most modern, mature enterprises.  Which excuse, I mean myth, do you subscribe to or perpetuate?

Isn't it about time to start dispelling these myths and start leveraging the power of QA organizations to bring better software security to our enterprises?  I think it is.

What do you think?

Cross-posted form Following the White Rabbit

Possibly Related Articles:
21943
Policy
Policy Software Security Testing Development QA Quality Assurance
Post Rating I Like this!
C787d4daae33f0e155e00c614f07b0ee
Robb Reck Great post. Thanks for sharing your experiences.
1294335677
The views expressed in this post are the opinions of the Infosec Island member that posted this content. Infosec Island is not responsible for the content or messaging of this post.

Unauthorized reproduction of this article (in part or in whole) is prohibited without the express written permission of Infosec Island and the Infosec Island member that posted this content--this includes using our RSS feed for any purpose other than personal use.