Following the White Rabbit Blog : quality

StarWest - Where QA and Security Will Collide

RafalLos — Mon, 06 Jul 2009 18:15:00 GMT

Is site security QA's problem too?!

Hi everyone, I can't wait for fall and the StarWest testing conference in Anaheim! I'm so psyched to be presenting "QA Techniques for Identifying Workflow-Based Security Defects" in what will hopefully be one of the better talks of the week. I've been promising many of you an explanation of why QA and IT Security cannot live without each other, as it pertains to web app security, and I aim to deliver.

This talk will be heavily focused on the reasons why IT Security still fails in many instances to find serious web application security defects - and what the Quality Teams can do about it. How about that... identifying security as more than just "security's problem" - it's an enterprise-wide problem that bleeds very much over into the QA testing organization. The days of the security teams doing "scans" and pitching the results over the cubicle wall to the developers are long, long over (were they ever really here?) and the days of collaborative defect mitigation throughout the application lifecycle are here. Come listen and learn some of the techniques that the QA testing teams can use to identify security-based defects in the web applications; and understand why it's not just security's problem anymore.

But wait! there's more! Just in case you're thinking to yourself... sure I'd love to go but I don't think I have the travel budget - there's a discount code yours truly has gotten made up just for you, my readers! Simply follow the instructions below - and I'll see you in Anaheim at StarWest.

Simply click this link: ( http://www.sqe.com/go?SW09Rafal ) and enter the code SWRL to get your discount...

Here's the abstract - just to get you psyched up too!

"Workflow-based web application security defects are especially difficult on enterprises, because they evade traditional, simple, point-and-scan vulnerability detection techniques. Understanding these defects, and how/why black-box scanners typically miss them is the key to creating a testing strategy for successful detection and mitigation. Rafal Los describes the critical role that application testers play in assessing application workflows and how business process based testing techniques uncover these flaws. Rafal demystifies the two main types of workflow based application vulnerabilities: business process/logic vulnerabilities and parameter-based vulnerabilities. As the complexity of web applications continues to increase, learn how to adjust your testing strategy to make sure you don’t miss these unique types of defects."

Quality Engineers & Testers - StarWest is Coming Up!

RafalLos — Thu, 02 Jul 2009 20:45:00 GMT

I'm thrilled to announce that I have been selected to speak at the StarWest 2009 Quality Conference (SQE) October 5-9th 2009, hosted at the DisneyLand Hotel in Annaheim, CA! Link to the conference website is here (http://www.sqe.com/starwest/Schedule/Default.aspx) and there are a number of awesome speakers as well!

The StarEast conference was chock-full of great speakers, vendors and of course yours-truly... speaking on Security topics and why the quality assurance teams are so crucial to the web application security process. That's right, I've been talking about Q/A engineering and testing teams and why they're so crucial to the success of any enterprise web application security program - but now for the first time you'll get the truth that the IT Security guys probably won't tell you - YOU are the key! My talk on this topic promises to be riveting and will certainly have an impact on formal testing and security organizations...

As an added bonus - if you sign up you'll get money OFF the price of your admission!

Normal 0 false false false EN-US X-NONE X-NONE

Register using special promo code SKWS and save up to $300! Register by September 4^th to add the Early Bird Discount for up to $600 in total savings! Call the client support group at 888.268.8770 or register online at: https://www.sqe.com/starwest/Register/SelectConference.aspx

I'll see you all there!

Defining Security as a Business Requirement

RafalLos — Thu, 05 Feb 2009 04:53:00 GMT

This post is a follow-up to the previous one on QA: Defect vs. Vulnerability. All the highly-intelligent responses I received got me thinking further, and so here I present my additional thoughts.

This may not be revolutionary - but given the response I received regarding the terminology difference between defect and vulnerability I think the only logical conclusion we can reach is that if security is not a foundational business requirement, we're sunk.

To expand on this point a little more I think it's important to follow non-technical critical-thinking here. Anything that does not make it into the functional specification of an application [web or otherwise] is an afterthought. It has been conclusively [and repeatedly] proven that anything that is not "baked in" as a requirement is nearly impossible to fix later on, as an after-thought. So we're presented with a puzzler. Security must be a business-level requirement. So how then does one translate vulnerabilities into a business requirement, sanely? Simply stating "... the application shall be free of unintended design flaws and security vulnerabilities" is like asking an architect to build a structure that will withstand every known (and unknown) possible attack - it's simply illogical.

Strangely, program leads that manage these large-scale web applications at the heart of nearly every major breach want concise, identified things to not put into the code... but since that list is a moving target the security team gets penalized for the nature of security itself. This is the reason why black-listing input is a losing proposition... you're always going to be in an arms race with the bad guys... and you'll never win.

I've heard some recent conversations hit the wire around using the CWE Top 25 or some other list as a definitive list of coding errors to avoid in web applications but I'm not sure if that will actually solve the problem. The problem with this approach is and will be that these lists are exclusionary measures. These lists illustrate what we must exclude to be [more] secure. Turning it around and making statements like validate all input makes little more sense, especially given that input validation must be defined in the context of the situation, and there is never a one-size-fits-all answer. To illustrate the point further - input validation may mean excluding certain character sets/patterns and pre-defining acceptable input options ... but this does not account for things like free-form input or other use-case specific examples.

In the end, the crux of the problem lies in the nature of security vulnerabilities. Security vulnerabilities are a moving target and although they can be loosely defined and lumpted into Top 7/10/25 lists it is not logical to consider these lists complete or even functional for designing software. Will a web application be secure if it follows the CWE Top 25 and addresses those issues? What about the OWASP Top 10? I don't think anyone has that answer, or at least is willing to stake their reputation on it.

So back to defining security as a business-level requirement... can it be done? Can one clearly articulate requirements to secure data/transactions/processes/whatever *before* the technologists get involved; meaning, before the means to execution are defined? I will leave that up for debate.

"Security Vulnerability" != "Defect" ; why?

Rafal Los — Tue, 01 Apr 2008 11:18:00 GMT

It's one of those obvious things. A defect is a defect, right? Whether the airbag is faulty, or the gas cap doesn't hold pressure... a defect is a defect. The strange thing is - it hasn't been that way, and still isn't that way, in most of the IT shops I've been in. Why?

The reason is simple. Historically, security vulnerabilities have been in a class all their own. In an attempt to put some urgency to the matter, security professionals have labeled defects in the security of their projects (in this case I'm talking about web applications) as an entirely different thing than a functional defect. What we didn't realize is that we were actually doing a dis-service to ourselves and the security cause. You may not agree with me right now - but I'll explain this more clearly, and I think you'll be on board with my thought process.

Let's talk about defects, in general and then apply it to the matter at hand. First, let's identify what a defect is... A defect is, in the dictionary sense (cut from dictionary.com):

de·fect [n. dee-fekt, di-fekt; v. di-fekt] Pronunciation Key - Show IPA Pronunciation

–noun

1.	a shortcoming, fault, or imperfection: a defect in an argument; a defect in a machine.

2.	lack or want, esp. of something essential to perfection or completeness; deficiency: a defect in hearing.

OK, easy enough right? So the first meaning is clear; a defect is a shortcoming, fault, or imperfection. It is reasoned that a defect in a web-based application results when functionality X doesn't work as required. Say you have a button, and the functional specification (we'll get back to this gem in a minute) calls for the button to perform some action, A. During the testing phase of the application, before release to production, a tester or tool is utilized to test the functionality of that button, but instead of action A happening, some other action B happens. This is a defect. There is no doubt in anyone's mind that this immediately gets classified as a defect, put into the defect tracking system and sent back to the developer for remediation. The defect is classified as a higher-priority defect if the function happens to be one that is showcased, or important to the overall functionality of the application. Those of you that already use the HP Quality Center tools know exactly what I'm talking about, and know how this process works. Here's the strange twist though - why is quality testing only done with good data? I understand that you want to make sure that the test cases work properly - but why are the testing options limited? The issue at hand here is a very narrow view of defects, and defect testing.

Back in college, I took very basic programming class and had to write a program that was a calculator. It would ask for two inputs of numbers, and then give you an option to perform either an addition, subtraction, multiplication or division of the inputs. Generally, it was assumed that these would be numbers, but what if they weren't numbers? Most of the students in the class, myself included, never thought about ... "What if someone enters a letter or some other unexpected input?" Well, luckily, the professor chose my application, put it up on the screen for the whole class to see, and promptly entered a and b for the two inputs and tried to add them. When my application core dumped, he explained to the class why I had gotten my first F on a project. I learned a very valuable lesson that day - developers must brace their applications for unexpected input. "Why would anyone want to enter something other than numbers?" wasn't a good enough answer to explain why my application failed. Let's apply this lesson I learned back in college to today's application programmers and functional testers.

Here are the reasons why I think security vulnerabilities aren't seen as "defects" in general...

Security professionals have insisted that a vulnerability is its own separate category - While it is true, some security vulnerabilities are a whole new level of "bad" they should be considered just like any other defect in the application for the sake of tracking and remediation. Web platform managers are generally concerned with meeting the demands of their customers and producing code that is defect-free - and it's our own fault that "vulnerabilities" of the security variety have become some ethereal, magical issue for security nerds to worry about. This matter can only be fixed by changing the naming back... a vulnerability is a defect, period. Security vulnerabilities must be explained as "high-criticality defects" to developers, managers and customers otherwise this situation will never change.
Functional specifications rarely, if ever - call for for security validation - Functional specifications aren't written by security professionals, generally. At best, security professionals have a chance to review the functional specification way too late into the process, while the code is being written and readied for production. This is, once again, our own fault most of the time. The answer to this dilemma is a two-pronged attack. We as security professionals must educate those that write functional specifications, and enlighten them to the need for security features. At the same time, we must work hard to have an active input in the writing and release of functional specification documents. These two vectors are critical to getting secure code as an end-product.
Programmers don't typically think about malicious users - While it would be great to think that one day all developers will inherently write secure code because they will have "learned" what security is, and how important it is to the application - the fact is that it's a pipe dream. Developers care about one thing... meeting functional specifications in the least amount of time possible, and moving on to the next project. Developers like to write optimized code that accomplishes the required tasks in as little time as possible. Solving #2 above will also partly solve this problem. In addition, developers must be given the tools (such as static and dynamic code analysis tools as plug-ins to their IDEs) to make their jobs easier. It is not reasonable to expect developers to be security experts in all aspects, so we must arm them with the tools to be experts, without having to do too much extra work or they won't use those tools.

So what have we learned today?

Security vulnerabilities must be re-classified as easily-understandable "functional defects"
Funcitonal specifications must be written to include provisions for security validation
Quality professionals must be given the tools to test for "security defects" in web applications to close the loop in the lifecycle
Developers must be educated and also given the tools to write more secure code with minimal additional effort

-- I welcome your comments!