Following the White Rabbit Blog : software quality

Is Anybody Listening?

RafalLos — Thu, 15 Oct 2009 16:22:00 GMT

Greetings, I am finally back home after an exhausting trip which had me speaking at 2 conferences back-to-back in separate countries and on opposite side of the coast! I did learn some valuable lessons from speaking at these two wildly different conferences thought, so I thought I would share them with you here for your benefit too.

First off, the Information Security conference I attended on Tuesday in Toronto called "SecTor" was brilliantly run and targeted towards Canadian-based information security professionals and wanna-be security professionals. It's OK to say it, there are plenty of people that attend these conferences who are looking to break into the business and want to learn about information security enough to get a grounding of what the industry is about... so they attend these conferences. My talk "When Web 2.0 Attacks" was well-attended and I even had some big names in my audience (thanks to RSnake, Hoff and a few others that wandered in and out) and I think the overall impression was that the stuff I presented was relevant to people's daily lives in Information Security. That's kind of the problem though...

You see, while I ordinarily wouldn't think twice about educating those in my field ... someone that's been doing this for a while longer than I reminded me a while back that this is what we would call "preaching to the choir". Sure, I tend to agree that even within Information Security not enough people understand Web App Sec well enough to build a program and actually reduce any real risks - but those folks have been hearing this talk for years upon years right? At some point I'm bound to hit the law of diminishing returns; and furthermore, people who didn't agree with me 6 months ago aren't likely to agree with me today. Great conference, great mind-share but it's definitely time to reach a broader audience.

That's where the next conference I spoke at comes in. Wednesday morning, at 4:00am Central time (yea, AM) while some of my colleagues were stumbling into their hotel rooms in downtown Toronto I was hopping into a car and being driven to the airport to head out west. My destination was Anaheim, CA where I would speak at StarWest later that day. I'm still not sure how through the delayed flight, sickness, and almost-missed connection I made it out to the West Coast by 2pm, but I did... and Star West was awesome.

StarWest (run by the SQE folks (www.SQE.com) is nicely put together and serves an entirely new audience of people. Here at StarWest (although I did find it strange that we were in the heart of DisneyLand!) the audience was almost entirely composed of software test engineers, managers and those related to the field. This was a completely different set of ears than what I'm used to ... this was a good thing.

The first thing I heard when I put my welcome slide up was "Hey, isn't security supposed to be done by the security people?" Love it. This is exactly the mentality and walls I was there to break down. I think as we went through the hour-long session on "Detective Work for Testers..." I managed to convince a few people in the audience that their jobs were closely tied to mine in Information Security. Maybe, maybe not. The bottom line is that there were many great folks who came up to me and talked afterwards and through the end of the conference about the absolutely missing component in their SDL that was security. I had one lady in the audience (although she fled before I could get more out of her, and had to track her down myself later on the show floor) tell me that her security team is the developers and that because they tell the bosses that they don't have security issues no one ever tests the code. I wish I could recall where she worked, hopefully no place important like a bank or anything ...

The point is - this was the right audience. If you were there and came to my talk, awesome! If you missed it, slides are posted and we can talk about it whenever you have some time.

Do you believe that Information Security and Software Quality testing is one and the same? Do you believe that a quality defect may as well be a security defect? Can you successfully explain the difference between a security and quality bug?

... I'm fairly sure I have my target audience for the next foreseeable future. Listen up quality testers - I'm coming to a conference near you!

Defining Security as a Business Requirement

RafalLos — Thu, 05 Feb 2009 04:53:00 GMT

This post is a follow-up to the previous one on QA: Defect vs. Vulnerability. All the highly-intelligent responses I received got me thinking further, and so here I present my additional thoughts.

This may not be revolutionary - but given the response I received regarding the terminology difference between defect and vulnerability I think the only logical conclusion we can reach is that if security is not a foundational business requirement, we're sunk.

To expand on this point a little more I think it's important to follow non-technical critical-thinking here. Anything that does not make it into the functional specification of an application [web or otherwise] is an afterthought. It has been conclusively [and repeatedly] proven that anything that is not "baked in" as a requirement is nearly impossible to fix later on, as an after-thought. So we're presented with a puzzler. Security must be a business-level requirement. So how then does one translate vulnerabilities into a business requirement, sanely? Simply stating "... the application shall be free of unintended design flaws and security vulnerabilities" is like asking an architect to build a structure that will withstand every known (and unknown) possible attack - it's simply illogical.

Strangely, program leads that manage these large-scale web applications at the heart of nearly every major breach want concise, identified things to not put into the code... but since that list is a moving target the security team gets penalized for the nature of security itself. This is the reason why black-listing input is a losing proposition... you're always going to be in an arms race with the bad guys... and you'll never win.

I've heard some recent conversations hit the wire around using the CWE Top 25 or some other list as a definitive list of coding errors to avoid in web applications but I'm not sure if that will actually solve the problem. The problem with this approach is and will be that these lists are exclusionary measures. These lists illustrate what we must exclude to be [more] secure. Turning it around and making statements like validate all input makes little more sense, especially given that input validation must be defined in the context of the situation, and there is never a one-size-fits-all answer. To illustrate the point further - input validation may mean excluding certain character sets/patterns and pre-defining acceptable input options ... but this does not account for things like free-form input or other use-case specific examples.

In the end, the crux of the problem lies in the nature of security vulnerabilities. Security vulnerabilities are a moving target and although they can be loosely defined and lumpted into Top 7/10/25 lists it is not logical to consider these lists complete or even functional for designing software. Will a web application be secure if it follows the CWE Top 25 and addresses those issues? What about the OWASP Top 10? I don't think anyone has that answer, or at least is willing to stake their reputation on it.

So back to defining security as a business-level requirement... can it be done? Can one clearly articulate requirements to secure data/transactions/processes/whatever *before* the technologists get involved; meaning, before the means to execution are defined? I will leave that up for debate.

Harsh Reality - Life in InfoSec

RafalLos — Mon, 08 Dec 2008 20:13:00 GMT

It's Monday again, and it's absolutely brain-numbingly cold here in Chicago... but I wanted to get these thoughts down before they fell out of my brain to make room for new stuff.

Last week I had the pleasure of meeting with a group of guys that are running the Information Security practice within one of the largest and most respected retailers to the "hip" crowd... these folks live sales volume and press... good or bad. I think they've got some extremely unique challenges so I wanted to present the angle I proposed in case it's useful to anyone else.

First off, they have a very small "security" team, mainly consistent of compliance activities and common "operational security" tasks such as identity provisioning, anti-virus, firewall, you get the picture. They also have a relatively well-established QA team which is critical to the success of their online retail component - so the established value of that team is there. This is unlike the value of the security team - which unfortunately doesn't have a good foot-hold... not for lack of trying from what I heard. Their problem? No one cares about security. (Sound familiar yet?)

To overcome some of these challenges we focused on what was important to the business from an IT perspective - Software Quality. More specifically the quality of the online application(s) was important to this customer. Having their eCommerce site(s) up, and available for business is top-priority. Given that information we can quickly re-tool our approach and make *security* a component of the overall quality cycle. I know, some of you security purists are probably mad at me right now, but this is the harsh reality of life in a downturn. Why not though, use the business-critical areas to get the job done? The Security guys know they need tighter security but maybe the business doesn't care so much - except to check the box of compliance (PCI-DSS) - so I think taking a modified approach is the only way to fly in cases like this.

Making security a sub-component of overall software quality works like this. Security, amongst other things, aims to keep a site/application "up and running" and resistant to hacking. Now, hacking often-times causes Denial-of-Service conditions so there we have link #1 to quality and uptime. The second link comes in a little more vague. Hacking an application means loss of data, potentially - and that can lead to downtime and disrupt the consumer's ability to purchase or buy - basically data corruption. I know these aren't ideal links, and you'll like the PCI "compliance" link even less I'm sure - but there you have it.

Those 3 links into application quality may be the difference between *zero* security budget and getting *some* security budget. Now, the question of TTH (from Jeremiah Grossman, Time-To-Hack) may come into play again... we have to ask ourselves if what we're doing makes any difference in the time that it takes to take the app down, and steal the data. Maybe yes, maybe no right? The main point here for these guys is to demonstrate due-dilligence for PCI comliance. While this is a bit of a sad commentary on the way of the world and how much security *really* matters... at least they're doing something.

Keep pushing guys, you're on the right track!