This week saw the release of the “Top 25 Most Dangerous Programming Errors” list from MITRE and SANS.  At first skim, I nearly discarded it as just an effort to pad resumes—after all, do we really need another “top X” list (every group with a barely pronounceable acronym has their own)? Weighing heavily into that opinion is that of the Top 25, there is some serious overlap that, I think, won’t really help developers.

For example, CWE-20 is “Improper Input Validation” and CWE-116 is “Improper Encoding or Escaping of Output.”  If developers fully understand these two and the associated risks of not doing them, they will almost certainly resolve several of the other 25, including:

-          CWE-89: Failure to Preserve SQL Query Structure (aka 'SQL Injection')

-          CWE-79: Failure to Preserve Web Page Structure (aka 'Cross-site Scripting')

-          CWE-78: Failure to Preserve OS Command Structure (aka 'OS Command Injection')

-          CWE-119: Failure to Constrain Operations within the Bounds of a Memory Buffer

Isn’t “Failure to Preserve SQL Query Structure” just another way of saying that you didn’t properly filter and encode tainted data? MITRE’s Steve Christey, in discussing the CWE Top 25, says the following about mixed reviews on including both input validation and encoding:

Part of it seems to come down to different ways of looking at the same problem. For example, is SQL injection strictly an input validation vulnerability, or output sanitization/validation/encoding or whatever you want to call it? In a database, the name "O'Reilly" may pass your input validation step, but you need to properly quote it before sending messages to the database. And the actual database platform itself has no domain model to "validate" whether the incoming query is consistent with business logic.

I’m not disagreeing with Steve that they should both be included, but rather that they should be included rather than the CWE numbers listed above.  After all, the programming error is not properly filtering and/or encoding tainted data, the resulting vulnerability is SQL injecting, Cross-Site Scripting, etc.

One other thing struck me as odd, which is the “Remediation Cost.”  This seems like sheer nonsense—it’s not something a document can define across the board without regard to the underlying application or technology, not to mention the (corporate) environment one has to work in. Is input validation remediation cost really “Low” when you might have to first educate your developers, make sure they understand the context and application logic (to make smart decisions about proper changes), and finally go through testing to ensure nothing is broken as a result? Likely not.

The truth is that any effort which gets more security knowledge into the heads of developers is a Good Thing and, hopefully, successful at causing applications to be more secure than they were. Also, any type of release that makes headlines outside of the security industry—where it may trigger a positive reaction in a programmer or developer—is definitely a win for security.  Certainly New York State took notice, now let’s hope their software suppliers do as well.

These are just a few issues that came to mind when reading the list. Gary McGraw wrote “Top 11 Reasons Why Top 10 (or Top 25) Lists Don’t Work,” and while I don’t agree with everything he says, there are a few that make reading the list worthwhile.  I’ll point out #8:

Automated tools can find bugs — let them. Teaching all developers the 700+ bad things in the Common Weakness Enumeration (or the even larger set of permutations allowed by languages like C++) is a futile exercise. We can start by using static analysis tools to remember all of the potential bugs and alert us to their presence.

I wholeheartedly agree with the second part of this one, and not just because I work for a company that sells automated security tools. With the complexity and sheer volume of code in existence these days, automated tools used throughout the dev lifecycle are a cost effective way to catch these issues.

I disagree with Gary that educating developers is a futile exercise—in fact it should be a requirement in every shop and school. Even if the developers are still not great at avoiding the problems in the first place, when the automated tool spits out a list of potential issues, an educated programmer will need to determine if/how/where to make a code change to fix the issue. Improper fixes have lead to dozens of incomplete bug remediations, and in some cases, additional vulnerabilities.

Anyway, to summarize (since I rambled on), I think the CWE Top 25 Programming Errors list is a little flawed, but given the attention it has received this week I can only applaud the efforts of MITRE and SANS for drawing more attention to security. After all, we’re all (ok, mostly) working toward the same goal: secure applications.