Blackbox vs Crystal Ball approach to webinspect

Do you wake up in the middle of the night thinking of ways you think web inspect could be better?

It occurs to me that 99% of what I am using webinspect for is testing apps that I own.  How much faster and better would webinspect be if it didn't have to guess about thinks is on the web server?  What if you had me put a small agent on the server that webinspect would query for information about the app?   Here are just a few ways I think that would help.

No more crawling... here is a directory listing.

Dont have to try linux and windows variants of attacks you know what it it ( ex:dont search for boot.ini for directory traversal)

Speaking of directory traversal.. Dont guess at how many ../ you need if you know the file structure

Grab config files not accessible in web path. (know table names, DB connection settings, etc(

Take a look at session tables while apps are running

"Hey agent, I just uploaded file xyz, where is it stored on the server" - Webinspect 

Do basic source code analysis to get a list of fields to attack 

Im just thinking I get better, faster scans the more you know about the app. 

maybe an early edition isn't a interactive agent that webinspect queries for data but is a small script which generates xml to feed Webinspect at the start of an assesment.  

You could digitally sign communicatons from webinspect to the agent so that you protect customers from themselves.  You can assume we will forget to remove the agents  ;)

 Just a thought.  Maybe now I can go back to sleep.
 

 

We actually talked about doing that in depth a couple of years ago. The only thing that keeps us from really developing an agent is the fact that customers don't seem to want it. Most enterprise customers that we talk to shudder at the thought of installing an agent on the webserver even if it makes the scans more accurate and obviously much faster. An agent would make something like AMP scheduled scans no longer necessary as the agent would update AMP as to any change that occurs and then only incremental scanning would be done. So the big question is do customers want this and will they pay for it? :)

I personally wouldn't leave the agent on the server after the assesment is done.  I too shudder at the thought of an agent on the server that gives up information for a pen test.   BUT i would definetly load an agent, scan the app and remove the agent if it meant I would get scans that had more accurate results and better protected my environment.  Most of our assements are done on non-production servers that aren't accessable from the internet so the agent doesn't create a significant risk. 

If we were to get really crazy with it, we could so something similar to DevInspect with the Static Analysis (depending on the language of course) and maybe even SecureObjects that report back to AMP for possible attacks.  A complete Security suite that tests against, protects and reports on attacks.  I can see how it would be an issue with security pen-testing but there are ways to limit the availability of the tool (shared encryption keys, internal only ports, encrypted output, etc...).  Several other security products use this method, BigFix is the one that comes to mind, but none, that I know of, in the Web App Security arena.  (OK, BigFix is a patch management solution but still can be used for some basic scanning of policies and vulnerabilities.)  It could create an environment to adding of patches and corporate policy, something comes up on the server and we automatically audit it with all possible parameters and report any vulnerabilities and if they are being attacked, all automatically or nearly so, maybe even help the web administrator setup the Application and Web Servers in the safest way possible while still getting everything to work.  It would definitely change the way we do what we do.

Installing agents on the server is a complete no-go for the vast majority of our customers ... think Configuration Control Boards, lights-out datacenters, very very remote systems (ie, other side of world in some cases) along with good ole' fashioned inter-departmental logistics and you're halfway towards realizing liife in a large org.  Even the relatively "simple" task of checking "Enable Directory Browsing" at the root then unticking it after an assessment could literally  takes weeks to approve and enact in a large, highly managed environment.

Oddly enough, however, many auditors and testers seem to be able to get a privileged login to the remote web server.  This takes some work, but nowhere near as much work as installing agents on a production system would.  One idea i always toyed with (and never got around to) is building a Custom Agent to do exactly this .... back in the day of managing tons of servers at Digex I got pretty good at ADSI, and since you can reference COM objects in Custom Agents you can do pretty much everything you listed.  You could even use WMI to get hardware stats as well, like polling CPU utilization during a scan.   Of course, this only works on Windows systems but I'm willing to bet that similar technologies exist in varying degrees of maturing for other operating systems.   Even if one couldn't get a privileged login to execute this remotely, it's feasible to hand the script off to a sysadmin and ask them to execute it for you and mail you the output ... the right Agent could then parse this out and recognize vulns in it and set an alert appropriately.

 I've yet to vet this idea amongst customers because frankly I really don't have time to build out this agent.  Many folks I've bounced the idea off didn't really seem to latch on but I'm guessing offhand that I could reduce scan time by about 50% for "high recursion audits" (ie, RCA set to 3 or higher) ....  imagine not doing *any* directory enums, file enums, extension checks, etc ... all "Discovery" would be done via the script without any "guessing" involved. 

So there you have it.    I highly recommend "Windows Shell Scripting" and "ADSI Scripting" by Tim Hill for further reading ... these should be mandatory reading for anyone who uses a Windows machine.   I used to run a commercial site called windowsshellscripting.com that had a really large collection of shell, ADSI, WMI and other scripts on it along with support forums, but completely ran out of time and desire to mod it years ago and then recently whilst on vacatiton the hosting expired and I just let it go ...

I have to deal with unmanned data centers, change control, and  Inter-department politics er..um.uh. logistics are always a real challenge.  But change control actually works to my advantage here.  Before changes are introduced to our production environment, change control requires that we do a security assessment.   Much of the time I am testing Web Application we have purchased from third parties.   They dont go through our internal DEV/QA environment and our first look at the product is its preproduction scan.   Taking a day or two to get an agent on those boxes to get a crystal ball assesment is definetly worth my time. Since its not production installing an agent for the test is do-able.  Yes it requies some paperwork, but I get better results.  The agent should be a non-intrusive install (simple install & remove).  You could use WMIC & Netbois to check windows boxes or perhaps its a PERL or JavaScript which you run once, collect info and build an XML which is loaded into webinspect. 

If an agent and some change control paperwork is what I need to assess all of the files on the webserver and catch security vulnerabilities that would otherwise go unnoticed, Im in.