Scrawlr Rants and Raves

erik.peterson — Wed, 25 Jun 2008 01:14:13 GMT

Let us know what you would like to see improved, questions on how best to use the tool or anything else.

Scrawlr & HacmeBank

Walker-SRS — Thu, 07 Aug 2008 11:28:27 GMT

I ran Scrawlr against Foundstones HacMe Bank web site, which is riddled with SQL Injection vulnerabilities, and it came up with nothing.

If the application cannot identify the base level vulnerabilities in a purposefully built vulnerable site, is it not leading others into a false sense of security?

Or am I missing something?

Is Scrawlr unlawful to use?

etaglio — Thu, 10 Jul 2008 20:28:02 GMT

Is it unlawful to enter a URL other than your own into the Scrawlr URL field?

Can the use of Scrawlr on a site the user doesn't own be construed as an attempted attack on that website? Or is the software, in and of itself, harmless?

1500 urls

voodoochicken — Tue, 12 Aug 2008 17:56:59 GMT

hi, does limited use to 1500 urls means this is shareware?

i mean, does it mean that after checking 1500 pages the tool will expire, or does it mean that it can only search 1500 pages in a single site

if the thing expires after 1500 uses, does it count 1 per each url or by each site?

Scrawlr FAQ

billyhoffman — Mon, 23 Jun 2008 23:10:31 GMT

Q: What is Scrawlr?

A: Scrawlr is a tool that will crawl a website and audit it for SQL Injection vulnerabilities. Specifically, Scrawlr is designed to detect SQL Injection vulnerabilities in dynamic web pages that will be indexed by search engines.

Q: Where can I download Scrawlr?

A: Download Scrawler Here

Q: Where can I see Scrawlr in action?

A: Here is a screen shot of Scrawlr in action

Q: What is SQL Injection?

A: SQL Injection is a common vulnerability in web applications that allows an attacker to execute their own SQL commands on your backend database system. Typical attacks include extracting confidential information, injecting malicious content, executing stored procedures or operating system commands, and disabling or destroying the database. You can learn more about SQL Injection by reading our whitepapers.

Q: What kind of websites can Scrawlr test?

A: Scrawlr can be used to test virtually any kind of website (provided you have permission to audit that website J). Scrawlr does have several limitations when compared to a traditional web vulnerability scanner which prevent it from crawling certain parts of your web application. These limitations include:

No submission of web forms
Does not interpret JavaScript or Flash
Only tests for SQL Injection vulnerabilities and only tests the query string parameters of URLs
Does not keep state or use Cookies
Crawl limit of 1500 pages
No authentication support

Q: Why does Scrawlr have these limitations?

A: Over the last several months, hackers have been using automated tools to perform mass exploitation of hundreds of thousands of websites. The attackers are using Google to find web applications built using Microsoft’s Active Server Pages and then performing SQL Injection attacks against these sites injecting various types of malware which are subsequently served to unsuspecting visitors. Scrawlr was specifically designed to help web developers test their website for SQL injection vulnerabilities that could be exposed to an attacker through a search engine. As such, Scrawlr crawls a websites using the same techniques as a search engine: it doesn’t keep state, or submit forms, or execute JavaScript or Flash. To fully test your web application for SQL Injection and other web vulnerabilities requires the use of a full featured web vulnerability scanner such as HP WebInspect.

Q: How do I know these vulnerabilities are real?

A: When Scrawlr detects what it thinks is a SQL Injection vulnerability, it will try to extract the database name and type, as well as the names of all the user defined tables in the database. This proves that data extraction is possible and that the SQL Injection vulnerability is real.

Q: Is Scrawlr limited to auditing only sites built with Microsoft’s ASP technology?

A: No, Scrawlr will also audit PHP, ASP.NET, and JSP pages that have query string parameters that are encountered during its search engine-like crawl of the web application.

Q: What specific technologies will Scrawlr crawl?

A: Scrawlr will crawl and audit any of the following file extensions:

htm/html
asp
aspx
php/php3/php4
jsp
js
txt
cfm
any file without an extension

Q: How much does Scrawlr Cost?

A: Scrawlr is a free tool created by HP’s Web Security Research Group. It is experimental software and not officially supported by HP.

Q: Is the source code available for Scrawlr?

A: No.

Q: Does Scrawlr support scanning a website through a proxy?

A: Yes, but Scrawlr only supports basic HTTP proxies that do not require authentication.

Q: Can Scrawlr scan web applications spread of many different hostnames and subdomains?

A: Yes, you can configure Scrawlr to audit multiple hosts using the “Allowed Hosts” option under the “Settings” menu. Please note that you must enter the hostname exactly as it appears. Scrawlr also display disallowed hostnames at the end of a scan, allowing you to start a new scan which will audit those discovered hostnames.

Q: How can I get help if I have problems with Scrawlr?

A: Scrawlr is released as experimental software and is not officially supported by HP. However, we have set up a forum so Scrawlr users can assist each other to work around issues.

Q: Will this tool be supported or extended?

A: This is an unsupported tool in the sense that individual problems and bug reports may not be addressed. However, the researchers who implemented the tool will actively read the forum postings, and there are plans to extend the tool in the future. We would love to hear your thoughts and constructive criticisms on the forum.

Q: Does Scrawlr find blind SQL injection vulnerabilities?

A: No. The tool only checks for verbose SQL injection.

Downloading Scrawlr outside of the US?

aunitt — Wed, 25 Jun 2008 10:47:19 GMT

We'd love to try Scrawlr out, however the form to download it only accepts Postcodes (ZIP codes) and telephone numbers in US format.

Can you open it up to the rest of the world?

Thanks

Ashley