JCrawler

JCrawler is an open-source (under the CPL) Stress-Testing Tool for web-applications. It comes with the crawling/exploratory feature. You can give JCrawler a set of starting URLs and it will begin crawling from that point onwards, going through any URLs it can find on its way and generating load on the web application. The load parameters (hits/sec) are configurable.

Features:
Crawling
Human pattern
Http Redirects and Cookies
OSS and tested
Console mode
Easy configuration
Platform independent