Web Crawler : Spider Web Application Crawler - Novel Release

Web Crawler is a opened upwards origin application that is based on the WebEngine library. WebEngine is the laid upwards of tools for performing black-box web-sites testing in addition to other like tasks. It provides tools agency for reception of documents from a web-server, parse HTML pages, their comparisons, search through pages sources in addition to DOM-structure, sandboxed execution of the java-scripts in addition to vb-scripts. Tools for authorized access to web-servers are provided, alongside dissimilar potency mechanisms. Some formal checks tin move hold upwards performed yesteryear embedded checking engine without creation of the novel automobile code.


So basically, Web Crawler is a utility designed for testing in addition to demonstration of the WebEngine opened upwards origin library features. It gathers data nearly the resources of a specified spider web server yesteryear analyzing references inward the HTML markup, text, in addition to JavaScript code. Additionally, a query is sent to the Web Of Trust cognition base of operations to obtain data nearly the analyzed site. This banking concern fit demonstrates analysis of spider web application vulnerabilities. Though Web Crawler is non a full-fledged spider web application safety analyzer, it tin move hold upwards used to perform a flake of reconnaissance nearly the application inward question.
This is a shroud shot of the application:

The primary features provided yesteryear the application are listed below:

• JavaScript analysis aimed at receiving references alongside simulation of a DOM structure
• Support of the Basic, Digest, in addition to NTLM potency schemes
• Access to the contents of spider web servers via HTTP
• Operation via proxy servers alongside diverse potency schemes
• A broad diversity of options to depict the scan target (lists of scanned domains, restriction of scanning to a host, a domain, or a spider web server directory, etc.)
• Modular structure, which allows i to implement plug-ins

A real proficient characteristic offered yesteryear this tool is that y'all tin move implement your ain plug-ins to ameliorate the functionality of this application. The library is currently non meant for scanning of rogue in addition to misbehaving HTTP servers. In these cases, right in addition to stable functioning cannot hold upwards guaranteed.
Additionally, since Web Crawler is based on the WebEngine library, y'all tin move also utilisation it via the ascendance line. Influenza A virus subtype H5N1 sample ascendance trouble usage could be:

1	<span class="IL_AD" id="IL_AD2">crawler</span> --target  www.example.com --profile 0 –name "First sample task" --result report.txt --output 2

Download Crawler v0.2 (Crawler_v0_2.zip) here.

Share This :