Smart Web Crawler - Effective Recursive Resource Grabber for Web

This book is designed using Visual ASP .Net 2010 and MS-SQL Server 2008. The coding language used is VB.NET. The project involves settings for various options such as searching the web pages and extracts the mail ids, web URLs, phone numbers, songs download links and tags such as heading tags. The depth level is given so that not only from the given web pages but the pages linked with this web page are searched and the content is extracted. Multi-threading approach is used so that each page is searched by a separate thread and this assists in faster process. In the proposed system multiple web-pages can be searched in same or different web site with an intelligent search, thus it takes the resource from a webpage with the usage of depths. The intelligent search is performed using an advanced extraction called regular expression and so any kind of information can be extracted. For example, the pattern \d{6} retrieves the Indian Pincodes.