Web Focused Crawlers

With the rapid growth of the Web, finding desirable information on the Internet is a tedious and time consuming task. Focused crawlers are the golden keys to solve this issue through mining of the Web content. In this regard, a variety of methods have been devised and implemented. In this book, we list and categorize these focused crawlers¿ methods into different classes by stating cons and pro of each one. Many of these methods, from information retrieval viewpoint, are not biased towards more informative terms in multi-term topics. In this research book also by considering information contents of terms, we propose our Term Frequency-Information Content (TF-IC) method which assigns appropriate weight to each term in a multi-term topic. We show TF-IC outperforms other methods such as Term Frequency-Inverse Document Frequency (TF-IDF) and Latent Semantic Indexing (LSI).