Optimal crawling strategies for web search engines (original) (raw)
AI-generated Abstract
Optimal crawling strategies for web search engines are crucial for efficient web data retrieval. The study investigates various crawling approaches, examining their impact on search performance and resource utilization. By applying mathematical models and simulations, the research identifies optimal strategies that enhance the freshness and relevance of indexed webpages, ultimately improving overall search engine effectiveness.
Sign up for access to the world's latest research.
checkGet notified about relevant papers
checkSave papers to use in your research
checkJoin the discussion with peers
checkTrack your impact
Related papers
Proceedings of the thiry-fourth annual ACM symposium on Theory of computing - STOC '02, 2002
A review of web crawling approaches
2021
Websites are getting richer and richer with information in different formats. The data that such sites possess today goes through millions of terabytes of data, but not every information that is on the net is useful. To enable the most efficient internet browsing for the user, one methodology is to use web crawler. This study presents web crawler methodology, the first steps of development, how it works, the different types of web crawlers, the benefits of using and comparing their operating methods which are the advantages and disadvantages of each algorithm used by them.
A Study on Web Crawlers and Crawling Algorithms
2019
Making use of search engines is most popular Internet task apart from email. Currently, all major search engines employ web crawlers because effective web crawling is a key to the success of modern search engines. Web crawlers can give vast amounts of web information possible to explore the web entirely by humans. Therefore, crawling algorithms are crucial in selecting the pages that satisfy the users’ needs. Crawling cultural and/or linguistic specific resources from the borderless Web raises many challenging issues. This paper will review various web crawlers used for searching the web while also exploring the use of various algorithms to retrieve web pages. Keyword: Web Search Engine, Web Crawlers, Web Crawling Algorithms.
Loading Preview
Sorry, preview is currently unavailable. You can download the paper by clicking the button above.