Web Crawler and Web Crawler Algorithms: A Perspective 
K Velkumar1, P Thendral2
1K Velkumar*, Assistant Professor, Department of Computer Science and Engineering, Theni Kammavar Sangam College of Technology, Theni, India.
2P Thendral, Associate Professor, Department of Computer Science and Engineering, Kalasalingam Academy of Research and Education, Krishnankovil.
Manuscript received on April 11, 2020. | Revised Manuscript received on May 15, 2020. | Manuscript published on June 30, 2020. | PP: 203-205 | Volume-9 Issue-5, June 2020. | Retrieval Number: E9362069520/2020©BEIESP | DOI: 10.35940/ijeat.E9362.069520
Open Access | Ethics and Policies | Cite | Mendeley
© The Authors. Blue Eyes Intelligence Engineering and Sciences Publication (BEIESP). This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)
Abstract: A web crawler is also called spider. For the intention of web indexing it automatically searches on the WWW. As the W3 is increasing day by day, globally the number of web pages grown massively. To make the search sociable for users, searching engine are mandatory. So to discover the particular data from the WWW search engines are operated. It would be almost challenging for mankind devoid of search engines to find anything from the web unless and until he identifies a particular URL address. A central depository of HTML documents in indexed form is sustained by every search Engine. Every time an operator gives the inquiry, searching is done at the database of indexed web pages. The size of a database of every search engine depends on the existing page on the internet. So to increase the proficiency of search engines, it is permitted to store only the most relevant and significant pages in the database.
Keywords: Web Crawler, Focused Crawler, Web crawler algorithms