concept
The web crawler uses the uniform resource locator URL to find the target webpage, and directly returns the data content that the user is concerned about to the user. It does not require the user to obtain information in the form of browsing the webpage, which saves the user time and effort, and improves the data The accuracy of the collection allows users to do well in the massive data. The ultimate goal of web crawlers is to obtain the information they need from web pages. And finally put it in the warehouse for its own business processing. In addition, a web crawler is essentially a computer program or script, which automatically crawls and downloads the web pages of the World Wide Web according to certain logic and algorithm rules, which is an important part of search engines.
Application scenario
Examples of crawlers:
-
Reptile source:
-
data source:
-
Crawl, analyze:
-
Business warehouse design:
-
data demonstration:
