What is an incremental web crawler?

It refers to a crawler that takes incremental updates to downloaded web pages and only crawls newly generated or changed web pages. It can ensure that the crawled pages are as new as possible to a certain extent.

Guess you like

Origin blog.csdn.net/weixin_55323026/article/details/115229716