Python: Beautiful syntax, concise code, high development efficiency, and many supported modules. The related HTTP request module and HTML parsing module are very rich. There are also Scrapy and Scrapy-redis frameworks that make it very easy for us to develop crawler variables.
PHP: It does not support multi-threading and asynchrony very well, and its concurrent processing capability is weak. Crawler is a tool program, which requires relatively high speed and efficiency.
Java: The language itself is cumbersome and the amount of code is huge. Refactoring costs are relatively high, and any modification will result in a large number of changes to the code. Crawlers often need to modify the collection code.
C/C++: The cost of learning and development is high. It may take up to half a day to write a small crawler program.