1. Why crawl?
First of all: It is said that now is the "big data era", where does the data come from?
User data generated by enterprises: Baidu Index , Ali Index , Tencent Browsing Index , Sina Weibo Index
Data platform purchase data: Datatang , Guoyun Data Market , Guiyang Big Data Exchange
Government/institution public data: data from the National Bureau of Statistics of the People's Republic of China, data from the World Bank, data from the United Nations , and Nasdaq .
Data management consulting companies: McKinsey, Accenture, iResearch
Crawling network data: If the data you need is not available in the market, or you are unwilling to buy it, you can choose to hire/be a crawler engineer and do it yourself.
Second, what to use to climb?
The era of big data has arrived, and it is more and more important to discover problems and make scientific and objective decisions based on data and analysis in business, economics, and other fields . In the research and application of data analysis technology, crawlers, as one of the sources of data acquisition , play a vital role.
Through learning, we must learn to use Python to crawl data from static web pages, dynamic web pages, web pages that need to be accessed after logging in, PC clients, and APPs , and combine theory with practice to engage in data crawling, analysis and research work in the future Lay the foundation.
In 1989, Guido van Rossum created the Python language.
Python advocates beauty, clarity, and simplicity, and is an excellent and widely used language.
1 
989 Nian, Guido van Rossum ( Guido · Fanluosumu) founded the Python language.