What is the most basic information of the Python crawler? Of course it is the URL. All the information we need must be obtained through the URL. Do you know the URL?
Today, take the URL of the Baidu picture as an example to learn some information about the URL.
Baidu Picture of Yang Mi, get URL:
https://image.baidu.com/search/index?tn=baiduimage&ct=201326592&lm=-1&cl=2&ie=gb18030&word=%D1%EE%C3%DD&fr=ala&ala=1&alatpl=adress&pos=0&hs=2&xthttps=111111
At this time, what is obtained is a waterfall web page, if we change the index to flip:
https://image.baidu.com/search/flip?tn=baiduimage&ct=201326592&lm=-1&cl=2&ie=gb18030&word=%D1%EE%C3%DD&fr=ala&ala=1&alatpl=adress&pos=0&hs=2&xthttps=111111
The way the picture is changed to the page number,
we found that the URL of the picture is not only an index, but also stores some information. If you analyze this URL as a whole, you can see that the first half is the URL of the Baidu picture https://image.baidu.com/, and the back It is composed of a key-value pair, and the two key-value pairs are separated by &. Some only have keys and no values. Deleting does not affect normal indexes.
tn=baiduimage&ct=201326592&lm=-1&cl=2&ie=gb18030&word=%D1%EE%C3%DD&fr=ala&ala=1&alatpl=adress&pos=0&hs=2&xthttps=111111