scrapy 中遇到的问题

2018-08-28 16:09:32 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (302) to <GET http://bj.ganji.com/fang1/3558462755x.htm?adtype=3> from <GET http://jing.58.com/adJump?adType=3&target=pZwY0jCfsvFJshI6
UhGGshPfUiqhmyOMnit1PHN3PjmzP1NdXaOCIAY_uadBnvuhujmQnBYvPhndsHwBm1mVmhNkriY1PHKBPym1Pynzryc_nHN1PHEYnH0LPHbQPHDksakhUAqMuv-8gLR1ugFxpyEqnaubpgPkgvP6IANqPzkznaubpgPkgvQfmv7_5HDzsjDLPauWUA--UMwxIgP-0-qGujYhuyOYpgwOp
yEqn1NdrjEvnW0dPiud0vRzpyEqP101rHmknHDQFh-VuybqFhR8IA-YXgwO0ANqPau-UMwGIZ-xmv7YuHYhuyOYpgwOgvQfmv7_5iubpgPkgLwGUyNqnHN1PHEYn10vrjDQraukULPGIA-fUWYzFhP_pyPopyEqmHEduAnduAEVnyFWnaYYnWcLsH--nHbVrHN1mWcvnjEOrjnLFMK60h
7V5iukUA7YuhqzUHYVniu_pgPYXhEqUA-1IZGbgvIJFMK6uvRxIZ-kuHYQsjD1FMKzUvwdmLwGujYQnjTQn9&end=end>
2018-08-28 16:09:32 [scrapy.core.engine] DEBUG: Crawled (200) <GET http://bj.ganji.com/fang1/3558465548x.htm?adtype=3> (referer: http://bj.ganji.com/fang1/chaoyang/o1/)

1.scrapy爬虫时遇到的重定向问题:

DEBUG:Redirecting(302) to <GET>类问题

产生原因:页面被跳转到登录页面了,触动了赶集网的防爬装置。

解决方法:

2.DEBUG: Crawled (200) <GET >类问题

猜你喜欢

转载自blog.csdn.net/u013155359/article/details/82149932