Have problems with proxy IP reptile how to do

Reptiles crawl when data are aware of the need to use a proxy IP, otherwise can not be carried out smoothly. Users use a proxy IP reptiles, there will be some problems, leading to reptiles can not continue. So, with the proxy IP reptiles have problems how to do it?
1. distributed crawlers. Use a distributed crawler way to avoid the problem to a certain extent, but also greatly enhance the effect of crawl data, improve work efficiency.
2. Save cookies. Would be more trouble when simulated landing after landing directly on the web can remove the cookie saved, together with reptiles, but this method is not a permanent solution, it may from time to time cookie will fail.
3. Solve the problem of verification code. Reptile let a long experience problems enter the verification code, so that the other sites have been identified to you is a crawler. After down code can be locally verified, enter codes manually.
4. Multi-account. Many sites will be judged by the frequency of access account, so you can grab a threshold test a single account, switch accounts change agent IP threshold.
These are the proxy IP add links describe simple solution crawler will encounter some problems, we can choose according to their own practical problems encountered. IP proxy flash cloud stable online, always switch IP, as well as professional and technical personnel online guide.

Guess you like

Origin blog.51cto.com/14338698/2406042