1. Background
Similar web crawler packet from the third party interface when the third-party platform interface will record IP analysis, judgment based on IP remote login or request the same IP request too much can lead to restriction of the packet anomalies. IP proxy IP restricted manner when found when another IP packet to continue acquiring request interface.
2, implementation
2.1, registered agent sesame (sesame agent platform provides a large number of available IP);
2.2, sesame agent after adding to whitelist can obtain an IP platform based on the interface provided;
2.3, extract the IP when the effective length of the different fees, the fee is valid long your (selected according to business needs), IP can choose where geographical province;
2.4, when the third party interface climb request data to the specified IP and port agents (The interface returns the result whether to switch the predetermined proxy IP);
3, the development version
3.1, V1.0 IP-based database storage agent
3.1.1, sesame initialization from the agent 10 acquires IP stored in the database, the timing 10s pool maintenance database IP 10 IP proxy;
3.1.2, database interface when the third party requests from obtaining a random IP agent, when the agent is limited to the IP switches and a restricted IP IP IP pool is removed from the database;
3.1.3 Dynamic IP pool database maintenance, when the proxy IP to the expiration time will be removed from the database IP pool;
3.2, V2.0 Redis-based deposit proxy IP
3.2.1 The effective length when the IP settings stored in the long redis valid IP 10, when the effective length of the IP to the automatic cleaning, redis Add a new dynamic IP redis stored IP pool maintenance;
3.2.2, when the request from the third party interface randomly obtain a pool redis stored IP proxy IP, IP is restricted when the agent switches to a restricted IP and IP to IP pool stores redis removed;
4, items in the scene
Under the background single request third-party interface project xx-secure-api calls when there is no third party interface IP restrictions, in order to hide IP switching once every few hours proxy IP