Python_01_IP agent pool _ achieve database module agent pool

　　Effect: for a set of proxies related database operations

　　Objective: to realize additions and deletions to the database search operation change

　　step:

　　　　1. In the init, the database connection is established, to obtain a set of operations, close the database connection process in del

　　　　2. Provide basic CRUD functions

　　　　　　1) to achieve insertion

　　　　　　2) modify the function to achieve

　　　　　　3) remove the agent functions to achieve

　　　　　　4) Search function ip proxy

　　　　3. Provide api proxy function modules used

　　　　　　1) implement queries, query depending on the conditions, you can specify the number of queries, the first score in descending order, in ascending speed, to ensure the quality of agency ip above.

　　　　　　2) achieve access to the proxy ip list according to the type of protocol and domain name to access the site

　　　　　　3) implement random Gets a proxy ip protocol type and domain name to access the full

　　　　　　4) to achieve the specified domain name to add to the list of designated ip in diable_domain

　　Code:

from pymongo import MongoClient

from settings import MONGO_URL

from utils.log import logger

from Domain Import the Proxy
 class MongoPool (Object):
     DEF  the __init__ (Self):
         # establish a connection to the database 
        self.client = MongoClient (MONGO_URL)
         # Get set to operate 
        self.proxies self.client = [ ' proxies_pool ' ] [ ' Proxies ' ]


    DEF  the __del__ (Self):
         # close the connection 
        self.client.close ()


    DEF insert_one (Self, proxy):
         # using the proxy ip as the primary key data in MongoDB: the _id 
        COUNT = self.proxies.count_documents ({ ' the _id ' : proxy.ip})
         IF COUNT == 0:
            dic=proxy.__dict__
            dic['_id']=proxy.id
            self.proxies.insert_one()
            logger.info ( " Insert New Agent: {} " .format (Proxy))
         the else :
            logger.warning ( " existing proxy} { " .format (Proxy))


    DEF update_one (Self, Proxy):
         # achieve editing 
        self.proxies.update_one ({ ' the _id ' : proxy.ip}, { ' $ SET ' :. Proxy the __dict__ })


    DEF delete_one (Self, Proxy):
         # delete functionality into 
        self.proxies.delete_one ({ ' the _id ' : proxy.ip})
        logger.info ( " Delete Proxy IP: {} " .format (Proxy))


    DEF find_all (Self):
         # queries all proxy ip 
        the Cursor = self.proxies.find ()
         for Item in the Cursor:
             # delete _id this Key 
            item.pop ( ' _id ' )
            proxy=Proxy(**item)
            yield proxy