These libraries will not, I will not say Python Reptile

These libraries will not, I will not say Python Reptile

AirPython  5 days ago

The following article comes from JAVAandPython king, the king of JAP

JAVAandPython 君

Focus on the technology platform of JAVA and Python. Share JAVA and Python-related original technical articles, tools, resources, course selection, quality resources, hot information, learning materials and so on.

 

Many of my friends do not know how Python reptile started, how to learn, in the end what to learn.

 

Today I come to you talk about learning reptiles, we must master some third-party libraries.

 

Ado, directly on dry goods.

 

 

 

1

 Library request

 

 

1. requests

 

GitHub：https://github.com/psf/requests

 

requests the library should be done now reptiles hottest and most practical library, and very user-friendly.

 

About requests to use the most detailed, you can refer to the official document: https: //requests.readthedocs.io/en/master/

 

Use small case:

•  
•  
•  
•  
•  
•  
•  
•  
•  
•  
•  
•  

>>> import requests>>> r = requests.get('https://api.github.com/user', auth=('user', 'pass'))>>> r.status_code200>>> r.headers['content-type']'application/json; charset=utf8'>>> r.encoding'utf-8'>>> r.textu'{"type":"User"...'>>> r.json(){u'disk_usage': 368627, u'private_gists': 484, ...}

 

 

2. urllib3

 

GitHub：https://github.com/urllib3/urllib3

 

urllib3 is a very powerful library http request, provide the functionality of a series of operational URL.

 

Details about its use can refer to: https: //urllib3.readthedocs.io/en/latest/

 

Use small case:

•  
•  
•  
•  
•  
•  
•  

>>> import urllib3>>> http = urllib3.PoolManager()>>> r = http.request('GET', 'http://httpbin.org/robots.txt')>>> r.status200>>> r.data'User-agent: *\nDisallow: /deny\n'

 

 

3.selenium

 

GitHub：https://github.com/SeleniumHQ/selenium

 

Automated testing tools. A call to the driver's browser, through this library you can call directly to the browser to complete certain operations, such as input verification code.

 

For this is not just a Python library to use, like JAVA, Python, C # and so be able to use this library selenium

 

Python language on how to use the library, you can check out the official documentation to access https://seleniumhq.github.io/selenium/docs/api/py/

 

 

Use small case:

•  
•  
•  
•  

from selenium import webdriver
browser = webdriver.Firefox()browser.get('http://seleniumhq.org/')

 

 

4.aiohttp

 

GitHub：https://github.com/aio-libs/aiohttp

 

HTTP-based implementation framework asyncio. Asynchronous operation by means of the async / await keyword library using asynchronous crawling, can greatly improve efficiency.

 

This belongs to the Advanced reptiles time must master asynchronous library. Details about the operation aiohttp, you can go to the official document: https: //aiohttp.readthedocs.io/en/stable/

 

Use small case:

•  
•  
•  
•  
•  
•  
•  
•  
•  
•  
•  
•  
•  
•  
•  

import aiohttpimport asyncio
async def fetch(session, url):    async with session.get(url) as response:        return await response.text()
async def main():    async with aiohttp.ClientSession() as session:        html = await fetch(session, 'http://python.org')        print(html)
if __name__ == '__main__':    loop = asyncio.get_event_loop()    loop.run_until_complete(main())

 

 

 

2

 Parsing library 

 

 

 

1、beautifulsoup

 

The official document: https: //www.crummy.com/software/BeautifulSoup/

 

html and XML parsing, extracting information from web pages, also has a strong and diverse API analytical methods. I often use a parsing library for parsing html is very easy to use. For the person who wrote this is the reptile must master library.

 

 

2、lxml

 

GitHub：https://github.com/lxml/lxml

 

Support for HTML and XML parsing, support XPath analytical methods, and analytical efficiency is very high.

 

 

3、pyquery

 

GitHub：https://github.com/gawel/pyquery

 

jQuery Python implementation can operate with jQuery syntax parsing HTML documents, ease of use and speed are good resolve.

 

 

 

 

3

 Data Repository 

 

 

1、pymysql

 

GitHub：https://github.com/PyMySQL/PyMySQL

 

The official document: https: //pymysql.readthedocs.io/en/latest/

 

A pure Python MySQL client library is implemented. Very practical, very simple.

 

 

2、pymongo

 

GitHub：https://github.com/mongodb/mongo-python-driver

 

The official document: https: //api.mongodb.com/python/

 

As the name suggests, a library for direct connection mongodb database query operations.

 

 

3, redisdump

 

Usage: https: //blog.csdn.net/zhwitbird/article/details/81279406

 

redis-dump is json redis and system conversion tools; redis-dump is based on the development of ruby, ruby ​​environmental needs, and the new version of redis-dump requirements above 2.2.2 ruby ​​version, centos yum can only be installed in version 2.0 ruby.

 

Need to install ruby ​​management tool rvm install a higher version of ruby.