Getting Started with Requests

First run cmd directly through the administrator, and then execute pip install requests to install the Requests library directly

There is a basic sentence

　　r = requests.get(url)

　　Through requests.get, construct a Request object that requests resources from the server (generated internally by python)

　　The content returned by get() is represented by r, where r is the Response object, which contains all the server resources, and we can get all the information we want from this object.

The full usage is r = requests.get( url, params = None, **kwargs)

In the object (i.e. r) property of Response

——r.status_code The return status of the HTTP request, 200 means the connection is successful, 404 or other means failure

——r.text The string form of the HTTP response content, that is, the page content corresponding to the url

--r.encoding Guess the encoding of the response content from the HTTP header

——r.apparent_encoding analyzes the variation of the response content from the content (commonly used alternative encoding methods, more accurate)

--r.content The binary form of the HTTP response content

 

Requests library exception

——requests.ConnectionError network connection error exception

--requests.HTTPError HTTP error exception

--requests.URLRequired URL missing exception

——requests.TooManyRedirects exceeds the maximum number of redirects, resulting in a redirection exception

——requests.ConnectTimeout The connection to the remote server timed out abnormally

——requests.Timeout The request URL times out and an exception is generated

 

Common code framework for crawling web pages

1  def getHTMLText(url):
 2      try :
 3          r = requests.get(url, timeout = 30 )
 4          r.raise_for_status() #If the     status is not 200, raise HTTPError 
5          r.encoding = r.apparent_encoding
 6          return r. text
 7      except :
 8          return  " An exception occurred "