requests basic usage

First, confirm: requests already installed

Some simple examples from the start.

send request

Requests using simple network request.

Requests to start importing module:

>>> import requests

Then, try to get a page. In this case, we Github to get the public timeline:

>>> r = requests.get('https://api.github.com/events')

Now, we have named  r the  Response object. We can get all the information we want from this object.

Requests simple API means that all types of HTTP requests are obvious. For example, you could send a HTTP POST request so that:

>>> r = requests.post('http://httpbin.org/post', data = {'key':'value'})

Pretty, is not it? Then other types of HTTP requests: PUT, DELETE, HEAD and OPTIONS how is it? It is the same simple:

>>> r = requests.put('http://httpbin.org/put', data = {'key':'value'})
>>> r = requests.delete('http://httpbin.org/delete')
>>> r = requests.head('http://httpbin.org/get')
>>> r = requests.options('http://httpbin.org/get')

It is very good, but this is only the tip of the iceberg Requests it.

 

URL parameter passed

You may often want to pass some data to the query string of the URL (query string). If you are manually constructed URL, then the data will be placed in the URL in the form of key / value, followed a question mark. For httpbin.org/get?key=valexample, . Requests allows you to use  params keyword arguments to a string dictionary to provide these parameters. For example, if you want to pass  key1=value1 and  key2=value2 to  httpbin.org/get , then you can use the following code:

>>> payload = {'key1': 'value1', 'key2': 'value2'}
>>> r = requests.get("http://httpbin.org/get", params=payload)

By printout of the URL, you can see the URL has been correctly coded:

>>> print(r.url)
http://httpbin.org/get?key2=value2&key1=value1

Note that the dictionary is  None the key will not be added to the query string of the URL's.

You can also list as the value passed:

>>> payload = {'key1': 'value1', 'key2': ['value2', 'value3']}

>>> r = requests.get('http://httpbin.org/get', params=payload)
>>> print(r.url)
http://httpbin.org/get?key1=value1&key2=value2&key2=value3

Response content

We can read the contents of the server response. Again GitHub time line, for example:

>>> import requests
>>> r = requests.get('https://api.github.com/events')
>>> r.text
u'[{"repository":{"open_issues":0,"url":"https://github.com/...

Requests will automatically decode content from the server. Most unicode character set can be seamlessly decoded.

After the request is made, Requests will make an educated guess based on the encoded HTTP response header. When you visit  r.text when it, Requests will use its presumed text encoding. You can find out what encoding Requests to use, and can use the r.encoding property to change it:

>>> r.encoding
'utf-8'
>>> r.encoding = 'ISO-8859-1'

If you change the code, whenever you visit  r.text , Request all will use  r.encoding the new value. You may wish to modify down in case of using a special encoding logic calculation of the encoded text. Such as HTTP and XML itself can specify the encoding. In this case, you should use  r.content to find the code, and then set  r.encoding the corresponding code. This allows the correct code resolution  r.text up.

In case you need to, Requests can also use custom coding. If you created your own code, and use the  codecsmodule to register, you can easily use the decoder name as  r.encoding a value, and then to deal with the coding for you by the Requests.

Binary response content

You can access byte of the request response thereof, a request for non-text:

>>> r.content
b'[{"repository":{"open_issues":0,"url":"https://github.com/...

Requests for you automatically decode  gzip and  deflate transmitting the encoded response data.

For example, to create a picture request binary data is returned, you can use the following code:

>>> from PIL import Image
>>> from io import BytesIO

>>> i = Image.open(BytesIO(r.content))

JSON response content

Requests also has a built-in decoder JSON, JSON data processing help you:

>>> import requests

>>> r = requests.get('https://api.github.com/events')
>>> r.json()
[{u'repository': {u'open_issues': 0, u'url': 'https://github.com/...

If the JSON decoding fails,  r.json() it will throw an exception. For example, the content of the response is a 401 (Unauthorized), attempts to access  r.json() it will throw  ValueError: No JSON object could be decoded an exception.

It should be noted that a successful call  r.json() and ** ** does not mean successful response. Some servers will contain a JSON object (such as HTTP 500 error details of) in response to a failed. This will be decoded JSON returned. To check whether the request was successful, use  r.raise_for_status() or check  r.status_code whether your expectations and the same.

Original response content

In rare cases, you may want to get raw sockets response from the server, then you can access  r.raw. If you really want to do this, make sure that you set up in the initial request  stream=True. Specifically you can do:

>>> r = requests.get('https://api.github.com/events', stream=True)
>>> r.raw
<requests.packages.urllib3.response.HTTPResponse object at 0x101194810>
>>> r.raw.read(10)
'\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x03'

However, under normal circumstances, you should save in the following mode text stream to a file:

with open(filename, 'wb') as fd:
    for chunk in r.iter_content(chunk_size):
        fd.write(chunk)

Use  Response.iter_content will handle a lot of you directly  Response.raw have to deal with. When the download stream, the top priority is the recommended way to get content. That note  chunk_size CAN BE Freely Adjusted to A Number The May that your use of Better Fit Cases.

Custom request headers

If you want to add HTTP headers for the request, simply pass a  dict given  headers parameter on it.

For example, we have a content-type is not specified in the previous example:

>>> url = 'https://api.github.com/some/endpoint'
>>> headers = {'user-agent': 'my-app/0.0.1'}

>>> r = requests.get(url, headers=headers)

Note: Custom header lower priority than some specific sources of information, such as:

• If you  .netrc set up user authentication information, use headers = set authorization will not take effect. If the set  auth= parameters, `` .netrc`` setting is invalid.
• If you are redirected to another host, the authorization header will be deleted.
• Proxy Authorization header will be overwritten URL provided in the proxy identity.
• In the case that we can determine the content length, header of Content-Length will be overwritten.

Still further, Requests will not change their behavior based on the specific circumstances of the custom header. Only in the last request, all of the header information will be passed into it.

Note: All values must be a header  string, bytestring or unicode. Although the transfer unicode header is allowed, but not recommended.

More complex POST requests

Usually, you want to send some encoded form in the form of data - much like an HTML form. To achieve this, simply pass a dictionary to data parameters. Your data dictionary at the time the request is automatically encoded form in the form of:

>>> payload = {'key1': 'value1', 'key2': 'value2'}

>>> r = requests.post("http://httpbin.org/post", data=payload)
>>> print(r.text)
{
  ...
  "form": {
    "key2": "value2",
    "key1": "value1"
  },
  ...
}

You can also  data arguments passed a list of tuples. When a plurality of elements in the form used for the same key, is particularly effective in this way:

>>> payload = (('key1', 'value1'), ('key1', 'value2'))
>>> r = requests.post('http://httpbin.org/post', data=payload)
>>> print(r.text)
{
  ...
  "form": {
    "key1": [
      "value1",
      "value2"
    ]
  },
  ...
}

Many times data is not coding for the form you want to send the form. If you pass a  string rather than a  dict, then the data will be released directly out.

For example, Github API v3 receiving the encoded as JSON POST / PATCH data:

>>> import json

>>> url = 'https://api.github.com/some/endpoint'
>>> payload = {'some': 'data'}

>>> r = requests.post(url, data=json.dumps(payload))

Here in addition to its own  dict encoding, you can use the  json parameters passed directly, then it will be encoded automatically. This is new in version 2.4.2:

>>> url = 'https://api.github.com/some/endpoint'
>>> payload = {'some': 'data'}

>>> r = requests.post(url, json=payload)

A multi-part encoded POST (Multipart-Encoded) file

Requests makes uploading multipart encoded file becomes very simple:

>>> url = 'http://httpbin.org/post'
>>> files = {'file': open('report.xls', 'rb')}

>>> r = requests.post(url, files=files)
>>> r.text
{
  ...
  "files": {
    "file": "<censored...binary...data>"
  },
  ...
}

You can explicitly set the file name, file type, and request headers:

>>> url = 'http://httpbin.org/post'
>>> files = {'file': ('report.xls', open('report.xls', 'rb'), 'application/vnd.ms-excel', {'Expires': '0'})}

>>> r = requests.post(url, files=files)
>>> r.text
{
  ...
  "files": {
    "file": "<censored...binary...data>"
  },
  ...
}

If you want, you can also send a file to a string as received:

>>> url = 'http://httpbin.org/post'
>>> files = {'file': ('report.csv', 'some,data,to,send\nanother,row,to,send\n')}

>>> r = requests.post(url, files=files)
>>> r.text
{
  ...
  "files": {
    "file": "some,data,to,send\\nanother,row,to,send\\n"
  },
  ...
}

If you are sending a very large file as  multipart/form-data requested, you may want to request to make the data stream. By default  requests it is not supported, but there is a third-party packages  requests-toolbelt are supported. You can read  toolbelt documentation to learn how to use.

Send multiple files in one request reference  advanced use of  one.

caveat

We strongly recommend that you use binary mode ( binary the MODE to open the file). This is because Requests may try to provide you with  Content-Length header, when it does so, the value will be set to the number of bytes of the file ( bytes ). If you open a file in text mode (text mode), an error may occur.

The response status code

We can detect the response status code:

>>> r = requests.get('http://httpbin.org/get')
>>> r.status_code
200

For the convenience of reference, Requests also comes with a built-in query object status code:

>>> r.status_code == requests.codes.ok
True

If you send a bad request (a client error 4XX, 5XX or server error response), we can by Response.raise_for_status() throwing an exception:

>>> bad_r = requests.get('http://httpbin.org/status/404')
>>> bad_r.status_code
404

>>> bad_r.raise_for_status()
Traceback (most recent call last):
  File "requests/models.py", line 832, in raise_for_status
    raise http_error
requests.exceptions.HTTPError: 404 Client Error

However, because our case  r is  status_code that  200 , when we call  raise_for_status() upon, was:

>>> r.raise_for_status()
None

Everything came harmonious Kazakhstan.

Response header

We can view the server to display a Python dictionary form of response headers:

>>> r.headers
{
    'content-encoding': 'gzip',
    'transfer-encoding': 'chunked',
    'connection': 'close',
    'server': 'nginx/1.0.4',
    'x-runtime': '148ms',
    'etag': '"e1ca502697e5c9317743dc078f67693f"',
    'content-type': 'application/json'
}

But the dictionary is special: it is only the HTTP headers born. According to  RFC 2616 , HTTP header is not case sensitive.

Therefore, we can use any uppercase to access these response header fields:

>>> r.headers['Content-Type']
'application/json'

>>> r.headers.get('content-type')
'application/json'

It also has a special point that many times the server can accept the same header, each time with a different value. But Requests will merge them, so that they can be represented by a map, see  RFC 7230 :

A recipient MAY combine multiple header fields with the same field name into one "field-name: field-value" pair, without changing the semantics of the message, by appending each subsequent field value to the combined field value in order, separated by a comma.

The recipient may combine multiple header fields of the same name, put them together as a "field-name: field-value" pairs, each of the subsequent fields are sequentially added to the value of the field values ​​in the combined, separated by commas can be opened, this does not change the semantics of information.

Cookie

If a response contains some of the cookie, you can quickly access them:

>>> url = 'http://example.com/some/cookie/setting/url'
>>> r = requests.get(url)

>>> r.cookies['example_cookie_name']
'example_cookie_value'

To send cookies to your server, you can use  cookies parameters:

>>> url = 'http://httpbin.org/cookies'
>>> cookies = dict(cookies_are='working')

>>> r = requests.get(url, cookies=cookies)
>>> r.text
'{"cookies": {"cookies_are": "working"}}'

Cookie to return the object  RequestsCookieJar, and its behavior is similar to the dictionary, but the interface is more complete, cross paths for cross-domain use. You can also spread to the Cookie Jar Requests in:

>>> jar = requests.cookies.RequestsCookieJar()
>>> jar.set('tasty_cookie', 'yum', domain='httpbin.org', path='/cookies')
>>> jar.set('gross_cookie', 'blech', domain='httpbin.org', path='/elsewhere')
>>> url = 'http://httpbin.org/cookies'
>>> r = requests.get(url, cookies=jar)
>>> r.text
'{"cookies": {"tasty_cookie": "yum"}}'

Redirect the request history

By default, in addition to HEAD, Requests automatically handles all the redirects.

You may be used in response to object  history methods to track redirection.

Response.history Is a  Response list of objects, in order to complete the request to create these objects. The list of objects sorted according to the most recent request from the most seasoned.

For example, Github redirect all HTTP requests to HTTPS:

>>> r = requests.get('http://github.com')

>>> r.url
'https://github.com/'

>>> r.status_code
200

>>> r.history
[<Response [301]>]

If you are using GET, OPTIONS, POST, PUT, PATCH , or DELETE, then you can  allow_redirects disable the redirection process parameters:

>>> r = requests.get('http://github.com', allow_redirects=False)
>>> r.status_code
301
>>> r.history
[]

If you use HEAD, you can enable the redirection:

>>> r = requests.head('http://github.com', allow_redirects=True)
>>> r.url
'https://github.com/'
>>> r.history
[<Response [301]>]

time out

You can tell requests in order to pass  timeout after a number of seconds parameter set to stop waiting for a response. Basically all of the production code should use this parameter. If not, your program may be lost forever response:

>>> requests.get('http://github.com', timeout=0.001)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
requests.exceptions.Timeout: HTTPConnectionPool(host='github.com', port=80): Request timed out. (timeout=0.001)

note

timeout Connection process is only effective, regardless of the Download Response body. timeout Download Response time is not limited whole, but if the server  timeout does not answer within seconds, will raise an exception (more precisely, at timeout any byte of data is not received from the base into the socket seconds) If no timeout is specified explicitly, requests do not time out.

Errors and exceptions

Experiencing network problems (such as: DNS lookup failed, refused connections, etc.) when, Requests will throw an  ConnectionError exception.

If the HTTP request returned an unsuccessful status code  Response.raise_for_status() will throw an  HTTPErrorexception.

If the request times out, throws an  Timeout exception.

If the request exceeds the maximum number of redirects set will throw an  TooManyRedirects exception.

All Requests explicitly throw exceptions inherit from  requests.exceptions.RequestException .