Python3 requests using the request, the decoding error: 'utf8' codec can not decode byte 0x83 in position 1: invalid start byte

In response to the request requests the content can be obtained by several properties:

 

response.text  

After the content, the decoding will select the character set Content-Type HTTP Header in accordance with the response. E.g

1	"'Content-Type': 'text/html;charset=UTF-8'"

They will use "UTF-8" decoding. Available character sets currently used by visiting response.encoding.

You can also modify the character set used

1	response.encoding =  'UTF-8'

Such calls response.text time again, it will return the contents of UTF-8 decoding.

 

response.content

Binary content, and has automatically gzip and deflate coding used in transmission is decoded.

 

response.raw

In response to the original content can be used to do some analysis. Only need to add the parameter stream = True at initialization, or acquired value b ''. But note that, after adding parameters (stream = True), text and content can not be used. . . Will be given:

1	requests.exceptions.ChunkedEncodingError: ( 'Connection broken: IncompleteRead(102 bytes read)' , IncompleteRead(102 bytes  read ))

 

response.json()

This is generally the case for the known return data as JSON format string. If the returned JSON data is unavailable will throw an exception:

1	ValueError: No JSON object could be decoded

 

Back up the problems encountered:

1	'utf8'  codec can't decode byte 0x83  in  position 1: invalid start byte

The problem occurs when you call response.content.decode ().

 

Solution:

　　1. Remove the request HTTP Header gzip:

1	"accept-encoding": "gzip, deflate, br",

　　2. The original content gzip decompression process