2020/11/09 The solution to using urlopen() in python3 to report errors

When using the urllib.request module in python3 to grab a webpage, the following code will report a urllib.error.URLError error

import urllib.request
response = urllib.request.urlopen('https://www.python.org')

urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1045)>

This error is due to a new feature introduced after Python 2.7.9. When you use urllib.urlopen an https, the SSL certificate will be verified once. When the target uses a self-signed certificate, urllib.error.URLError will be reported. The solution is as follows:

import urllib.request
import ssl
ssl._create_default_https_context = ssl._create_unverified_context
response = urllib.request.urlopen('https://www.python.org')
print(response.read().decode('utf-8'))

By importing the ssl module, change the certificate verification to not require verification.

Reprinted from the solution of using urlopen() in python3 to report errors