python crawler environment configuration

python download and installation

Python download:
Website: https://www.python.org/downloads/release/python-3105/
Insert image description here
Remember the installation location

Configure environment variables:
Right-click "Computer" and then click "Properties";
Then click "Advanced System Settings"
Select "Path" under the "System Variables" window and double-click it!
Then in the "Path" line, just add the python installation path (my D:\Python32), so later, just add the path. ps: Remember, the path is directly separated by a semicolon ""!
After the final setting is successful, enter the command "python" on the cmd command line, and the relevant display will be displayed.

pyCharm download and installation PyCharm download and configuration:

Website: https://www.jetbrains.com/pycharm/download/#section=windows
Insert image description here
Then it’s next.

After the installation is complete, add the python interpreter in pycharm
Click the file to select settings
and then select the python interpreter in the project
Insert image description here
The python version you choose to add is the location where you just downloaded python

The next step is to download the software packages needed for the crawler
Select the python interpreter to add the crawler Software package

Just search and install it

Basic crawler code

# 第一个爬虫示例,爬取百度页面

import requests  # 导入爬虫的库，不然调用不了爬虫的函数

response = requests.get("http://www.baidu.com")  # 生成一个response对象

response.encoding = response.apparent_encoding  # 设置编码格式

print("状态码:" + str(response.status_code))  # 打印状态码

print(response.text)  # 输出爬取的信息