Since Python 3 is to be used to develop crawlers, the first step must be to install Python 3. Here will introduce the installation process under the three platforms of Windows, Linux and Mac.
1. Related Links
-
Official website: http://python.org
-
Download link: https://www.python.org/downloads
-
Third-party library: https://pypi.python.org/pypi
-
Official document: https://docs.python.org/3
-
Chinese tutorial: http://www.runoob.com/python3/python3-tutorial.html
-
Awesome Python：https://github.com/vinta/awesome-python
-
Awesome Python Chinese version: https://github.com/jobbole/awesome-python-cn
2. Installation under Windows
There are two ways to install Python 3 under Windows.
-
One is to install via Anaconda, which provides a scientific computing environment for Python, which comes with Python and commonly used libraries. If you choose this method, the subsequent environment configuration method will be more convenient.
-
The other is to download and install the installation package directly, that is, the standard installation method.
Below we introduce these two installation methods in turn, you can choose one of them.
(1) Anaconda installation
The official download link of Anaconda is https://www.continuum.io/downloads , select the installation package of the Python 3 version to download, as shown in Figure 1-1.
Figure 1-1 Anaconda Windows download page
If the download speed is too slow, you can choose to use the mirror of Tsinghua University. The download list link is https://mirrors.tuna.tsinghua.edu.cn/anaconda/archive/ and the instruction link is https://mirrors.tuna.tsinghua. edu.cn/help/anaconda/ .
After the download is complete, just double-click the installation package to install it. After the installation is complete, the Python 3 environment is configured.
(2) Installation package installation
We recommend downloading the installation package directly to install. At this time, you can download the Python 3 installation package directly from the official website: https://www.python.org/downloads/ .
At the time of writing, the latest version 1 of Python is 3.6.2, and its download link is https://www.python.org/downloads/release/python-362/ , and the download page is shown in Figure 1-2. It should be noted that the actual latest version of Python is subject to the official website.
Figure 1-2 Python download page
-
Unless otherwise specified, the latest version in the book is the situation when the author wrote the book, and will not be explained one by one later.
Windows x86-64 executable installer can be downloaded for 64-bit systems, and Windows x86 executable installer can be downloaded for 32-bit systems.
After the download is complete, double-click the Python installation package directly, and install it through the graphical interface, and then set the Python installation path. After completion, configure the Python 3 and Python 3 Scripts directories to environment variables.
Regarding the configuration of environment variables, here is a demonstration using the Windows 10 system as an example.
If the path of Python 3 after installation is C:\Python36, open the path from the Explorer, as shown in Figure 1-3.
Figure 1-3 Python installation directory
Copy the path.
After that, right-click on "Computer" and select "Properties". The system properties window will open, as shown in Figure 1-4.
Figure 1-4 System properties
Click "Advanced System Settings" on the left, you can see the "Environment Variables" button at the bottom of the pop-up dialog box, as shown in Figure 1-5.
Figure 1-5 Advanced system settings
Click the "Environment Variables" button, find the Path variable under System Variables, and then click the "Edit" button, as shown in Figure 1-6.
Figure 1-6 Environment variables
Then click "New" to create a new entry and copy the C:\Python36 just copied into it. What needs to be explained here is that the path here is your Python 3 installation directory, please replace it yourself. Then, copy the C:\Python36\Scripts path into it, as shown in Figure 1-7.
Figure 1-7 Edit environment variables
Finally, click the "OK" button to complete the configuration of the environment variables.
After configuring the environment variables, we can execute directly on the command line executable in the path environment variable, such as python, pipcommands.
(3) Add alias
The installation can be completed by choosing one of the above two installation methods, but if you have installed Python 2 before, it may cause version conflicts. For example, you do pythonn't know if you are calling Python 2 or Python 3 when you enter it on the command line . In order to solve this problem, it is recommended to make a copy of python.exe in the installation directory and name it python3.exe so that the python3commands can be called . In fact, it pythonis exactly the same as the command, just to better distinguish the Python version. Of course, if you have not installed Python 2 before, it is also recommended to add this alias. The effect after the addition is shown in Figure 1-8.
Figure 1-8 Add alias
For pip, the pip3.exe executable file is included in the installation package, and we can also use the pip3command directly without additional configuration.
(4) Test verification
After the installation is complete, you can test whether the installation is successful through the command line. Search in the "Start" menu cmd, find the command prompt, and then enter the command line mode. Enter pythonto test whether you can successfully call Python. If you have added an alias, you can enter a python3test. The python3test result is shown in Figure 1-9.
Figure 1-9 Test verification page
The output is similar to the following:
$ python3
Python 3.6.1 (v3.6.1:69c0db5, Mar 21 2017, 17:54:52) [MSC v.1900 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> print('Hello World')
Hello World
>>> exit()
$ pip3 -V
pip 9.0.1 from c:\python36\lib\site-packages (python 3.6)
If a prompt similar to the above appears, it proves that both Python 3 and pip 3 are installed successfully; if the prompt command does not exist, please check the configuration of environment variables.
3. Installation under Linux
There are many installation methods under Linux: command installation, source installation and Anaconda installation.
Installation using source code requires self-compilation, which takes a long time. It is recommended to use the commands that come with the system or Anaconda installation, which is simple and efficient. Here are the three installation methods.
(1) Command line installation
Different Linux distributions have different installation methods, which are introduced separately here.
CentOS、Red Hat
If it is a CentOS or Red Hat version, just use the yumcommand to install it.
The installation methods of Python 3.5 and Python 3.4 are listed below, you can choose by yourself.
Python 3.5 version:
sudo yum install -y https://centos7.iuscommunity.org/ius-release.rpm sudo yum update sudo yum install -y python35u python35u-libs python35u-devel python35u-pip
After the execution is complete, you can successfully install Python 3.5 and pip 3.
Python 3.4 version:
sudo yum groupinstall -y development tools sudo yum install -y epel-release python34-devel libxslt-devel libxml2-devel openssl-devel sudo yum install -y python34 sudo yum install -y python34-setuptools sudo easy_install-3.4 pip
After the execution is complete, you can successfully install Python 3.4 and pip 3.
Ubuntu、Debian和Deepin
First install Python 3, just apt-getinstall it here . Before installation, you need to install some basic libraries. The related commands are as follows:
sudo apt-get install -y python3-dev build-essential libssl-dev libffi-dev libxml2 libxml2-dev libxslt1-dev zlib1g-dev libcurl4-openssl-dev sudo apt-get install -y python3
After executing the above command, you can successfully install Python 3.
Then you need to install pip 3. You can still use the apt-getinstallation here . The related commands are as follows:
sudo apt-get install -y python3-pip
After the execution is complete, you can successfully install Python 3 and pip 3.
(2) Source code installation
If there is a problem with the command line installation method, you can also download the Python 3 source code to install it.
The source code download address is https://www.python.org/ftp/python/ , you can choose the version you want to install. Here we take Python 3.6.2 as an example, and the installation path is set to /usr/local/python3.
First, create the installation directory, the related commands are as follows:
1 sudo mkdir /usr/local/python3
Then download the installation package and unzip it to enter, the relevant commands are as follows:
wget --no-check-certificate https://www.python.org/ftp/python/3.6.2/Python-3.6.2.tgz tar -xzvf Python-3.6.2.tgz cd Python-3.6.2
Next, compile and install. It may take a long time, please be patient, the command is as follows:
sudo ./configure --prefix=/usr/local/python3 sudo make sudo make install
After the installation is complete, create a Python 3 link. The related commands are as follows:
sudo ln -s /usr/local/python3/bin/python3 /usr/bin/python3
Then download the pip installation package and install it, the command is as follows:
wget --no-check-certificate https://github.com/pypa/pip/archive/9.0.1.tar.gz tar -xzvf 9.0.1.tar.gz cd pip-9.0.1 python3 setup.py install
After the installation is complete, create a pip 3 link. The related commands are as follows:
sudo ln -s /usr/local/python3/bin/pip /usr/bin/pip3
In this way, Python 3 and pip 3 are successfully installed.
(3) Anaconda installation
Anaconda also supports Linux. Its official download link is https://www.continuum.io/downloads . Just select the Python 3 version of the installation package to download, as shown in Figure 1-10.
Figure 1-10 Anaconda Linux download page
If the download speed is too slow, you can also use Tsinghua Mirror. For details, please refer to the introduction of Windows, so I won’t repeat it here.
(4) Test verification
Test whether Python 3 and pip 3 are installed successfully under the command line interface:
$ python3 Python 3.5.2 (default, Nov 17 2016, 17:05:23) Type "help", "copyright", "credits" or "license" for more information. >>> exit() $ pip3 -V pip 8.1.1 from /usr/lib/python3/dist-packages (python 3.5)
If a prompt similar to the above appears, it proves that Python 3 and pip 3 have been installed successfully.
4. Installation under Mac
There are also multiple installation methods under Mac, such as Homebrew, installation package installation, Anaconda installation, etc. Homebrew installation is recommended here.
(1) Homebrew installation
Homebrew is a powerful package management tool under the Mac platform. Its official website is https://brew.sh/ .
Execute the following command to install Homebrew:
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
After the installation is complete, you can brewinstall Python 3 and pip 3 using the command:
brew install python3
After the command is executed, we find that both Python 3 and pip 3 have been successfully installed.
(2) Installation package installation
You can download the Python 3 installation package from the official website. The link is https://www.python.org/downloads/ , and the page is shown in Figure 1-2.
Under the Mac platform, you can choose to download the Mac OS X 64-bit/32-bit installer. After the download is complete, open the installation package and follow the prompts to install.
(3) Anaconda installation
Anaconda also supports Mac, its official download link is: https://www.continuum.io/downloads , select the Python 3 version of the installation package to download, as shown in Figure 1-11.
Figure 1-11 Anaconda Mac download page
If the download speed is too slow, you can also use Tsinghua Mirror. For details, please refer to the introduction of Windows, so I won’t repeat it here.
(4) Test verification
Open the terminal and test whether Python 3 and pip 3 are successfully installed in the command line interface, as shown in Figure 1-12.
Figure 1-12 Test verification page
If the above prompt appears, it proves that Python 3 and pip 3 are installed successfully.