Step-by-step guide to get you started with deep learning (1): Nanny-level Anaconda and PyTorch environment configuration guide

Station B: Update soon! ! ! _bilibili
CSDN: Take you step by step to get started with deep learning (1): Nanny-level Anaconda and PyTorch environment configuration guide_Closed after a hundred years - CSDN blog Github: Closed after a hundred years Public
account :
Closed after a hundred years

1. Preface and Preparatory Work

Hello, I am Feng Bi!

Today, with the continuous evolution of deep learning technology, our lives are undergoing earth-shaking changes. Whether it is computer vision, natural language processing or recommendation technology, there are varying degrees of innovation. With the continuous improvement of tools and knowledge, the threshold for us to learn deep learning is getting lower and lower. For students who want to get started quickly, they often need to collect various materials, and it is difficult to find a complete introductory tutorial. Here, in order to satisfy those who are new to deep learning To meet the needs of students, let’s record the series of articles about getting started with deep learning. When it comes to deep learning frameworks, PyTorch is easy to get started and is used most in academic circles. To configure the PyTorch environment, it is best to first install the python environment and Anaconda package. Management tools, this article will introduce you to deep learning - the whole process of Python and Anaconda environment configuration, and teach you step by step how to complete the Python deep learning environment configuration!

1.1 The relationship between python, anaconda and pytorch

I believe that friends who are new to deep learning may be confused by the terms python, anaconda and pytorch. They don’t know which one should be installed first and what is the use of each one. Here, I will briefly introduce what these three are and their respective functions. I believe that after understanding the relationship between these three, we will be more comfortable when configuring ourselves! Let’s first take a look at what the three of them are:

• Python is a widely used language 编程语言and has applications in many fields. It has a concise syntax, is easy to learn, and has a large number of third-party libraries to use.

• Anaconda is a Python 包和环境管理软件that provides many libraries and tools for data science, machine learning and scientific computing. It also provides a package manager, conda, to easily install and manage Python libraries and other packages.

• PyTorch is a tool 开源深度学习框架developed based on Python. It provides a large number of functions and tools to help users train and develop neural network models. PyTorch can be installed via pip or conda, so it has a close relationship with Anaconda and Python.

So through the above explanation, combined it means:

First of all, we need to configure the Python language environment for our basic programming; however, in actual development, we often need to install different third-party libraries for different projects, including several packages of project A and packages of project B. The versions may be conflicting, so it may be necessary to repeatedly change the package version in the environment. Isn't this very troublesome? At this time, Anaconda is needed to help us create different virtual environments. You can configure the packages and versions corresponding to the project between each environment, so that there will be no conflicts and it will appear very elegant. So the last pytorch, it actually relies on the framework of some other third-party libraries. Of course, it can also be regarded as a third-party library. It can be considered that its role is a framework specifically used for deep learning, so we generally use it specifically for deep learning. He creates a virtual environment. Isn’t it very impressive? That’s right.

2. Anconda installation

Many students may be confused. Isn’t it said that you need to configure the Python environment first? Why is anaconda mentioned here? In fact, it is not the case. Anaconda itself will come with one after installation, which contains the latest python interpreter and related package management base环境tools. So we don’t need to install python separately at all, and if you just use python to kill chickens, then you don’t need anaconda, hhh~.

Here we mainly demonstrate based on the configuration method of the anaconda environment under Windows. The configurations on Linux and Mac are actually similar.

Without further ado, let’s get started!

2.1 Install anaconda

1. First go to anaconda official website to download the installation package
   

Generally, it is very fast to download. If you think the download speed is slow, you can also go to Tsinghua Source to have a look. As shown below, just choose the latest version that is suitable for your computer. For Windows, it is Anaconda3-2022.10 -Windows-x86_64.exe , if you are using a Mac, then you can choose Anaconda3-2022.10-MacOSX-x86_64.pkg above .

2. Open the installation package you just downloaded, and then keep pressing Next. You need to install it step by step according to the requirements. Just keep pressing Next, as shown below.

Then comes the installation process. I won’t continue here. Just wait for the installation. You can save this article!

4. Check whether anaconda is installed successfully

Press win+ rkey, enter cmd, opencmd

Input conda -V, there are generally two situations. The following one is installed.

If an error is reported, you need to find the installation path of anaconda first, for example, it is in C:\Users\xxx\anaconda3, then you need to add the following paths to your system environment variables, paththat is, add the binand Scriptspath of anaconda Go to the environment variables as follows:

C:\Users\xxx\anaconda3\bin
C:\Users\xxx\anaconda3\Scripts
C:\Users\xxx\anaconda3

After the operation, just check whether conda is installed. 一般出问题就是环境变量没加入进去Otherwise, there is no problem.

2.2 Change pip source and anaconda source

When we install packages, we often encounter problems with poor network or slow downloading. At this time, we need to configure the pip source and anaconda source. The configuration methods of the pip source and anaconda source are given below:

2.2.1 Configure pip source

In order to solve the problem of installing related dependency packages and downloading python in China very slowly, we need to configure the pip mirror source first. The specific method is as follows:

1. C:\Users\xxxCreate a pipfolder named in, and then create a file in it . pip.iniNote that you need to modify the content of this file later, so you can change the name to first pip.txt, and then change it topip.ini
   
2. Modify pip.inithe file content and save it! It's that simple~

[global]
index-url = https://pypi.tuna.tsinghua.edu.cn/simple

Of course, you can also use the following sources to replace the values pip.ini​​in the above file index-url, as follows:

• Tsinghua University: https://pypi.tuna.tsinghua.edu.cn/simple/
• Alibaba Cloud: http://mirrors.aliyun.com/pypi/simple/
• University of Science and Technology of China: http://pypi.mirrors.ustc.edu.cn/simple/

2.2.2 Configure anaconda source

The configuration method of anconda source is as follows:

1. C:\Users\xxxFind a file called in , .condarcif it doesn’t exist, create one!

2. Then right-click to open it with Notepad or vscode! Paste the following paragraph into this file

channels:
  - defaults
show_channel_urls: true
default_channels:
  - http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
  - http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r
  - http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2
custom_channels:
  conda-forge: http://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
  msys2: http://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
  bioconda: http://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
  menpo: http://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
  pytorch: http://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
  simpleitk: http://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud

If you are unclear, you can also refer to this official tutorial from Tsinghua Source or leave a comment to me! In the official tutorial, the address may start with https, but it may actually be wrong. Then you can change httpsit httpand it will be ok. What I give you here is http~

2.3 Create a virtual environment

Okay, everything is ready! We can finally create our own virtual environment for different project needs smoothly, and install different python versions and third-party libraries in the virtual environment. Without further ado, let’s get started!

1. Let’s first take a look at what virtual environments we have

Let’s open cmd first and enter conda env list, as shown in the picture below. You can see that we have many virtual environments. The most basic one is the base environment, and the others were created by me before. The front is the environment name, followed by the corresponding environment-related dependencies and package paths.

2. Create a virtual environment

Then we enter the following command to create a virtual environment 3.9named python version my_env, as shown below:

conda create -n my_env python=3.9

Enter yto continue to the next step, he will quickly install some packages, and then it will be ok when you see the following

3. Activate the virtual environment and take a look

Okay, you already have your own virtual environment, let's take a look at what's in it? And how to use the python interpreter of your virtual environment to run python code?

First, open cmd and use activate my_envor conda activate my_envto activate your virtual environment. You will see a (my_env) in front of the command line, indicating that you have entered. Then enter conda list, and you can see the packages and version numbers in your virtual environment.

Let’s briefly test python here. First cmdenter in python, enter the python terminal, and then enter the following code, magic will be generated and love will be emitted!

print('\n'.join([''.join([('Love'[(x-y) % len('Love')] if ((x*0.05)**2+(y*0.1)**2-1)**3-(x*0.05)**2*(y*0.1)**3 <= 0 else ' ') for x in range(-30, 30)]) for y in range(30, -30, -1)]))

Of course, if you want to exit, exit()just enter.

2.4 Commonly used conda commands

Here are some commonly used conda commands for your reference:

# 建立新环境
conda create -n new env_name  python=3.8
# conda初始化
conda init
# 激活虚拟环境
conda activate env_name 或者  activate env_name
# 查看虚拟环境
conda env list
# 删除虚拟环境
conda remove -n env_name --all

3. PyTorch installation

Okay, now we have reached the last stage of environment configuration. Thank you for your persistence!

So far, there have been many frameworks for us to use in deep learning research. In fact, in the final analysis, we can think of the deep learning framework as an alchemy furnace. Others have already built the furnace for us. We only need to follow the steps to use the furnace to make elixirs. Currently, the more mainstream deep learning frameworks include: PyTorch, Tensorflow, Caffe, etc. Of course, in order to meet the needs of client-side model development and deployment, client-side frameworks such as MNN and TNN are also very commonly used. PyTorch is used more in scientific research, so let’s follow the installation of PyTorch as an example here. If you have other needs, you can leave a message and let me know, and I can make an extra chapter hhh~

Configuring the Pytorch environment is actually divided into two steps. It will be different according to the limitations of our own computing power equipment. It is mainly divided into CPU and GPU versions. GPU is mainly for NVIDIA graphics cards. It is relatively troublesome to install. CPU The version can be used for some testing. Compared with the GPU version, it lacks parallel acceleration during training, so the speed will be much slower. How to tell if you have an NVIDIA graphics card? Open the task manager and click GPU. If the words in the red box appear, then you are very lucky. You are a distinguished graphics card owner and can install the GPU version of PyTorch.

But if like the picture below, you don’t have a graphics card and only have an integrated graphics card, don’t panic. We also have a CPU version of PyTorch, if you can tolerate the buzzing sound of the fan during model training and the endless training progress. If so, that’s totally fine hahaha, or if you just want to run a code test, then the CPU is completely fine.

Without further ado, let’s get started!

3.1 Install CPU version PyTorch

3.1.1 Quick installation

The installation of the CPU version is very simple.

1. First, let’s go to PyTorch’s official website and take a look:
   

2. Select windows, pythonand cpuversion and we can get the blue command, see!

conda install pytorch torchvision torchaudio cpuonly -c pytorch

Where should this command be run? In fact, we only need to activate the my_env virtual environment created by Anaconda in the second step in cmd, and then enter the above command, as shown below:

processdEnter it yand wait patiently for automatic installation~

3. Verify installation was successful

We can enter in the cmd after installation, pythonand then enter the two pieces of code in the red box. The first piece is to import the torch package, then create a 5*3 tensor vector and print it out; the second piece is to check the gpu Whether it is available, the cpu version of gpu is definitely not available (you can’t make something out of nothing, hahaha):

import torch
x = torch.rand(5, 3)
print(x)

import torch
torch.cuda.is_available()

If you have a result similar to the one shown above, errorbut it does not appear, congratulations, the installation is successful. You can already run many models and codes pre-trained by others~

3.1.2 Install according to the specified version (advanced)

Many times we need to install the PyTorch version according to the requirements of the project. At this time, there are generally two types:

1. Install according to the project's requirement.txt

For example, we need to configure the yolov5 environment for target detection. First download the yolov5 project, and then pay attention to its contents requirements.txt, as shown in the figure below.

We enter the folder of yolov5, then open cmd, activate the virtual environment, and then enter pip install -r requirements.txt, as shown in the figure below, to install according to the project requirements.

2. Choose the version you need to install

This degree of freedom is higher. When you have clear requirements for your version, you can directly go to the historical version of pytorch to find the package you want. For example: if your platform is Linux or Windows, you need to use pip to install your package. , and want to install the CPU version of pytorch, version 1.8.1, then you can directly copy it pip命令, as shown below:

pip install torch==1.8.1+cpu torchvision==0.9.1+cpu torchaudio==0.8.1 -f https://download.pytorch.org/whl/torch_stable.html

Of course, you can also use conda to install, or choose different instructions according to the platform, package version, cpu or gpu. This is suitable for students who have a good understanding of environmental requirements! Students who are new to learning can skip it first. After you have been beaten by the environment configuration, you should be more clear about your needs.

3.2 Install GPU version PyTorch

This section mainly involves three parts:

• One is the installation of the graphics card driver, that iscuda
• One is cudnnthe installation and cudatoolkit installation
• There is also gpu版本PyTorchthe installation. This is similar to the cpu version. Just select the command to install it and it will be ok.

You can refer to my previous article for this article. The nvidia 3080 graphics card was used at that time, the instructions for configuring the pytorch1.7.1 version on the windows platform, and the detailed installation process of 3080+pytorch1.7+windows .