Some possible solutions to the slow speed of conda install / Solving environment in Anaconda

Some possible solutions to the slow speed of conda install / Solving environment in Anaconda

question

Today, I encountered a problem when using the Linux host in the laboratory to create a new conda environment. When using the conda install in Anaconda to install the package, I will be stuck at the Solving environment step, and sometimes I will fail to retry many times .

reason

Conda contains more and more software, and different versions of the software are preserved. The index file of the software is getting bigger and bigger. When installing a new software, the search space for the software that meets all the software dependencies in the environment will be more and more. The larger the value, the slower the solving environment will be. It is very time-consuming to determine the compatibility between the dependent packages of the package to be installed and the compatibility between the installed software. It is very time-consuming to obtain the package that needs to be downloaded and the corresponding version.

Solution

Method 1: Use pip install instead of conda install

Don’t worry, this is something I want to understand recently, and it is also the only useful method I have tried so far. Although it is stupid, it is useful ( I am a novice and stupid, and the big guys laugh ):

No conda install, use directly pip installorpip3 install

principle

Rather than talking about the principle, it is better to talk about the reason why my head is down. The Windows system on my own computer has the local original Python environment and the environment in Anaconda's base , so calling python directly may call the local unconfigured environment. Corresponding Of course, the pip command will also be written into the corresponding local environment
, but it can be realized by moving up the environment variable of conda or deleting the local environment

But when you enter the Anaconda environment you set up, whether it is conda installor pip installwill be installed in the corresponding conda virtual environment, so you only need to consider one of the installation methods

Many places say condathat the speed is pipfaster and so on, so it is recommended to use it conda, but after setting up the mirror, the original pipmethod is actually very fast
. Let’s put a few domestic mirror sources first:

#中科大源、清华源、豆瓣源
https://pypi.mirrors.ustc.edu.cn/simple/
https://pypi.tuna.tsinghua.edu.cn/simple
https//pypi.doubanio.com/simple/ 

If seeing this can help you, congratulations! If you don't understand the specific operation, please continue to read below!

Method 2: Install with mamba

Mamba is a CLI tool used to manage the environment. Compared with conda's management of packages and environments, the process of checking packages and dependencies during installation is the most time-consuming process. Mamba can implement parallel computing and the speed is relatively fast

install mamba

 conda install mamba -n base -c conda-forge

The way to install through mamba is the same as conda

mamba install package

However, the process of installing mamba also requires conda install and then gets stuck in the Solving environment. The logic is closed.

Method 3: Replace the image source

This is also one of the methods mentioned in many blogs. I think it is a temporary solution, not the root cause, or even a temporary solution , because the speed bottleneck is mainly in checking packages and dependencies , and has little to do with network speed. Regarding the method of setting and changing mirror sources , I It is introduced in the later part, if you need it, you can look back!

Method 4: Update conda

conda update -n env_name conda

This method has not been verified, but you may encounter an update failure when updating. Sometimes it is because the update is miniconda, you can try to condachange it to anaconda:

conda update -n env_name anaconda

conda common virtual environment operations

Create a virtual environment

conda  create  --name  env_name python=version 

The -nlatter is the name of your own virtual environment, and versionpart of it is the Python version you specified. Note that the Python version basically cannot be changed after setting, otherwise many related dependent versions will also change,
but in the environment after you create it There won’t be so many rich packages in Anaconda’s native base environment (I don’t know if it’s my problem), so I choose to clone the base environment directly. This has the advantage that many necessary packages do not need to be downloaded again. The disadvantage is that the Python version is dead

Clone the virtual environment

conda  create  -n  new_env_name  --clone  old_env_name

Among them new_env_nameis the name of your newly created environment, old_env_namewhich is the environment you want to clone, if baseso:

conda  create  -n  new_env_name  --clone  base

Enter the virtual environment

conda activate env_name

Exit the virtual environment

conda deactivate

Delete the virtual environment

conda remove  --name  env_name  --all

View virtual environment

conda env list

or

conda info -e

Add mirror source

Use pip to add mirror source path

When using pipit, it can be realized by adding directly in front of the path -i, that is, adding after the command:

#中科大源、清华源、豆瓣源
-i https://pypi.mirrors.ustc.edu.cn/simple/
-i https://pypi.tuna.tsinghua.edu.cn/simple
-i https//pypi.doubanio.com/simple/ 

Use conda to add mirror source configuration

When using conda, it is generally necessary to configure the mirror source first instead of adding instructions directly after the command

Instructions to view mirror sources

conda config --show channels

You can also pass the following command:

conda config --show-sources

The one in the picture - defaultsis the default source. Even if it is not set, there will be some domestic mirror sources. For example, when I installed PyTorch, I downloaded it from the mirror source of Alibaba Cloud:

Collecting nvidia-cudnn-cu11==8.5.0.96
  Downloading https://mirrors.aliyun.com/pypi/packages/dc/30/66d4347d6e864334da5bb1c7571305e501dcb11b9155971421bb7bb5315f/nvidia_cudnn_cu11-8.5.0.96-2-py3-none-manylinux1_x86_64.whl (557.1 MB)

Instructions for adding mirror sources

conda config --add channels mirror_url

where mirror_urlis the URL of the mirror source

# 清华源
conda config --add channels https://pypi.tuna.tsinghua.edu.cn/simple

# 阿里源
conda config --add channels https://mirrors.aliyun.com/pypi/simple/

#豆瓣源
conda config --add channels https://pypi.doubanio.com/simple/ 

#中科大源
conda config --add channels https://pypi.mirrors.ustc.edu.cn/simple/

Command to delete mirror source

conda config --remove channels https://pypi.tuna.tsinghua.edu.cn/simple

Sometimes the mirror source fails. At this time, you can delete the corresponding mirror source through the above command.

example

I didn’t have time to record the process of installing PyTorch and TensorFlow. The following is the situation of changing the PyTorch version later.
When installing PyTorch, I installed version 11.6, but it is not compatible with the graphics card driver. You can check the CUDA version through the following command:

nvidia-smi

You can see that the CUDA version is 10.1 in the upper right corner.
Since the PyTorch version is strictly bound to the CUDA version, if you use a higher version of PyTorch to force the call, it will display that the CUDA and graphics card driver versions are too old. Because it is troublesome to change the graphics card driver and CUDA version, I tried to install the 10.1 version of the PyTorch
official website. The current version is only 11.6 and 11.7, but the previous version can be found through the following link:
PyTorch official website old version link
contains 10.1 Versions up to PyTorch version only up to 1.7.1:

# CUDA 9.2
conda install pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 cudatoolkit=9.2 -c pytorch

# CUDA 10.1
conda install pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 cudatoolkit=10.1 -c pytorch

# CUDA 10.2
conda install pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 cudatoolkit=10.2 -c pytorch

# CUDA 11.0
conda install pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 cudatoolkit=11.0 -c pytorch

# CPU Only
conda install pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 cpuonly -c pytorch

However, in addition to the problem of conda install, the above command will also encounter the situation that some mirror sources cannot find the corresponding version, so the following method is used:

# CUDA 11.0
pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

# CUDA 10.2
pip install torch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2

# CUDA 10.1
pip install torch==1.7.1+cu101 torchvision==0.8.2+cu101 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

# CUDA 9.2
pip install torch==1.7.1+cu92 torchvision==0.8.2+cu92 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

# CPU only
pip install torch==1.7.1+cpu torchvision==0.8.2+cpu torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

I installed version 1.6.0 of PyTorch's 10.1CUDA, but I encountered a torchaudioversion mismatch problem:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
torchaudio 0.13.1 requires torch==1.13.1, but you have torch 1.6.0+cu101 which is incompatible.

It doesn't matter, I will take action, just update torchaudioto the specified version, you can know that the 1.6.0 version of PyTorch corresponds to the 0.6.0 version , and the python version is 3.6~3.8 by consulting the torchaudio official documentation
torchaudio

torchaudioAfter successfully installing version 0.6.0,
test it:

All works fine!