Let’s talk about the conclusion first. If conda and pip are used together in the same environment, especially if these two tools are used frequently to install packages, the environment may be confused.
Just like other package managers, most of these problems stem from the limited ability to manage uninstalled packages. Running conda after installing it via pip may overwrite or potentially corrupt packages installed via pip. Similarly, pip may upgrade or remove packages that a certain package installed by conda depends on.
There are some steps you can take to avoid breaking the environment that can cause problems when using conda and pip together. A reliable way is to just use the conda package. If you need a package that conda does not provide, you can use conda build to create it. For projects available on PyPI, the conda skeleton command can generate a recipe for creating a conda package with little modification.
https://docs.conda.io/projects/conda-build/en/stable/
But this can become a burden if the environment involves a large number of packages that are only available on PyPI. In these cases, the safest thing to do is to install all supported packages via conda first and then use pip to install unsupported packages. Additionally, pip should be run with the "--upgrade-strategy only-if-needed" parameter to prevent unnecessary upgrades of packages installed via conda. This is the default when running pip and should not be changed
If a project is expected to require mixing conda and pip, the best practice is to put this installation into a dedicated conda environment to protect other environments from any modifications that pip may make, because the conda environment is independent and allows different version of the package exists
https://conda.io/docs/user-guide/tasks/manage-environments.html
In a conda environment, use hard links instead of copying files whenever possible. If the same set of packages is installed, each new conda environment will require only a small amount of additional disk space. Many users rely on a "base" conda environment created by installing Anaconda or Miniconda, and if this environment is mixed with packages installed by pip and conda, it will be more difficult to restore. On the other hand, creating a separate conda environment allows the entire environment to be deleted and recreated at any time without affecting core conda functionality
When using pip to install packages in a conda environment, conda will not be aware of these changes and risks damaging the environment. A reliable method is to create a new environment and install the required packages through conda first, and then run pip. Again, the main problem is the "statefulness (or associativity)" of pip - the order in which packages are installed will have more state associated with it, which will make things more difficult to work properly
For environments that need to be rebuilt frequently, best practice is to store conda and pip package requirements in text files. Package requirements can be provided to conda through the –file parameter, and to pip through the –r or –requirement (some summaries about requirement have actually been written long ago and have been lying in my draft box)
A file containing conda and pip requirements can be exported or controlled via the conda env command. The advantage of these two methods is that the files describing the environment can be easily versioned and shared with others.
Anaconda is very aware of the difficulties you may encounter when combining pip and conda. We want the process of setting up a data science environment to be as easy as possible. That's why we've been adding new features to the next version of conda to make this process easier. Although still in beta, conda 4.6.0 allows conda to take pip-installed packages into account and replace them as needed or satisfy dependencies of existing packages. We are still testing these new features, but expect the interaction between conda and pip to be greatly improved in the near future.
Summarize
Use pip after using conda
-
Use conda to install as many supported packages as possible first, then use pip
-
Run pip with "--upgrade-strategy only-if-needed" (default setting)
-
Do not use the "--user" parameter to avoid all "user" installations
Using conda for environment isolation
-
Create a conda environment to isolate any changes made by pip
-
Due to the hard link feature, repeated installation of packages in different environments will not copy files, and the environment occupies very little space.
-
Care should be taken to avoid running pip in the default conda environment (such as base)
If changes need to be made to the environment, recreate the environment
-
Once you use pip to install a package in the conda environment, conda cannot locate changes in the environment.
-
To install additional conda packages, it is best to recreate the environment
Store conda and pip related package requirements in the form of text files
-
Provide package requirements to conda via the --file parameter
-
Provided to pip via the -r or --requirement parameter
-
conda env will export or create an environment based on a file containing conda and pip requirements