Python has a large number of third-party libraries, references to these libraries are also very convenient, by pip installthese third-party libraries can be installed to a local Python library files in the directory, then you can importinto the project, which greatly enhance the coding efficiency of the developer.
But it also raises the question: When A and B projects project references both Lib library, Lib A project requires version 1.0, B project requires Lib version is 2.0. So using the pip installcommand Lib installed directly into the local environment in the global conflict occurs, it may cause the operating environment of A and B can not be satisfied at the same time and fail.
So virtual environment (virtualenv) appeared. Its core idea is to provide an independent operating environment for each project, so that different project dependencies will not conflict.
(I want to learn programming from a small partner search circle T community , more and more industry-related industry information about free video tutorials. Oh, absolutely free!)
0x00 using venv + pip
1. Create venv
Installation is very simple virtual environment, you can use the venvmodule, for example, in the project directory Python3to create a virtual environment
➜ python3 -m venv venv
So in the project directory will have one venvof the file directory. This directory is the virtual environment of the project.
To use the virtual environment must be activated
➜ source venv/bin/activate
Then it will appear in the command line
(venv) ➜
Description virtual environment has been activated.
To cancel the virtual environment, use
(venv) ➜ deactivate
2, pip install / uninstall
After activating virtual environment can use pip command to install dependencies of the project. E.g
(venv) ➜ pip install requests
pipIt will requestsinstall venv/lib/python3.7/site-packagesdirectory.
To uninstall the dependent libraries
(venv) ➜ pip uninstall requests
3、pip list
To view the venvinstalled which dependent libraries
(venv) ➜ pip list
Package Version
-------------- --------
beautifulsoup4 4.7.1
certifi 2019.3.9
requests 2.21.0
4、pip freeze
Use pip freezecan be saved depend on the library list to a requirements.txtfile, you can let other project collaborators can quickly establish the operating environment of the project.
(venv) ➜ pip freeze > requirements.txt
So you want to install Project Dependencies
(venv) ➜ pip install -r requirements.txt
So that you can build up a unified operating environment.
5, venv + pip program problems
So far everything is running well, this should be a good solution for it. In fact this program I have been using now also, did not encounter any problems.
Until one day someone said to use venv + pip can not guarantee the consistency and reliability of my operating environment.
His reason is simple, the library project is dependent on the environment requirements.txt, and requirements.txtthe library has a version number is not specified, so that the use pip install -r requirements.txt time will lead to uncertainty version installed.
E.g
beautifulsoup4
certifi
requests
Of course, you can specify the exact version number to each library to solve, and he
beautifulsoup4 4.7.1
certifi 2019.3.9
requests 2.21.0
This will solve the above problems, but if a third-party library fixes a vulnerability, you want to use pip install --upgradeto update these dependent libraries, then it is not so easy.
Another problem requirements.txtis also likely to depend on the library version conflict occurs. The situation is this:
ALib -> sublib_1.0
BLib -> sublib_2.1
ALibAnd BLibat the same time rely on sublib, but they are not dependent on the version, the ultimate use pip install -r requirements.txtmay also be dependent on the library dependencies sub-versions are not compatible and lead project fail.
So it waspipenv
0x01 pipenv
To tell the truth before that I have been using venv+pip, when you see the above question, I feel the need to look at pipenv.
1, the installation pipenv
➜ pip install pipenv
Once installed it will refer to two files, Pipfileand Pipfile.lock. The former is used to replace requirements.txtthe file, which is used to ensure that dependent libraries certainty and consistency.
In fact, pipenvthe command also encapsulate the bottom pipand venvoperation, and provides a simple interactive commands.
Create a virtual environment
to use (such as pipenvdemo) in the project directory
➜ pipenv shell
Description similar to the following message appears when the virtual environment created and activated successfully, of course, you can also use --twoor --threeparameter to specify the use Python2or Python3create, you can also use the Python version number as determined by -python3.7
➜ pipenv shell --three
➜ pipenv shell --python3.7
Since my local computer environment is Python 3.7, so it will default here directly point to the local default version
Creating a virtualenv for this project…
Pipfile: /Users/mac/PycharmProjects/pipenvdemo/Pipfile
Using /usr/local/opt/python/bin/python3.7 (3.7.3) to create virtualenv…
⠋ Creating virtual environment...Already using interpreter /usr/local/opt/python/bin/python3.7
Using base prefix '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7'
New python executable in /Users/mac/.local/share/virtualenvs/pipenvdemo-fHPp2Wq9/bin/python3.7
Also creating executable in /Users/mac/.local/share/virtualenvs/pipenvdemo-fHPp2Wq9/bin/python
Installing setuptools, pip, wheel...
done.
✔ Successfully created virtual environment!
Virtualenv location: /Users/mac/.local/share/virtualenvs/pipenvdemo-fHPp2Wq9
Creating a Pipfile for this project…
Launching subshell in virtual environment…
. /Users/mac/.local/share/virtualenvs/pipenvdemo-fHPp2Wq9/bin/activate
Pipfile while generating a file in the project directory (such as pipenvdemo). Similarly, the command line will appear the following styles
(pipenvdemo) ➜ pipenvdemo
Where Pipfilethe content is
[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true
[dev-packages]
[packages]
[requires]
python_version = "3.7"
2、pipenv install
When you want to install third-party libraries when used directly pipenv install command, the version number specified below flask for installation.
➜ pipenv install flask==1.0.1
You may not specify a version number
➜ pipenv install flask
The following information will appear similar to the terminal
Installing flask==1.0.1…
Adding flask to Pipfile's [packages]…
✔ Installation Succeeded
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
⠙ Locking...
You can also use the -dev parameter is used to install the development environment dependent libraries
➜ pipenv install pytest --dev
In the project directory and generate another file Pipfile.lock, and Pipfile file is also updated
[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true
[dev-packages]
pytest = "*"
[packages]
flask = "==1.0.1"
[requires]
python_version = "3.7"
See the [packages]label listed under libs flask and version number above and a [dev-packages]is a marked development version dependencies, which can be used to distinguish between the production and development environments. Only used in the command --devparameters will be installed [dev-packages]following a dependent libraries.
Pipfile.lockThe contents of the file to be richer, mainly contains the library dependencies (including sub-library dependencies) version number and file hash information, which can ensure dependencies are identified.
{
"_meta": {
"hash": {
"sha256": "3bba1f1c4de8dd6f8d132dda17cd3c720372a1eed94b9523b9c23013e951c8ae"
},
"pipfile-spec": 6,
"requires": {
"python_version": "3.7"
},
"sources": [
{
"name": "pypi",
"url": "https://pypi.org/simple",
"verify_ssl": true
}
]
},
"default": {
"click": {
"hashes": [
"sha256:2335065e6395b9e67ca716de5f7526736bfa6ceead690adf616d925bdc622b13",
"sha256:5b94b49521f6456670fdb30cd82a4eca9412788a93fa6dd6df72c94d5a8ff2d7"
],
"version": "==7.0"
},
"flask": {
"hashes": [
"sha256:cfc15b45622f9cfee6b5803723070fd0f489b3bd662179195e702cb95fd924c8",
"sha256:dbe2a9f539f4d0fe26fa44c08d6e556e2a4a4dd3a3fb0550f39954cf57571363"
],
"index": "pypi",
"version": "==1.0.1"
},
"itsdangerous": {
"hashes": [
"sha256:321b033d07f2a4136d3ec762eac9f16a10ccd60f53c0c91af90217ace7ba1f19",
"sha256:b12271b2047cb23eeb98c8b5622e2e5c5e9abd9784a153e9d8ef9cb4dd09d749"
],
"version": "==1.1.0"
},
"jinja2": {
"hashes": [
"sha256:065c4f02ebe7f7cf559e49ee5a95fb800a9e4528727aec6f24402a5374c65013",
"sha256:14dd6caf1527abb21f08f86c784eac40853ba93edb79552aa1e4b8aef1b61c7b"
],
"version": "==2.10.1"
},
"markupsafe": {
"hashes": [
"sha256:00bc623926325b26bb9605ae9eae8a215691f33cae5df11ca5424f06f2d1f473",
"sha256:09027a7803a62ca78792ad89403b1b7a73a01c8cb65909cd876f7fcebd79b161",
"sha256:09c4b7f37d6c648cb13f9230d847adf22f8171b1ccc4d5682398e77f40309235",
"sha256:1027c282dad077d0bae18be6794e6b6b8c91d58ed8a8d89a89d59693b9131db5",
"sha256:24982cc2533820871eba85ba648cd53d8623687ff11cbb805be4ff7b4c971aff",
"sha256:29872e92839765e546828bb7754a68c418d927cd064fd4708fab9fe9c8bb116b",
"sha256:43a55c2930bbc139570ac2452adf3d70cdbb3cfe5912c71cdce1c2c6bbd9c5d1",
"sha256:46c99d2de99945ec5cb54f23c8cd5689f6d7177305ebff350a58ce5f8de1669e",
"sha256:500d4957e52ddc3351cabf489e79c91c17f6e0899158447047588650b5e69183",
"sha256:535f6fc4d397c1563d08b88e485c3496cf5784e927af890fb3c3aac7f933ec66",
"sha256:62fe6c95e3ec8a7fad637b7f3d372c15ec1caa01ab47926cfdf7a75b40e0eac1",
"sha256:6dd73240d2af64df90aa7c4e7481e23825ea70af4b4922f8ede5b9e35f78a3b1",
"sha256:717ba8fe3ae9cc0006d7c451f0bb265ee07739daf76355d06366154ee68d221e",
"sha256:79855e1c5b8da654cf486b830bd42c06e8780cea587384cf6545b7d9ac013a0b",
"sha256:7c1699dfe0cf8ff607dbdcc1e9b9af1755371f92a68f706051cc8c37d447c905",
"sha256:88e5fcfb52ee7b911e8bb6d6aa2fd21fbecc674eadd44118a9cc3863f938e735",
"sha256:8defac2f2ccd6805ebf65f5eeb132adcf2ab57aa11fdf4c0dd5169a004710e7d",
"sha256:98c7086708b163d425c67c7a91bad6e466bb99d797aa64f965e9d25c12111a5e",
"sha256:9add70b36c5666a2ed02b43b335fe19002ee5235efd4b8a89bfcf9005bebac0d",
"sha256:9bf40443012702a1d2070043cb6291650a0841ece432556f784f004937f0f32c",
"sha256:ade5e387d2ad0d7ebf59146cc00c8044acbd863725f887353a10df825fc8ae21",
"sha256:b00c1de48212e4cc9603895652c5c410df699856a2853135b3967591e4beebc2",
"sha256:b1282f8c00509d99fef04d8ba936b156d419be841854fe901d8ae224c59f0be5",
"sha256:b2051432115498d3562c084a49bba65d97cf251f5a331c64a12ee7e04dacc51b",
"sha256:ba59edeaa2fc6114428f1637ffff42da1e311e29382d81b339c1817d37ec93c6",
"sha256:c8716a48d94b06bb3b2524c2b77e055fb313aeb4ea620c8dd03a105574ba704f",
"sha256:cd5df75523866410809ca100dc9681e301e3c27567cf498077e8551b6d20e42f",
"sha256:e249096428b3ae81b08327a63a485ad0878de3fb939049038579ac0ef61e17e7"
],
"version": "==1.1.1"
},
"werkzeug": {
"hashes": [
"sha256:865856ebb55c4dcd0630cdd8f3331a1847a819dda7e8c750d3db6f2aa6c0209c",
"sha256:a0b915f0815982fb2a09161cb8f31708052d0951c3ba433ccc5e1aa276507ca6"
],
"version": "==0.15.4"
}
},
"develop": {}
}
3、pipenv lock
Suppose we want to publish a project to a production environment, then you should use the pipenv lockcommand
➜ pipenv lock
Terminal will be similar to the following
Locking [dev-packages] dependencies…
✔ Success!
Locking [packages] dependencies…
This command will create or update Pipfile.lock file, you need to pay attention to it is that we should not manually edit this file.
You can then use the following command to restore the environment in the production environment
➜ pipenv install --ignore-pipfile
Specify the --ignore-pipfilemeaning of the parameters, as long as the recovery Pipfile.lockdependent libraries and sub-list of dependencies. If you want to restore a development environment dependent libraries that install [dev-packages]the following dependencies, you can use
➜ pipenv install --dev
These are the simple use of pipenv.
4、pipenv graph
Now let's look at the problem mentioned above
ALib -> sublib_1.0
BLib -> sublib_2.1
A and B are dependent on the same library module, the version number of dependencies but is not the same. Then use pipenv install appeared similar to the following information
Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
Could not find a version that matches sublib=1.0,sublib=2.1
can use
➜ pipenv graph
View the current sub-project dependencies and dependencies information in a tree hierarchy fashion show
Flask==1.0.1
- click [required: >=5.1, installed: 7.0]
- itsdangerous [required: >=0.24, installed: 1.1.0]
- Jinja2 [required: >=2.10, installed: 2.10.1]
- MarkupSafe [required: >=0.23, installed: 1.1.1]
- Werkzeug [required: >=0.14, installed: 0.15.4]
pytest==4.6.2
- atomicwrites [required: >=1.0, installed: 1.3.0]
- attrs [required: >=17.4.0, installed: 19.1.0]
- importlib-metadata [required: >=0.12, installed: 0.17]
- zipp [required: >=0.5, installed: 0.5.1]
- more-itertools [required: >=4.0.0, installed: 7.0.0]
- packaging [required: Any, installed: 19.0]
- pyparsing [required: >=2.0.2, installed: 2.4.0]
- six [required: Any, installed: 1.12.0]
- pluggy [required: >=0.12,<1.0, installed: 0.12.0]
- importlib-metadata [required: >=0.12, installed: 0.17]
- zipp [required: >=0.5, installed: 0.5.1]
- py [required: >=1.5.0, installed: 1.8.0]
- six [required: >=1.10.0, installed: 1.12.0]
- wcwidth [required: Any, installed: 0.1.7]
In this example we installed the flask and pytest, pipenv graph shows the dependence of their respective commands required. You can also add parameters -reverse
➜ pipenv graph --reverse
Using this parameter can easily analyze a conflict of libraries.
atomicwrites==1.3.0
- pytest==4.6.2 [requires: atomicwrites>=1.0]
attrs==19.1.0
- pytest==4.6.2 [requires: attrs>=17.4.0]
click==7.0
- Flask==1.0.1 [requires: click>=5.1]
itsdangerous==1.1.0
- Flask==1.0.1 [requires: itsdangerous>=0.24]
MarkupSafe==1.1.1
- Jinja2==2.10.1 [requires: MarkupSafe>=0.23]
- Flask==1.0.1 [requires: Jinja2>=2.10]
more-itertools==7.0.0
- pytest==4.6.2 [requires: more-itertools>=4.0.0]
pip==19.1.1
py==1.8.0
- pytest==4.6.2 [requires: py>=1.5.0]
pyparsing==2.4.0
- packaging==19.0 [requires: pyparsing>=2.0.2]
- pytest==4.6.2 [requires: packaging]
setuptools==41.0.1
six==1.12.0
- packaging==19.0 [requires: six]
- pytest==4.6.2 [requires: packaging]
- pytest==4.6.2 [requires: six>=1.10.0]
wcwidth==0.1.7
- pytest==4.6.2 [requires: wcwidth]
Werkzeug==0.15.4
- Flask==1.0.1 [requires: Werkzeug>=0.14]
wheel==0.33.4
zipp==0.5.1
- importlib-metadata==0.17 [requires: zipp>=0.5]
- pluggy==0.12.0 [requires: importlib-metadata>=0.12]
- pytest==4.6.2 [requires: pluggy>=0.12,<1.0]
- pytest==4.6.2 [requires: importlib-metadata>=0.12]
5, other commands
To delete a dependent package
➜ pipenv uninstall numpy
Copy the code to remove all dependent libraries
➜ pipenv uninstall --all
➜ pipenv uninstall --all-dev
--all-devIt is designated to delete all dependent on development environment.
View venv directory path
➜ pipenv --venv
/Users/mac/.local/share/virtualenvs/pipenvdemo-fHPp2Wq9
View the current project path
➜ pipenv --where
/Users/mac/PycharmProjects/pipenvdemo
0x02 summarize
From the current use of pip+venvproblem encountered departure package management tool, referred to problems encountered in this way is dependent libraries and their dependencies and sub-question version management and conflict may occur. So it was suggested that a new package management tool pipenv. It has two core files Pipfileand Pipfile.lockdocuments, the former is used to specify libraries directly dependent on information about the current project, which is designated a library information dependent dependent dependent by hashdetermining the value of dependent libraries consistency and version number.
0x03 references
- https://docs.python.org/3/tutorial/venv.html
virtual environment and package management official documents - https://realpython.com/pipenv-guide/
A Guide to the New Python Packaging Tool - https://www.kennethreitz.org/essays/a-better-pip-workflow
pipenv authors describe the use of pip on the issue of virtualenv - https://packaging.python.org/tutorials/managing-dependencies/#managing-dependencies
rely on database management official documents