Python project packaging and deployment (3): the actual operation process of packaging and deployment

Other chapters

• Python project packaging and deployment (1): The concept and relationship between modules and packages
• Python project packaging and deployment (2): The role and content of init.py
• Project structure and code organization best practices for various Python projects

In the actual project development process, standardized project packaging and deployment processes, for open source projects, can help you publish the project to a public warehouse to facilitate users to download and install. For commercial projects, you can establish a private PyPi Server on a cloud server to achieve online publishing and improve deployment efficiency.

Packaging and deployment process of Python project

1. Create and configure projects
2. Package project
3. Upload the project to PyPi or a private repository
4. Download and install projects

Project release component

After packaging, two main publishing components (artifect) will be generated

• Source file package : Python source file, abbreviated as sdist, including .py, resource files, data files, etc.
• Binary package , compiled binary format, usually wheel format, can also be merged and packaged with non-Python third-party libraries. Its installer is pip.

There are two formats of Python binary packages: Wheel and Egg. The Egg format was introduced by setuptools in 2004, and the Wheel format was introduced in PEP 427 in 2012. Wheel is currently considered the standard for built-in and binary packaging of Python.

3. Introduction to packaging tools

The oldest python packaging tool is distutils , which uses setup.pyas the configuration file. It was later replaced by the setuptools tool, and the configuration file is pyproject.toml, but it can also be used setup.pyas a configuration file.
Other packaging tools include: Poetry tool. Pipenv, PDM

setuptools packaging steps

4. Detailed explanation of packaging steps

This article uses the officially recommended packaging tool setuptools,configuration file pyproject.toml .

1) Install setuptools

Installation command

pip install --upgrade setuptools

Update build module

pip install --upgrade build

2) Basic usage

Create 1 pyproject.toml, add build-system section configuration

[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

Usually, you need to add more section configuration sections, metadata, contents, dependencies, etc. These contents can also be included in setup.cfgfiles.

Such as pyproject.toml

[project]
name = "mypackage"
version = "0.0.1"
dependencies = [
    "requests",
    'importlib-metadata; python_version<"3.8"',
]

setup.cfg

[metadata]
name = mypackage
version = 0.0.1

[options]
install_requires =
    requests
    importlib-metadata; python_version < "3.8"

3) Organize the project structure

Project files can be organized as follows:

mypackage
├── pyproject.toml  
|   README.rst or README.md 
|   # LICENCE (properly chosen license information, e.g. MIT, BSD-3, GPL-3, MPL-2, etc...)
└── mypackage
    ├── __init__.py
    └── ... (other Python files)

4) Compile project

In the pyproject.toml directory, run the packaging command,

py -m build

This command will automatically search the package subdirectory and generate sdist and wheel artifacts.

dist/
├── example_package_YOUR_USERNAME_HERE-0.0.1-py3-none-any.whl
└── example_package_YOUR_USERNAME_HERE-0.0.1.tar.gz

Among them: *.tar.gz is the source file package, .whl is the built distribution, which is the compiled binary package. During installation, pip installs the wheel package first.

Specify the source file directory and only generate sdist packages

python -m build --sdist source-directory

For commercial projects, if you do not need to release the source code, you can only generate the wheel package.

python3 -m build --wheel source-tree-directory

5) Upload the package to PyPi

To register the same account

Test account address: https://test.pypi.org/account/register/

To upload files, a token is also required.

In order to upload your project securely, you need a PyPiI API token . It can be created at https://test.pypi.org/manage/account/#api-tokens. Set "Scope" to "Entie Account" . Do not close the page before copying and saving the token - otherwise, it will not work. You will see the token again

The tool for uploading files is twine. The installation command is as follows

py -m pip install --upgrade twine

Run in the dist subdirectory to upload all files

py -m twine upload --repository testpypi dist/*

After the above command is run, you will be prompted to enter username and password. Use username __token__. For password, use toke value, including pypi-prefix.

D:\workplace\python\guangda>python -m twine upload --repository testpypi dist/*
Uploading distributions to https://test.pypi.org/legacy/
Enter your username: peter
Enter your password:
Uploading face_push-0.0.1-py3-none-any.whl
100% ---------------------------------------- 18.6/18.6 kB • 00:00 • ?
Uploading face_push-0.0.1.tar.gz
100% ---------------------------------------- 17.5/17.5 kB • 00:00 • ?

View at:
https://test.pypi.org/project/face-push/0.0.1/

After uploading is complete, verify

https://test.pypi.org/project/example_package

5. Download and install projects

It is recommended to add a new virtual environment for verification. Check if the installation is successful

py -m pip install --index-url https://test.pypi.org/simple/ --no-deps example-package

py -m pip install --index-url https://test.pypi.org/simple/ --no-deps example-package-YOUR-USERNAME-HERE

Note that TestPyPI is used for testing.

After the installation is complete, import the package for testing

from example_package import example
example.add_one(2)

After getting familiar with it, the official PyPI account registration address:

https://pypi.org

To privately deploy PyPI server, please refer to the official documentation of pypiserver