Publish python packages to PyPI and make whl files

Reference link:
The difference between wheel and egg
How to convey your own package to PyPi and
publish your own wheel - PyPI packaging and uploading practice
PyPI official website uploading package tutorial

wheel file

Wheel and Egg are both python packaging formats. The purpose is to support the installation process that does not require compilation or production. In fact, it is also a compressed file. Change the suffix of .whl to .zip to see the contents of the compressed package. According to the official website, wheels are the new standard for Python distributions and will replace .egg.
The Egg format was introduced by setuptools in 2004, while the Wheel format was defined by PEP427 in 2012. Wheel is now considered the standard format for Python's binary packages.

Here are the main differences between Wheel and Egg:

• Wheel has an official PEP427 definition, while Egg has no PEP definition.
• Wheel is a distribution format, a packaging format. Egg is both a distribution format and a runtime installation format, and can be imported.
• Wheel file will not contain .pyc file
• Wheel uses the PEP376 compatible .dist-info directory, while Egg uses the .egg-info directory.
• Wheel has a richer naming convention.
• Wheels are versioned, and each Wheel file contains a version of the wheel specification and an implementation that packages it.
• Wheel is managed internally by the sysconfig path type, so moving to other formats is also easier.

distutils and setuptools tools

It is used to build and install additional modules in the Python environment. The module can be based on Python, or it can be an extension module written in C/C++, or it can be a python package, which contains modules written in C and Python. setuptools is a sub-project of the Python Enterprise Application Kit (PEAK), which is an enhanced version of a set of Python distutilsde tools (applicable to Python 2.3. Makes it easier for programmers to create and distribute Python packages, especially those with dependencies on other packages.

The focus of setuptools is to write the setup.py file:

Setup.py parameter introduction:

• name : the filename of the packaged package
• version : version number, added as the suffix of the packaged file
• author : author
• author_email : the email address of the author
• py_modules : packaged .py files
• packages: packaged python folders
• include_package_data : There will be some non-py files in the project, such as html and js, etc. At this time, it is necessary to rely on include_package_data and package_data to specify. package_data: Generally written as {'your_package_name': ["files"]}, include_package_data is not finished yet, and the MANIFEST.in file needs to be modified. The syntax of the MANIFEST.in file is: include xxx/xxx/xxx/ .ini / (all with . The file ending with ini can also directly specify the file name)
• license : supported open source protocols
• description : a brief description of the project
• ext_modules : is a list containing Extension instances, and the definition of Extension also has some parameters.
• ext_package : define the relative path of the extension
• requires : defines which modules depend on
• provides : defines which modules can provide dependencies
• data_files : Specifies other files (such as configuration files) that specify which files are installed in which directories. If the directory name is a relative path, it is relative to sys.prefix or sys.exec_prefix. If no template is provided, it will be added to the MANIFEST file.

Encapsulate python files (.py) into modules that can be installed and used

Reference: https://docs.python.org/3.6/distutils/introduction.html#distutils-simple-example
printtest.py

def test():
    print('print test')

To make the above .py file into a python module, you need to create a setup.py file in the same directory, and enter the configuration information in setup.py:

from setuptools import setup
setup(name='printtest',
      version='1.0',
      py_modules=['printtest'],
      )

Open a terminal, navigate to the folder, and enter:

python setup.py sdist

At this time, a dist folder is generated in the directory, and there is a testpg-1.0.tar.gz file in the folder. If the user installs, only the testpg-1.0.tar.gz file is needed. Unzip this file to get the testpg-1.0 folder, you will find that the folder contains the 3 py files we just wrote, and a PKG-INFO, open the file, and the specific information of the module will be displayed: Since we have not set it, So for UNKOWN

Metadata-Version: 1.0
Name: printtest
Version: 1.0
Summary: UNKNOWN
Home-page: UNKNOWN
Author: UNKNOWN
Author-email: UNKNOWN
License: UNKNOWN
Description: UNKNOWN
Platform: UNKNOWN

Locate the terminal to this folder, enter the following command, the module will be installed in the Lib/site-packages directory corresponding to the interpreter:

python setup.py install

After installation, you will find the printtest.py file and the printtest-1.0-py3.6.egg-info
application in the Lib/site-packages directory:

import printtest

printtest.test()

output:

print test

Wrapping Python packages

A single Python file becomes a Python module when imported, while a folder containing multiple Python files becomes a Python package. This section mainly describes how to encapsulate a Python package.
1. Create a folder, put the pagtest folder that needs to be encapsulated (the .py file in it, need to include a __init__.pyfile, and the content can be empty) into the folder, and then create a setup.py file to configure the package:

from setuptools import setup

setup(name='pagtest',
      version='1.0.0',
      description='A print test for PyPI',
      author='winycg',
      author_email='[email protected]',
      url='https://www.python.org/',
      license='MIT',
      keywords='ga nn',
      project_urls={
            'Documentation': 'https://packaging.python.org/tutorials/distributing-packages/',
            'Funding': 'https://donate.pypi.org',
            'Source': 'https://github.com/pypa/sampleproject/',
            'Tracker': 'https://github.com/pypa/sampleproject/issues',
      },
      packages=['pagtest'],
      install_requires=['numpy>=1.14', 'tensorflow>=1.7'],
      python_requires='>=3'
     )

2. Create a README.txt file to describe the installation and usage information of the
file 3. The directory structure of the current folder is:

pagtest/
    __init__.py
    print1.py
    print2.py
setup.py
README.txt

Enter the following command to package and make a source distribution. This command will package all the content in the dist/ directory as pagtest-1.0.0.tar.gz

python setup.py sdist

4. Upload to PyPI. When uploading, you can create an account verification file ~/.pypirc (Windows cannot, because the file cannot be named empty), or you can enter the account and password when uploading.

[distutils]
index-servers=pypi

[pypi]
repository = https://upload.pypi.org/legacy/
username = <username>
password = <password>

5. Locate the command line to this folder, and enter the command to upload the pagtest-1.0.0.tar.gzt package in the dist directory. Twine is a Python package that needs to be installed:

twine upload dist/*

Make python package as wheel file

A wheel is a package that has been compiled and does not require a compilation process during installation. Installing the whl file is faster than the published source file installation.
After step 2 above, enter the following command to generate .whl

python setup.py bdist_wheel

The .whl file is in the dist directory and uploaded to PyPI:

twine upload dist/*

Update Python packages

Modify the version number in setup.py and upload it directly