Python3 Quick Start (XVIII) - PyInstaller packaged and released

Python3 Quick Start (XVIII) - PyInstaller packaged and released

A, PyInstaller Profile

1, PyInstaller Profile

PyInstaller is a cross-platform Python application packaging tools, support Windows / Linux / MacOS three major platforms, able to Python scripts and Python interpreter where packaged into an executable file, allowing end users without the need to install Python in execution of the application.
PyInstaller produced the executable file is not cross-platform, if you need packaged as different platforms, it is necessary to run PyInstaller packaged on the corresponding platform.

2, PyInstaller installation

pip install PyInstaller

Two, PyInstaller basic usage

1, PyInstaller use

pyinstaller main.py
PyInstaller easiest to use only need to specify the script file as a program entry. After performing Packager PyInstaller will be created in the current directory the following files and directories:
main.spec file with the same name prefix and scripts, specify various parameters required for packaging;
Build subdirectory, which generated during the storage of packaged Temporary Files. warnxxxx.txt file records generated in the process of warning / error messages. If you have a problem PyInstaller run, you need to check warnxxxx.txt file to get the details wrong. xref-xxxx.html script file output obtained PyInstaller module dependency graph analysis.
dist subdirectory to store the final document generated. If a single file mode only a single executable file; If you use the directory mode then there will be a subdirectory and script of the same name, which is the real inside executable files and attached documents.

2, PyInstaller command-line options

PyInstaller command-line options can be viewed by helping information:
pyinstaller --help
-y | --noconfirm: direct coverage of the output file without prompting, when repeatedly run the command to avoid repeatedly confirmed.
-D | --onedir: build directory contains the executable file (the default behavior).
-F | --onefile: generate a single executable file, not recommended.
-i | --icon [.ico | .exe | .icns]: Specifies the icon for the executable file Windows / Mac platforms.
--version-file [filename]: Add file version information.
-c | --console | --nowindowed: the console window to run the program and allocate the standard input / output (default behavior).
-w | --windowed | --noconsole: Do not create a console window, do not allocate the standard input / output, mainly used to run GUI programs. Do not enter debug output will bring some difficulties, so even a GUI program is recommended to disable this option when debugging, and then open the final release.
--add-data [file:dir]: Adding data files. If there is more than one file needs to be added, this option can appear multiple times. Format parameter is the output directory name + file name, divided by the path separator used in the Windows;, using other systems: If the output to the same directory as the script is used as the output directory.
--add-binary [file:dir]: Adding binary files that are required to run the program .exe / .dll / .so and so on.

3, single directory mode

PyInstaller single directory mode is the Python compile a plurality of files in the same directory, which is the entry point xxxx.exe (xxxx is the name of the script file can be modified by the command line). Single mode is the default directory PyInstaller may add their own or -D --onedir explicit switch is turned on.
Single directory schema package generated in addition to the executable file directory, further comprising the Python interpreter (PythonXX.dll), the runtime system (ucrtbase.dll and other apixx.dll), as well as some compiled Python module (.pyd files) .

4, single-file mode

Single-file mode is the entire program is compiled into a single executable file. At the command line or add --onefile -F switch on.
Python scripts are interpreted program, rather than the native compiler, and can not produce a true single executable file. If a single file pattern, PyInstaller package generated automatically extracting program, need to first extract all files into a temporary directory (usually called _MEIxxxx, XXXX is a random number), and then the Subsidiary loading the interpreter file from the temporary directory. After the program is complete, if all goes well, it will then delete the temporary directory.
Python interpreter will modify PyInstaller runtime. If the direct run Python scripts, sys.frozen variable does not exist, if by PyInstaller generated executable file to run, PyInstaller sys.frozen variable is set to True; if you use single file mode, sys._MEIPASS variable contains the name of the temporary directory created automatically PyInstaller .
Single-file mode because of the temporary directory and extract the files process, so the program startup speed will be slower. If the program runs half collapse, the temporary directory will be no chance of being deleted.

Three, PyInstaller specification file

PyInstaller creates a corresponding .spec file while generating a file, the file is essentially a special .spec Python script, record the required instruction is generated.

1, Spec file generation

When using pyinstaller [options] xxx.py packaged, PyInstaller according .spec will first generate a corresponding file option, then performing the specified file .spec process of generating a final document. Therefore, you can specify the spec file directly performing a packing process.
pyinstaller [options] xxx.spec

2, Spec file format

Single pattern generated spec file directory format is as follows:

a = Analysis(...)
pyz = PYZ(...)
exe = EXE(...)
coll = COLLECT(...)

Single-file pattern generated spec file format is as follows:

a = Analysis(...)
pyz = PYZ(...)
exe = EXE(...)

Single-file mode is to unify all of the content packaged into .exe, and single directory mode in addition to the .exe, but also need to copy other ancillary documents.
Analysis for analyzing the relationship between a reference script, and to find all relevant content recorded in the internal structure, using the steps that follow;
; PYZ all modules compiled Python script and package into a corresponding .pyd
Python packaged after: EXE and other documents with the generating module executable file structure;
COLLECT: subsidiary references to files are copied to the corresponding position of the generated directory.
If the data file cause a lot Analysis is too long, it can be extracted as a separate variable.

data_files = [(...)]
a = Analysis(...,
             datas=data_files,
             ...)

Wildcards can be specified as a data / binary file, so as to match a plurality of files of the same type.

a = Analysis(...,
             datas=[('media/*.mp3', 'media')],
             ...)

Specified files and directories can be specified package to package, as follows:

a = Analysis(...,
             datas=[('config.ini', '.'), ('data', 'data')],
             ...)

When the config.ini file is packaged executable files in the current directory, the directory data package to the next executable files in the current directory.

Four, PyInstaller Hook mechanism

1, PyInstaller Hook Profile

PyInstaller using a recursive method, analyzed one by one from a script file entry, you can acquire module.
PyInstaller recognition module can call ctypes, SWIG, Cython other forms, but the file name must be literal. But PyInstaller not recognize the dynamic and calls, such as import, exec, eval, as well as variable as a parameter called.
After completion PyInstaller identify all modules will be configured in a tree structure showing the internal call graph, call relationship will also be output (xref-xxxx.html file) when generating object. PYZ step will all identified modules pooled, if necessary, be translated into .pyd, then packaged file. But there are still the following problems:
(1) Due to the dynamic module calls may not be able to automatically identify, and therefore will not be packaged into a file, there will be when asked to perform.
(2) Some modules are not in the form of modules, but through the file system to access .py file, the code is the same problem occurs at runtime.
In order to solve the above problem, PyInstaller introduced Hooks mechanism for the two issues into two types of Hook. Hook mainly two kinds of distinction in accordance with load times, the first one is not explicitly named Hook in PyInstaller document is called when the generation process, introducing a specific module called Import Hook; the second is Runtime Hook, in during the execution file at startup, load a particular module call.

2、Import Hooks

All Hook PyInstaller defined in PyInstaller installation directory hooks subdirectory named files are hook- [module name] .py form, namely Import Hook.
When the generation process PyInstaller find specific import module, it will look to the Hook hooks directory corresponding to the presence or absence, and if present, it is executed.
hook-PyQt5.py file as follows:

import os

from PyInstaller.utils.hooks import collect_system_data_files
from PyInstaller.utils.hooks.qt import pyqt5_library_info, get_qt_binaries

# Ensure PyQt5 is importable before adding info depending on it.
if pyqt5_library_info.version:
    hiddenimports = [
        # PyQt5.10 and earlier uses sip in an separate package;
        'sip',
        # PyQt5.11 and later provides SIP in a private package. Support both.
        'PyQt5.sip'
    ]

    # Collect the ``qt.conf`` file.
    datas = [x for x in
             collect_system_data_files(pyqt5_library_info.location['PrefixPath'],
                                       'PyQt5')
             if os.path.basename(x[0]) == 'qt.conf']

    # Collect required Qt binaries.
    binaries = get_qt_binaries(pyqt5_library_info)

hiddenimports PyInstaller is used to describe not expressly introduced through the import, the mechanism but by the dynamic loading of other modules.

3、Runtime Hooks

Runtime Hooks are located PyInstaller installation loader \ rthooks subdirectories, and naming is pyi_rth_[模块名称].py(rth on behalf of run time hook).
loader \ rthooks.dat content is a dictionary, the system records all supported Runtime Hooks. rthooks.dat file as follows:

{
    'certifi':    ['pyi_rth_certifi.py'],
    'django':     ['pyi_rth_django.py'],
    'enchant':    ['pyi_rth_enchant.py'],
    'gi':         ['pyi_rth_gi.py'],
    'gi.repository.Gio':    ['pyi_rth_gio.py'],
    'gi.repository.GLib':   ['pyi_rth_glib.py'],
    'gi.repository.GdkPixbuf':    ['pyi_rth_gdkpixbuf.py'],
    'gi.repository.Gtk':    ['pyi_rth_gtk.py'],
    'gi.repository.Gst':    ['pyi_rth_gstreamer.py'],
    'gst':        ['pyi_rth_gstreamer.py'],
    'kivy':       ['pyi_rth_kivy.py'],
    'kivy.lib.gstplayer': ['pyi_rth_gstreamer.py'],
    'matplotlib': ['pyi_rth_mplconfig.py', 'pyi_rth_mpldata.py'],
    'osgeo':      ['pyi_rth_osgeo.py'],
    'pkg_resources':  ['pyi_rth_pkgres.py'],
    'PyQt4':      ['pyi_rth_qt4plugins.py'],
    'PyQt5':      ['pyi_rth_pyqt5.py'],
    'PyQt5.QtWebEngineWidgets': ['pyi_rth_pyqt5webengine.py'],
    'PySide':      ['pyi_rth_qt4plugins.py'],
    'PySide2':      ['pyi_rth_pyside2.py'],
    'PySide2.QtWebEngineWidgets': ['pyi_rth_pyside2webengine.py'],
    '_tkinter':    ['pyi_rth__tkinter.py'],
    'traitlets':  ['pyi_rth_traitlets.py'],
    'twisted.internet.reactor':        ['pyi_rth_twisted.py'],
    'usb':        ['pyi_rth_usb.py'],
    'win32com':   ['pyi_rth_win32comgenpy.py'],
    'multiprocessing': ['pyi_rth_multiprocessing.py'],
    'nltk': ['pyi_rth_nltk.py'],
}

Runtime Hooks are performed during the execution file is run. PyInstaller modified module loading mechanism, when any module is loaded during operation, checks for PyInstaller corresponding Runtime Hook, if so, the appropriate operation Hook. Therefore, Runtime Hooks and scripts are compiled into an executable file together.
pyi_rth_pyqt5.py file as follows:

import os
import sys

# The path to Qt's components may not default to the wheel layout for
# self-compiled PyQt5 installations. Mandate the wheel layout. See
# ``utils/hooks/qt.py`` for more details.
pyqt_path = os.path.join(sys._MEIPASS, 'PyQt5', 'Qt')
os.environ['QT_PLUGIN_PATH'] = os.path.join(pyqt_path, 'plugins')
os.environ['QML2_IMPORT_PATH'] = os.path.join(pyqt_path, 'qml')

Fifth, error debugging

When using PyInstaller packaged, the most common error is Failed to execute script xxx, common practice is to use pyinstaller -c xxx.py the application packaged as a console application, the corresponding command line executable program to view the error output, and then one by one exclusion errors.