Python3 package import rules

For reprint, please indicate the source.
win10+Python 3.6.3

Once the multi-layer file structure is used, it is easy to encounter import pits! Haha.

1. Understand some basic concepts

1. Module, package
**Module module: ** Generally, it is a file with a suffix of .py . Other file types that can be used as modules include ".pyo", ".pyc", ".pyd", ".so", and ".dll", but they are rarely used by Python beginners.
Module can be regarded as a tool class, which can share or hide code details. Place related code in a module to make the code easier to use and understand, and let the coder focus on high-level logic.
Module can define functions, classes, variables, and can also contain executable code. There are 3 types of module sources:
①Python built-in modules (standard library); ②third-
party modules; ③custom
modules.

Package: In order to avoid module name conflicts, Python introduces a method of organizing modules according to directories, called packages. A package is a folder containing Python modules.
Write picture description here
When there is a next folder the init .py , the folder is intended for a package (package), a plurality of modules (Module1) which form a whole under, and these modules (Module1) available through the same package (package ) Import other codes.

Wherein    the init    .py file for tissue package (Package), facilitate the management of reference between the various modules, controls the behavior of introduced package.
The file can not write anything, that is, it is an empty file (when it is empty, nothing can be done with import [the package]), and it can exist, which is equivalent to a mark.
But to use this form from import * pacakge_1 wording, in need   init   .py the add:    All    = [ 'FILE_A', 'FILE_B'] and there file_a.py lower file_b.py # package_1, when introduced    init    The .py file will be executed.
But it is not recommended in    the init    .py write modules to ensure that the file is simple. But can    the init    .py importing modules we need, in order to avoid a inductions, easy to use.

Among them, all is an important variable used to specify which modules (modules) will be imported into the [current scope] when this package is imported *. Modules not in the all list will not be referenced by other programs. You can rewrite all , such as all = ['current package module 1 name','module 1 name'], if this is written, it will be imported according to the module name in the list.

In fuzzy import, the form is like from package import *, * is defined by __all__.

Import accurately, such as from package import *, import package.class.

Path is also a common variable, a list, and by default there is only one element, the path of the current package. Modifying the path can change the search path in the package.

When we introduced a package (Package) (will be loaded the init .py definition module is introduced, and then run the other code), it is actually introduced the init .py file (import, the file is automatically run, help Let's import multiple modules in the package). We can the init .py reintroduced other packages (package) or modules or custom classes.

2. sys.modules, namespace, module built-in attributes
2.1
Official explanation of sys.modules : Linking
sys.modules is a dictionary that maps module names (module_name) to loaded modules (modules). Can be used to force modules to be reloaded. As soon as Python starts, it will be loaded into memory.
When we import new modules, sys.modules will automatically record the module; when we import the module for the second time, Python will look it up in the dictionary directly to speed up the running speed.

It is a dictionary , so it has all the methods of the dictionary, such as sys.modules.keys(), sys.modules.values(), sys.modules['os']. But please do not replace the dictionary easily, or delete an element from the dictionary, it may cause Python to fail to run.

<span style="color:#000000"><code>import sys
print(sys.modules)#打印，查看该字典具体内容。
</code></span>

2.2 Namespace
Like a dict, key is the name of the variable, and value is the value of the variable.

Each function function has its own namespace, called local namespace , which records the variables of the function.
Each module has its own namespace, called global namespace , which records the variables of the module, including functions, classes, imported modules, module-level variables and constants .
The build-in namespace contains build-in functions and exceptions, which can be accessed by any module.

When a certain piece of Python code accesses the variable x, Python will look for the variable in all namespaces in the order:

local namespace is the current function or class method. If found, stop searching;
global namespace is the current module. If found, stop searching;
build-in namespace Python will assume that the variable x is a function or variable of build-in. If the variable x is not a built-in function or variable of build-in, Python will report an error NameError.
For closures, if the variable cannot be found in the local namespace, the next search target is the local namespace of the parent function.

Example: namespace_test.py code

<span style="color:#000000"><code>def func(a=1):
	b = 2
	print(locals())#打印当前函数（方法）的局部命名空间
	'''
	locs = locals()#只读，不可写。将报错！
	locs['c'] = 3
	print(c)
	'''
	return a+b
func()
glos = globals()
glos['d'] = 4
print(d)

print(globals())#打印当前模块namespace_test.py的全局命名空间
</code></span>

The built-in functions locals() and globals() return a dictionary. Difference: the former is read-only, the latter is writable.

The namespace is reflected in from module_name import and import module_name: the from keyword is a part of the imported module or package.

from module_A import X: The function/variable of the module will be imported into the namespace of the current module, no need to use module_A.X to access.
import module_A: Modules_A itself is imported, but its original namespace is saved, so it is necessary to use module_A.X to access its functions or variables.

2.3 Module built-in attributes

The name runs the module directly, the name value is main ; the import module, the name value is the module name.
file absolute path of the current module
dict
doc
package
path

3. Absolute import, relative import
Write picture description here
3.1 Absolute import : All module imports start from the "root node". The location of the root node is determined by the path in sys.path, and the root directory of the project is usually automatically in sys.path. If you want the program to be executed everywhere, you need to manually modify sys.path.
Example 1: Import B package/B1 sub-package/b1.py module in c.py

<span style="color:#000000"><code>import sys,os
BASE_DIR = os.path.dirname(os.path.abspath(__file__))#存放c.py所在的绝对路径

sys.path.append(BASE_DIR)

from B.B1 import b1#导入B包中子包B1中的模块b1
</code></span>

Example 2: Import b2.py module in b1.py

<span style="color:#000000"><code>from B.B1 import b2#从B包中的子包B1中导入模块b2
</code></span>

3.2 Relative Import : Just care about the module position relative to your current directory. Cannot be executed directly inside the package (error will be reported). No matter where the root node is, the relative position of the modules in the package is correct.
b1.py code

<span style="color:#000000"><code>#from . import b2 #这种导入方式会报错。
import b2#正确
b2.print_b2()
</code></span>

b2.py code

<span style="color:#000000"><code>def print_b2():
	print('b2')
</code></span>

Run b1.py and print: b2.

When using relative import, you may encounter ValueError: Attempted relative import beyond toplevel package
Solution: Refer to this article, link .

3.3 Import a package separately: When importing a package name separately , all the submodules contained in the package will not be imported.
c.py imports the b2 module of the sub-package B1 package of the same level directory B package, and executes the print_b2() method of the b2 module:
c.py code

<span style="color:#000000"><code>import B
B.B1.b2.print_b2()
</code></span>

Run c.py, an error will be reported.

Solution :
B/ init .py code

<span style="color:#000000"><code>from . import B1#其中的.表示当前目录
</code></span>

B/B1/ init .py code

<span style="color:#000000"><code>from . import b2
</code></span>

At this point, execute c.py and successfully print: b2.

3.4 Extra
① One .py file calls the class in another .py file.
For example, a.py (class A), b.py (class B), a.py calls class B in b.py with: from b import B
②The class in one .py file inherits the class in another .py file . For example, a.py (class A), b.py (class B), class A in a.py inherits class B in b.py.

<span style="color:#000000"><code>from b import B
class A(B):
pass
</code></span>

2. Python operating mechanism: understand what Python does when it executes the import statement (importing built-in (Python's own) or third-party module (already in sys.path))?

step1: create a new, empty module object (it may contain multiple modules);
step2: insert the module object into sys.modules;
step3: load the code of the module (if necessary, compile first);
step4: execute The corresponding code in the new module.

When executing step3, first find the location of the module program. If the imported module name is mod_1, the interpreter must find the mod_1.py file. The search order is:
current path (or sys.path specified in the current directory) --- -->PYTHONPATH----->Default path related to Python installation settings.

For those that are not in sys.path, you must avoid importing submodules of custom packages with import, and use absolute or relative import from...import..., and relative import of packages can only Use the from form.

1. "Standard" import, top import
Write picture description here
With the above basic knowledge, it is easy to understand this mind map. When using the variables or functions of the module, it will be handy.

2. Nested import

2.1 Order import-import
Write picture description here

PS: The Local namespace of each module is independent . That is: after the
test module is imported into moduleA, only moduleA can be accessed, not moduleB. Although moduleB has been loaded into memory, to access it, moduleB must be explicitly imported in the test module. Actually print locals(), only moduleA in the dictionary, not moduleB.

2.2 Loop import/nested import-import
Write picture description here
is like from moduleB import ClassB statement, according to Python's internal import mechanism , perform subdivision steps:

Find the symbol "moduleB" in sys.modules;
If the symbol "moduleB" exists, the module object corresponding to the symbol "moduleB" is obtained; the object corresponding to the symbol "ClassB" is obtained
from the dict__. If "ClassB" does not exist, throw an exception "ImportError: cannot import name'classB'"
If the symbol "moduleB" does not exist, create a new module object. However, the dict of the new module object is empty at this time . Then execute the statements in the moduleB.py file to fill in the dict .

Summary: from moduleB import ClassB has two processes , first from module, then import ClassB.
Write picture description here

Of course, change the moduleA.py statement from moduleB import ClassB to: import moduleB, and an error will be reported when the moduleB.py statement from moduleA import ClassA is executed the second time: ImportError: cannot import name'classA'

To solve this circular import method:
Example: When installing a wireless network card, you need to download the network card driver
from the Internet; wheninstalling a compression software, the compressed software installation program downloaded from the Internet is a compressed file.
Method 1 ----->Lazy import: Write the import statement in the method/function, and limit its scope to the local. (This method may cause performance problems)
Method 2 ----->Changefrom x import y to import xy form
Method 3 ----->Organize the code (refactor code): Change the code layout, which can be combined or separated Competing for resources.
Combine----->write all into one .py file;
separate-->extract the resources that need to be imported into a third-party .py file.
In short, turn the loop into one-way.

3, bag (package) import
in a file while the init .py documents, files, and other modules, i.e., the folder as a bag (package). Import and import module packages are basically the same, just import the package when, will execute this the init .py, rather than the module statement.
Further, if the ** simply introduced as package [Form: import xxx] **, and package the init .py have not explicitly initializing operation of the other, is: in this package module will not be automatically introduced . Of course, the package will be imported successfully, and the package name will be placed in the Local namespace of the current .py.
Write picture description here
[D: youcaihua\test\PkgDemo\mod.py] file
[D: youcaihua\test\PkgDemo\pkg1\pkg1_mod.py] file
[D: youcaihua\test\PkgDemo\pkg2\pkg2_mod.py] file, three files The same code:

<span style="color:#000000"><code>def getName():
	print(__name__)

if __name__ == '__main__':
	getName()
</code></span>

[D: youcaihua\test\test.py] file

<span style="color:#000000"><code>import PkgDemo.mod#1
print(locals(),'\n')
import PkgDemo.pkg1#2
print(locals(),'\n')
import PkgDemo.pkg1.pkg1_mod as m1#3
print(locals(),'\n')
import PkgDemo.pkg2.pkg2_mod#4
PkgDemo.mod.getName()#5
print('调用mod.py----', locals(), '\n')
m1.getName()#6
PkgDemo.pkg2.pkg2_mod.getName()#7
</code></span>

After executing #1 , add PkgDemo and PkgDemo.mod to sys.modules. At this time, you can call any class or function of PkgDemo.mod. When you cannot call any modules in the package PkgDemo.pkg1 or pkg2. But currently only PkgDemo exists in the Local namespace of the test.py file .

After executing #2 , just load PkgDemo.pkg1 into memory, and sys.modules will have three modules: PkgDemo, PkgDemo.mod, and PkgDemo.pkg1. However, any module under PkgDemo.pkg1 are not automatically loaded into memory, so at this point: PkgDemo.pkg1.pkg1_mod.getName () will be wrong. Currently, the Local namespace of test.py is still only PkgDemo.

After #3 is executed , pkg1_mod will be loaded into the memory, and sys.modules will have four modules: PkgDemo, PkgDemo.mod, PkgDemo.pkg1, PkgDemo.pkg1.pkg1_mod, and then PkgDemo.pkg1.pkg1_mod.getName() can be executed. Due to the use of as , the current Local namespace will additionally add m1 (as an alias of PkgDemo.pkg1.pkg1_mod), and of course PkgDemo.

After executing #4 , PkgDemo.pkg2, PkgDemo.pkg2.pkg2_mod will be loaded into the memory, and there will be PkgDemo, PkgDemo.mod, PkgDemo.pkg1, PkgDemo.pkg1.pkg1_mod, PkgDemo.pkg2, PkgDemo.pkg2 in sys.modules. There are six modules of pkg2_mod, of course: the current Local namespace is still only PkgDemo, m1.

#5 , #6 , #7 can of course be executed correctly.

3. How to avoid Python circle import error? How to avoid Python's circular import problem?

Code layout, (architecture) design problems, the solution is: turn the cycle into one-way. Adopt hierarchical, time-consuming import, relative import (it is recommended not to exceed two levels)

Note: If you execute Python xx.py on the command line and in the IDE, the results may be different.

For reprint, please indicate the source.
Reference:
official specification