Python Advanced Tutorial Series: Python Closures and Decorators

Today we will continue to explain closures and decorators in Python. Although we haven't learned these two concepts in detail, we already use decorators frequently in object-oriented programming. Decorators can add extra functionality to functions, as sweet as syntactic sugar. In Python, the format of a decorator is usually to add the @ symbol above the function.

Take a look at the learning objectives for this chapter:

Able to know the composition conditions of the closure
Ability to know the basic syntax for defining closures
Be able to know the keywords needed to modify the external variables used in the closure
Be able to know the basic syntax and function of defining decorators
Ability to write generic decorators
Ability to decorate a function with multiple decorators
Ability to write decorators that take parameters
Ability to know the use of class decorators

Closure

Below we will introduce the concept of closures. Before that, we need to explain how variables are stored in memory. As we mentioned earlier, variables in Python code are divided into two types: global variables and local variables. Closures are about local variables. Next, we will explain closures in detail.

How variables are stored in memory

Preface: access to global variables and local variables

Access scope of global variables:

①Global variables can be accessed normally in the global scope

②Global variables can also be accessed normally in the local scope

Access scope of local variables:

① Local variables can be accessed normally in the local scope

② Local variables cannot be accessed normally in the global scope, because after the function is executed, the internal variables will disappear. Shown in the code as follows:

The '''
scope is the scope of the variable, where it can be used and where it cannot be used.
With the emergence of functions, variables in Python can be divided into two types of scope: ① global scope ② local scope
With the emergence of scope, variables are also forced to be divided into two types: ① global variables ② local variables
① global variables : Variables defined in the global scope are global variables
② Local variables: Variables defined in the local scope are local variables
'''
# Global scope
num1 = 10 # Global variables
def func():
# Local scope
num2 = 100 # local variables

Extension: Why do the internal variables disappear after the function is executed? Answer: The garbage collection mechanism of memory

1. The role of closure

We have learned about functions before. We know that when the function is called, the variables defined in the function are destroyed, but sometimes we need to save this variable in the function, and complete a series of operations on the basis of this variable every time. For example: every time the sum is calculated with other numbers on the basis of this variable, what should I do?

We can solve this requirement through the closure we learned today.

Definition of closure:

Under the premise of function nesting, the inner function uses the variables of the outer function, and the outer function returns the address of the inner function. We call this inner function that uses the variables of the outer function a closure.

2. The conditions for the closure

Through the definition of closure, we can know the conditions for the formation of closure:

① Under the premise of function nesting (the function is defined inside the function)

②The internal function uses the variables of the external function (also includes the parameters of the external function)

③ The external function returns the internal function

3. Sample code of closure

# Define an external function
def func_out(num1):
# Define an internal function
def func_inner(num2):
# The internal function uses the variable of the external function (num1)
result = num1 + num2
print("The result is:", result)
# The external function returns the internal function, and the internal function returned here is the closure
return func_inner
# Create a closure instance
f = func_out(1)
# Execute the closure
f(2)
f(3)

4. The role of closure

Closures can save variables in external functions and will not be destroyed when the external function is called. Note: Since the closure refers to the variables of the external function, the variables of the external function are not released in time, consuming memory.

5. Summary

① When the returned internal function uses the variables of the external function, a closure is formed

②The closure can save the variables of the external function

③Standard format for realizing closure:

# External function
def test1(a):
b = 10
# Internal function
def test2():
# The internal function uses variables or parameters of the external function
print(a, b)
# Returns the internal function, the internal function returned here is a closure Example
return test2

Modify external variables used inside the closure

1. Modify the external variables used in the closure

Wrong version demo:

def outer(): # local variable num = 10
in the outer function def inner(): num = 20 print(f'original variable: {num}') # 10 inner() print(f'modified variable:{ num}') # 10 return inner f = outer()

This code defines a function outer, which contains another function inner. In the outer function, a local variable num is defined and assigned a value of 10. In the inner function, a local variable num with the same name is defined and assigned a value of 20. But due to Python's scoping rules, num in the inner function is just a new local variable of num in the outer function, and the two do not affect each other. Therefore, after calling the inner function, the num in the outer function is still 10. Finally, the outer function returns a reference to the inner function and assigns it to the variable f. In this way, the variable f can call the inner function.

2. Modify the external variables used in the closure

Demonstration of the correct version:

def outer():
# local variable num in outer function
= 10 def inner():
nonlocal num
# local variable in inner function
num = 20
print(f'original variable: {num}') # 10 inner()
print (f'modified variable: {num}') # 20
return inner
f = outer()

This code defines a function outer, which contains a local variable num with an initial value of 10. In addition, the function outer also defines an inner function inner. In the inner function, the nonlocal keyword is used to declare that the num variable is a local variable in the outer function, not a local variable of the inner function. Then, modify the value of num to 20, and output the value before and after modification. Finally, the outer function returns a reference to the inner function and assigns it to the variable f. In this way, the variable f becomes a function object pointing to the inner function.

3. Comprehensive case of closure

def outer():
result = 0
def inner(num):
nonlocal result
result += num
print(result)
return inner
f = outer()
f(1)
f(2)
f(3) # Ask about the execution result this time how many?

This code defines a function outer(), which returns the inner function inner(), and inner() can access the variable result in the outer() function.

In the function outer(), we define a variable result and initialize it to 0. Next, we define the inner function inner(), which takes a parameter num. Inside the inner() function, we use the nonlocal keyword to declare the result variable to be the variable from the outer() function and add it to num and print the value of result after each addition. Finally, we return the inner function inner().

Next, we assign the inner() function to the variable f using f = outer() when the function is called. This means that we can use the f() function in subsequent code to call the internal function we defined.

We then call the f() function multiple times, each calling the inner function and passing it different parameters. Because in the internal function, we use the non-local variable result to record the result of the previous call, so the function will print out the result of 6 when it is called for the third time. So, the output of this code is:

decorator

1. Definition of decorator

It is a function that adds additional functions to existing functions, which is essentially a closure function. Features of the decorator:

① Do not modify the source code of existing functions

② Do not modify the calling method of existing functions

③ Add additional functions to existing functions

2. Decorator code

Sample code:

# Add a login verification function
def check(fn):
def inner():
print("Please log in first...")
fn()
return inner

def comment():
print("Post a comment")

# Use decoration Decorator to decorate the function
comment = check(comment)
comment()

This code defines a decorator function check that takes a function as an argument and returns a new function inner. Before the new function inner executes the original function fn, it will print a prompt message of "Please log in first".

Next, a function comment is defined, which is used to post comments. Finally, use a decorator to decorate the function comment, that is, pass the function comment into the check function, and assign the returned new function inner to comment.

The last line of code calls the decorated function comment, which first executes the prompt information in the inner function, and then executes the function of posting comments in the original function comment. This implements a simple login verification function.

# The basic prototype of the decorator
# def decorator(fn): # fn: target function.
# def inner():
# '''Before executing the function'''
# fn() # Execute the decorated function
# '''Execute After the function'''
# return inner

This code is the basic prototype of a decorator, which defines a function called decorator, which accepts a parameter fn, which is the target function. Inside the decorator function, a function called inner is defined, which performs some operations before and after executing the target function. Finally, the decorator function returns the inner function to realize the decoration of the target function. Specifically, when we define a function, we can use the syntax of @decorator to apply the decorator function to the target function, so as to realize the decoration of the target function.

3. Syntactic sugar for decorators

If there are multiple functions that need to add the login verification function, it is still cumbersome to write code like func = check(func) to decorate the existing functions every time.
Python provides a simpler way to write decorative functions, that is, syntactic sugar. The writing format of syntactic sugar is: @decorator name, and the decoration of existing functions can also be completed through syntactic sugar.

# Add a login verification function
def check(fn):
print("The decorator function is executed")
def inner():
print("Please log in first...")
fn()
return inner

convert

# Use syntactic sugar to decorate the function
@check
def comment():
print("Post a comment")

comment()

The role of decorators

1. Use scenarios of decorators

Statistical program execution time: You can write a decorator to record the start time and end time of the function, and then calculate the execution time of the function. This decorator can be used to optimize program performance or for debugging.
Auxiliary system function output log information: You can write a decorator to record the execution process of the function and output log information. This decorator can be used for debugging, monitoring and error handling.

2. The decorator realizes the statistics of the execution time of existing functions

import time
def get_time(func):
def inner():
# Start timing
begin = time.time()
func()
# End timing
end = time.time()
print("Function execution costs %f" % (end - begin ))

return inner

This code defines a decorator function get_time that takes a function as an argument. A nested function inner is defined inside the decorator function, which is used to wrap the incoming function func. The inner function records the time before and after calling func, calculates the execution time of the function, and finally outputs the time spent in function execution.

The decorator function get_time returns the inner function, that is, returns a new function. The function of this new function is to record the time before and after the execution of the original function and output the execution time. Use the decorator @get_time to apply this decorator to the function that needs to be timed, so as to easily obtain the time of function execution. converted to

@get_time
def demo():
for i in range(100000):
print(i)

demo()

This code defines a decorator function @get_time and applies it to the function demo(). The function of the decorator function @get_time is to record the current time before and after the execution of the function demo(), calculate the time difference twice, and finally output the time difference.

The function demo() contains a loop, the number of loops is 100000, and each loop outputs a value. When the demo() function is called, it will execute the loop and output the value, and it will also trigger the execution of the decorator function @get_time, thereby outputting the execution time of the function.

Use of generic decorators

1. Use scenarios of decorators

Statistical program execution time: You can write a decorator to record the start time and end time of the function, and then calculate the execution time of the function. This decorator can be used to optimize program performance or for debugging.
Auxiliary system function output log information: You can write a decorator to record the execution process of the function and output log information. This decorator can be used for debugging, monitoring and error handling.

2. The decorator realizes the statistics of the execution time of existing functions

import time
def get_time(func):
def inner():
# Start timing
begin = time.time()
func()
# End timing
end = time.time()
print("Function execution costs %f" % (end - begin ))

return inner

@get_time
def demo():
for i in range(100000):
print(i)

demo()

Use of generic decorators

1. Decorate functions with parameters

def logging(fn):
def inner(num1, num2):
print('-- trying to calculate--')
fn(num1, num2)
return inner

@logging
def sum_num(num1, num2):
result = num1 + num2
print (result)

sum_num(10, 20)

This code defines a decorator function record that takes a function fn as an argument. logging returns an internal function inner, which accepts two parameters num1 and num2, and prints a message before calling fn. Finally, logging returns the inner function.

@logging is a decorator syntax that can apply the decorator function record to the following function sum_num. This means that sum_num will be redefined as the inner function, that is, calling sum_num is actually calling the inner function.

Inside the sum_num function, it calculates the sum of num1 and num2 and prints the result. Since sum_num is modified by the logging decorator, a message will be printed before sum_num is called. Finally, calling sum_num(10, 20) will output the following:

--trying to calculate--

2. Decorate functions with return values

def logging(fn):
def inner(num1, num2):
print('--trying to calculate--')
result = fn(num1, num2)
return result
return inner

@logging
def sum_num(num1, num2):
result = num1 + num2
return result

print(sum_num(10, 20))

This code defines a decorator function record that takes a function as an argument and returns a new function inner. The inner function will print a message "--working hard to calculate--" before executing the original function, then execute the original function, and finally return the return value of the original function.

Next, use the @logging decorator to decorate the function sum_num, which is equivalent to passing the sum_num function to the logging function and reassigning the returned new function to sum_num. In this way, when the sum_num function is called, the inner function is actually executed.

Finally, the sum_num function is called and the parameters 10 and 20 are passed in, and the output result is 30. At the same time, the message "-- trying to calculate--" will also be printed.

3. Decorate functions with variable length parameters

def logging(fn):
def inner(*args, **kwargs):
print('--trying to compute--')
fn(*args, **kwargs)
return inner

This code defines a decorator function record that takes a function fn as an argument and returns a function inner.

The function inner takes any number of positional and keyword arguments and prints a message "-- Trying to evaluate --" before executing the fn function. Then, the function inner calls the fn function and passes in the same parameters.

This decorator function can be used to add log information to other functions, so as to print some useful information when the function is executed, such as function start execution, execution end, etc.

@logging
def sum_num(*args, **kwargs):
result = 0
for i in args:
result += i

for i in kwargs.values():
result += i
print(result)

sum_num(10, 20, a=30)

This code defines a function sum_num() that takes as input any number of positional and keyword arguments and adds them up. Inside the function, all positional parameters are first added, then the values of all keyword parameters are added, and finally the sum of the two is printed out.

The decorator @logging in the code indicates that a function named logging will be called when the function is executed (this function is not given), which may record the execution of the function (such as start time, end time, parameter values, etc.) . In this way, we can record and log output when calling the function, which is convenient for us to debug and troubleshoot.

Finally, we make a call to the function sum_num(10, 20, a=30) which outputs 60. This is because 10 and 20 are positional parameters that add up to 30, and a=30 is a keyword parameter with a value of 30, so the sum is 60.

operation result:

4. Universal decorator

# Add the function of outputting logs
def logging(fn):
def inner(*args, **kwargs):
print("--trying to calculate--")
result = fn(*args, **kwargs)
return result

return inner

This code defines a decorator function record that takes a function fn as an argument and returns a new function inner.

The new function inner accepts any number of positional and keyword arguments and outputs a log indicating that evaluation is in progress before the function executes. Then call the original function fn and return its result.

This decorator function can be applied to other functions to add the ability to output logs so that a log is output each time the function executes. For example:

@logging
def add(a, b):
return a + b

When add(2, 3) is called, it prints "--trying to compute--" and then returns 5.

# Use syntactic sugar to decorate the function
@logging
def sum_num(*args, **kwargs):
result = 0
for value in args:
result += value

for value in kwargs.values():
result += value

return result

@logging
def subtraction (a, b):
result = a - b
print(result)

This code defines two functions, which are decorated with the decorator @logging.

The decorator @logging is a function that is used to decorate other functions. Its function is to print some log information before and after the execution of the decorated function, so as to facilitate debugging and tracking code execution.

The function sum_num(args, *kwargs) is a variadic function that calculates the sum of all arguments passed in. Its argument list includes a variable positional parameter args and a variable keyword parameter kwargs. The function body uses a loop to traverse args and kwargs.values(), adds their values, and returns the result.

The function subtraction(a, b) is a simple function that calculates the difference between a and b and prints the result. It has only one line of code in its function body.

Because these two functions are decorated with the decorator @logging, log information will be printed before and after their execution. These log information includes function name, parameter list and return value, etc. This makes it easy to trace function execution and debug code.

result = sum_num(1, 2, a=10)
print(result)

subtraction(4, 2)

This code defines two functions sum_num() and subtraction() and calls them. Here is a detailed explanation of these codes:

Part 1: Defining the Function

1. sum_num(1, 2, a=10): defines a function named sum_num() that accepts two positional arguments and a keyword argument named a. What the function does is return the sum of the two positional arguments plus the value of a.

2. subtraction(4, 2): defines a function called subtraction(), which receives two position parameters and returns the difference between the two parameters.

Part Two: Calling the Function

1. result = sum_num(1, 2, a=10): Call the sum_num() function and pass it the parameters 1 and 2 as positional parameters and 10 as the value of the keyword parameter a. After the function is executed, return 1 + 2 + 10 = 13, and then assign this result to the result variable.

2. print(result): Print the value of the result variable, which is 13.

3. subtraction(4, 2): Call the subtraction() function and pass the parameters 4 and 2 to it as positional parameters. After the function is executed, 4 - 2 = 2 is returned, but this result is not stored in any variable, nor printed.

Introduction to decorators with parameters

1. Introduction to decorators with parameters

A decorator with parameters means that specified parameters can be passed in when the decorator is used to decorate the function. The syntax format is: @decorator(parameter,...)

error demo

def decorator(fn, flag):
def inner(num1, num2):
if flag == "+":
print("--working addition calculation--")
elif flag == "-":
print("-- Working on subtraction calculation--")
result = fn(num1, num2)
return result
return inner

This is a decorator function that takes two arguments: a function and a flag. Returns an inner function inner.

The inner function outputs different prompt information according to different flags. Then call the incoming function fn, pass in two parameters num1 and num2, and return the result. Finally the result is returned to the caller.

The function of the decorator function is to add additional functions to the function without changing the code of the original function. In this example, the decorator function can add hints to the addition and subtraction functions to make the program more friendly.

@decorator('+')
def add(a, b):
result = a + b
return result

result = add(1, 3)
print(result)

This is a decorator function. Decorator functions can add functionality or modify the behavior of a function without changing the source code. Here, the decorator function adds two numbers with the symbol '+' and returns the result of the calculation.

This decorator function is applied to the add(a, b) function, which adds the two arguments and returns the result. Therefore, when the add(a, b) function is called, the logic in the decorator function is actually executed. This function adds two arguments and returns the result of the calculation.

Finally, the add(1, 3) function is instantiated and the result (4) is printed. Results of the:

Traceback (most recent call last):
File "/home/python/Desktop/test/hho.py", line 12, in <module>
@decorator('+')
TypeError: decorator() missing 1 required positional argument: 'flag'

2. Correct grammar

Wrap a function outside the decorator, let the outermost function receive parameters, and return the decorator, because the @ symbol must be followed by a decorator instance.

# Add the function of outputting logs
def logging(flag):

def decorator(fn):
def inner(num1, num2):
if flag == "+":
print("--working addition calculation--")
elif flag = = "-":
print("--working on subtraction calculation--")
result = fn(num1, num2)
return result
return inner

# return decorator
return decorator

This code defines a decorator function logging that takes a string parameter flag and returns a decorator function decorator.

The decorator function takes a function fn as an argument, and it defines an inner function inner that takes two arguments, num1 and num2. In the inner function, output the corresponding log information according to the value of the flag passed in, and then call the original function fn and return its result.

Finally, the decorator function returns the inner function as a decorator, which is applied to the decorated function. In this way, when the decorated function is called, the corresponding log information will be output first, and then the original function will be executed.

# Use the decorator to decorate the function
@logging("+")
def add(a, b):
result = a + b
return result

@logging("-")
def sub(a, b):
result = a - b
return result

result = add(1, 2)
print(result)
result = sub(1, 2)
print(result)

This code defines two functions add and sub, which implement addition and subtraction respectively. At the same time, use the decorator @logging to decorate these two functions. The decorator function logging will print the log information before and after the execution of the function, where "+" and "-" are used as the prefix of the log information. Finally, call the add and sub functions respectively, and output their return results.

The class decorator uses

1. Introduction to class decorators

Another special use of decorators is class decorators, which is to decorate functions by defining a class.

class Check(object):
def __init__(self, fn):
# The initialization operation is completed here
self.__fn = fn

# Implement the __call__ method, which means calling the class like calling a function.
def __call__(self, *args, **kwargs):
# Add decoration function
print("Please log in first...")
self.__fn()

This code defines a decorator class Check, which is used to perform some operations before the decorated function is executed, such as checking whether the user is logged in.

In the initialization method init of the class, save the decorated function fn in the instance attribute fn. A special method call is defined in the class, which will be executed when the class instance is called like a function. In the call method, first output a prompt message, and then call the function stored in the fn attribute.

When we use this decorator to decorate a function, we actually create an instance of the Check class, and pass the decorated function as a parameter into the initialization method init of the instance, and then use the instance as a new definition of the function . Therefore, when we call the decorated function, we actually call the instance of the Check class, and then call the call method, thus realizing the function of the decorator.

@Check
def comment():
print("Comment")

comment()

2. Code Description

@Check is equivalent to comment = Check(comment), so you need to provide an init method and add an additional fn parameter.

If you want the instance object of the class to be called like a function, you need to use the call method in the class to turn the instance of the class into a callable object (callable), that is to say, it can be called like a function.

Decorate the fn function in the call method to add additional functions.