Reprinted from: Ctrip Technology Center
Deep understanding of Python decorators
About the Author
Zeng Fanwei, Senior Security Engineer of Ctrip Information Security Department, joined Ctrip in 2015 and is mainly responsible for the design and development of security automation products, including various scanners, vulnerability management platforms, and security SaaS platforms.
Python is a language that pursues elegant programming, it is easy to learn, and it is easy to write Italian-style code. This article will introduce how to use Python's advanced programming decorators to help you write more concise and readable code.
The full text is mainly divided into four parts:
The first part: early adopters, by explaining a simple example of decorators, let you have a preliminary perceptual understanding of the usage and function of decorators;
The second part: Unveiling, will introduce the use of decorators without syntactic sugar, and help you understand the essential principles of decorators;
The third part: Strike while the iron is hot, will introduce the practical usage of decorators at work, and you can directly apply the retry decorators to the project code;
Part 4: Going further, will introduce more advanced usage of decorators to help you master decorators in all aspects.
early adopters
Let's start with a simple decorator example. First define a decorator log:
def log(f):
def wrapper():
print “before”
f()
print “after”
return wrapper
Use the decorator log to decorate the greeting function and call it:
@log
def greeting():
print “Hello, World!”
greeting()
Output result:
before
Hello, World!
after
As you can see, using the decorator we have implemented the debug log before and after the function greeting.
reveal the mask
What is a decorator? From the literal meaning, we can roughly infer that its function is used for decoration. In daily life, everyone has seen many decorations, such as the colored paper decorated on the Christmas tree, or the protective case that is placed on the outside of the iPhone. The existence of the protective case does not change the internal functions of the iPhone. The significance of its existence is to enhance the anti-drop performance of the iPhone. The same is true for the decorator in Python. It does not change the internal logic of the decorated object, but allows it to gain some additional capabilities in a non-invasive way, such as logging, authorization authentication, and failure retry. etc.
Python decorators may seem inscrutable, but their implementation is actually very simple. We know that in Python, everything is an object, and a function is a special object that can be passed as a parameter to another function. The working principle of decorators is based on this feature. The default syntax for decorators is to use @ to call them, which is really just syntactic sugar. Let's see how to call the decorator without using syntactic sugar:
def greeting():
print “Hello, World!”
greeting = log(greeting)
Just pass the function greeting as a parameter to the decorator function log! For the decorator log, it takes a function as an argument, returns a new function, and finally assigns it to the greeting identifier. This results in an enhanced function with the same name as before.
strike while the iron is hot
A decorator is a programming tool that allows any decorated object to gain additional functionality with just one modification. Rolling up our sleeves, let's take a look at the concrete application of decorators in programming practice.
We know that after the program runs, some factors are often uncontrollable, such as network connectivity. For fault tolerance, we may add try-except statements to catch exceptions; considering that there is a certain probability of request failure, we may be able to use multiple retry strategies to improve the success rate. Let's simulate a non_steady function first:
import random
def non_steady():
if random.random() <= 0.5:
# The probability of failure is 0.5
raise Exception(“died”)
else:
# The probability of success is 0.5
return “survived”
The probability of this function returning successfully is 0.5. Obviously, the success rate of a single call is too low, what if it is retried 10 times? Calculate it: 1 - (0.5) ^ 10, that is, the probability of success will increase to 0.9990. Compared with 0.5 for a single call, the success rate of retry is greatly improved.
According to the above description, we first use the for loop to improve the success rate of calling non_steady:
def non_steady_with_retry(times=10):
for i in xrange(times):
try:
return non_steady()
except Exception as e:
if (i + 1) < times:
# The maximum number of retries has not yet been reached, and the exception is silently swallowed
pass
else:
# Continuously retry, if an exception still occurs when a larger number of times is reached, an exception will be thrown
raise e
The effect of improving the success rate has been achieved, but there are several problems with this implementation:
Seamless upgrades are not supported. If the function non_steady is called n times in the code, then this means that you need to modify n places at the same time (change the call to non_steady to call non_steady_with_retry);
Code reuse is not supported. If you have a second function non_steady1, you also need to upgrade the retry mechanism, which means the same retry code, you need to rewrite it again.
Try using a decorator to increase the success rate of calling non_steady. Define a retry decorator:
def retry(times=10):
def outer(f):
def inner(*args, **kwargs):
for i in xrange(times):
try:
return f(*args, **kwargs)
except Exception as e:
if (i + 1) < times:
pass
else:
raise e
return inner
return outer
Try it out:
import random
@retry(10)
def non_steady():
if random.random() <= 0.5:
# The probability of failure is 0.5
raise Exception(“died”)
else:
# The probability of success is 0.5
return “survived”
It can be seen that as long as a line of code @retry(10) is added in front of the function, the retry mechanism can be upgraded for it. Change it in one place, don't worry about it everywhere. At the same time, for other functions that want to be upgraded, only one place needs to be changed, and the same code does not need to be rewritten many times.
Go one step further
A function can apply multiple decorators at the same time. For example, the following two decorators are used to decorate the greeting function:
@log
@retry(10)
def greeting():
print “Hello, World!”
This code is equivalent to:
def greeting():
print “Hello, World!”
temp = retry(10)(greeting)
greeting = log(temp)
It can be seen that the order in which the superimposed decorators take effect is from the inside out. This requires special attention when using it.
The syntax of annotations in Java is very similar to the decorators in Python. The order of its annotations is not as strict as the decorators in Python. Be careful when using it.
In addition to functions, classes can also be used to define a decorator:
class Log(object):
def __init__(self, f):
self.f = f
def __call__(self, *args, **kwargs):
print “before”
self.f()
print “after”
Class decorators are mainly implemented through its __call__ method. Compared with function decorators, class decorators have a series of features supported by object-oriented programming, such as high cohesion, encapsulation and flexibility. Use class decorators to decorate functions:
@Log
def greeting():
print “Hello, World!”
greeting()
The output is the same as using the function decorator:
before
Hello, World!
after
In fact, any callable object in Python can be used to define a decorator.
Epilogue
Using Python decorators can make your code easier to maintain and improve readability. I believe that you have also encountered many scenes of using decorators in your daily work. Welcome to leave a message to share! Life is short, I use Python.
Article source: Ctrip Technology Center