Decorator function (function decorator) can function "labeled", function to provide more features.
Before appreciated decorator to be understood that the closure (closure) . Python3.0 the introduction of reserved keywords nonlocal , use a closure is also inseparable from nonlocal. Incidentally, closures used on decorators for asynchronous programming is also very important concept in addition.
Decorator (Decorator) is called a decorative function, which receives a function as argument.
Suppose have defined a decorator the decorate (actually the decorate function and a receiving function returns a function), then the following two codes are equivalent.
@decorate def target(): print('running target()')
with
def target(): print('running target()') target = decorate(target)
It can be seen @ mark this syntax is actually a syntax sugar. After the label has become the target of another function decorate (target).
Let us look at an example of a practical definition of decorate:
def decorator(func): def inner(): print('running inner()') return inner @decorator def target1(): print('running target1()') def target2(): print('running target2()') inner_func1 = target1() print(inner_func1) print('-' * 10) print(target1) print('*' * 10) inner_func2 = decorator(target2)() print(inner_func2) print('-' * 10) print(decorator(target2))
Output:
running inner() None ---------- <function decorator.<locals>.inner at 0x10ae3f598> ********** running inner() None ---------- <function decorator.<locals>.inner at 0x10aee7510>
Further validate our understanding based on the code and results. By attaching the decorator function, a function can be changed to another function. As for how the conversion is based decorator function itself defined. Input and output function is a function of decorator, which defines the transformation functions .
The execution order of the decorator
When A is defined a function, if it is marked with the decorative B, then A definition when the code has been performed decorator B, rather than A when calling the function performed. which is:
def decorator(func): print('running decorator(func)') def inner(): print('running inner()') return inner @decorator def target(): print('target()') print('before calling target.') target()
Equivalent to
def decorator(func): print('running decorator(func)') def inner(): print('running inner()') return inner def target(): print('target()') target = decorator(target) print('before calling target.') target()
The results are:
running decorator(func)
before calling target.
running inner()
For this reason, decorator often introduced when the module executes (import time), and the function to be decorated (decorator return function) is executed when an explicit call (Runtime) .
It does not change the original function decorator
Most decorators often will change the original function, but there are some scenarios will not change the original function. E.g:
registry = [] def register(func): registry.append(func) return func
This decoration will collect used its functions.
Variable scope rules
The following code because the variable b is not defined and given:
def f1(a): print(a) print(b) f1(3)
3 --------------------------------------------------------------------------- NameError Traceback (most recent call last) <ipython-input-28-25d665eb58d1> in <module> 3 print(b) 4 ----> 5 f1(3) <ipython-input-28-25d665eb58d1> in f1(a) 1 def f1(a): 2 print(a) ----> 3 print(b) 4 5 f1(3) NameError: name 'b' is not defined
And because of the following code is a global variable b without error.
6 b =
f1 ( 3)
3 6
Next is the key, the following error code:
b = 6 def f2(a): print(a) print(b) b = 9 f2(3)
3 --------------------------------------------------------------------------- UnboundLocalError Traceback (most recent call last) <ipython-input-30-55c0dd1a1ffb> in <module> 5 b = 9 6 ----> 7 f2(3) <ipython-input-30-55c0dd1a1ffb> in f2(a) 2 def f2(a): 3 print(a) ----> 4 print(b) 5 b = 9 6 UnboundLocalError: local variable 'b' referenced before assignment
This is because when compiling Python function, was found b assignment within a function, so it is considered a local variable b. So the generated bytecode think that b is a local variable, it will try to find in the local environment, b, then the error.
If you need to fix this problem, we need to be explicitly declared globally:
b = 6 def f3(a): global b print(a) print(b) b = 9 f3(3)
3 6
Closure
Since often anonymous function defines a function in the function, the closure also uses nested functions, so both are often confused.
But in fact, the focus of closures is not anonymous or not. Closure is a function, which is bound non-global variable in a function outside the function .
We look at such a calculation of the average number of closures.
def make_averager(): series = [] def averager(new_value): series.append(new_value) total = sum(series) return total/len(series) return averager avg = make_averager() avg(10) avg(15) avg(20)
Output:
10.0 12.5 15.0
Series here called free variables (as Free variable) , although make_averager has been called over, but the series is still included in the closure but did not destroy, averager function can still use it.
View avg of variables:
Awg. __kod__ Ksio_varnames avg. __kod__ Ksio_farewars
Output:
('new_value', 'total')
('series',)
Therefore, the closure is a function contains free variables.
In order to facilitate the analogy, we look at an implementation class-based:
class Averager(): def __init__(self): self.series = [] def __call__(self, new_value): self.series.append(new_value) total = sum(self.series) return total/len(self.series) avg = Averager() avg(10) avg(15) avg(20)
Output:
10.0 12.5 15.0
nonlocal
A more elegant closure, avoid storing a list:
def make_averager(): count = 0 total = 0 def averager(new_value): nonlocal count, total count += 1 total += new_value return total / count return averager avg = make_averager() avg(10) avg(15) avg(20)
This uses nonlocal, it indicates that the variable is not local variables. This keyword is used to process variable scoping rules previously mentioned: When there is a function assignment, Python would think that this variable is a local variable. Clearly, we should let the count and total freedom as variables outside averager, so the need to add nonlocal keyword.
Implement a simple decorator
The following is an example of a timing decorators:
import time def clock(func): def clocked(*args, **kwargs): """ clocked doc """ t0 = time.perf_counter() result = func(*args, **kwargs) elapsed = time.perf_counter() - t0 name = func.__name__ arg_str = ', '.join(repr(arg) for arg in args) print('[%0.8fs] %s(%s) -> %r' % (elapsed, name, arg_str, result)) return result return clocked @clock def snooze(seconds): """ snooze doc """ time.sleep(seconds) @clock def factorial(n): """ factorial doc """ return 1 if n < 2 else n*factorial(n-1) print('*' * 40, 'Calling snooze(.123)') snooze(.123) print('*' * 40, 'Calling factorial(6)') print('6! =', factorial(6))
Output:
**************************************** Calling snooze(.123) [0.12775655s] snooze(0.123) -> None **************************************** Calling factorial(6) [0.00000100s] factorial(1) -> 1 [0.00006883s] factorial(2) -> 2 [0.00012012s] factorial(3) -> 6 [0.00016687s] factorial(4) -> 24 [0.00022555s] factorial(5) -> 120 [0.00030625s] factorial(6) -> 720 6! = 720
The result is self-evident, decorators original packaging function, into a new function, increase the timing information. More than decorators have some minor flaws:
snooze.__name__ snooze.__doc__ factorial.__name__ factorial.__doc__
'clocked' '\n clocked doc\n ' 'clocked' '\n clocked doc\n
You can see decorator "pollution" of the property by some decorative function.
Use functools.wrap: A more elegant approach:
import time import functools def clock(func): @functools.wraps(func) def clocked(*args, **kwargs): t0 = time.time() result = func(*args, **kwargs) elapsed = time.time() - t0 name = func.__name__ arg_lst = [] if args: arg_lst.append(', '.join(repr(arg) for arg in args)) if kwargs: pairs = ['%s=%r' % (k, w) for k, w in sorted(kwargs.items())] arg_lst.append(', '.join(pairs)) arg_str = ', '.join(arg_lst) print('[%0.8fs] %s(%s) -> %r ' % (elapsed, name, arg_str, result)) return result return clocked @clock def snooze(seconds): """ snooze doc """ time.sleep(seconds) @clock def factorial(n): """ factorial doc """ return 1 if n < 2 else n*factorial(n-1) print('*' * 40, 'Calling snooze(.123)') snooze(.123) print('*' * 40, 'Calling factorial(6)') print('6! =', factorial(6)) snooze.__name__ snooze.__doc__ factorial.__name__ factorial.__doc__
Output:
**************************************** Calling snooze(.123) [0.12614298s] snooze(0.123) -> None **************************************** Calling factorial(6) [0.00000119s] factorial(1) -> 1 [0.00012970s] factorial(2) -> 2 [0.00022101s] factorial(3) -> 6 [0.00031495s] factorial(4) -> 24 [0.00039506s] factorial(5) -> 120 [0.00047684s] factorial(6) -> 720 6! = 720 'snooze' '\n snooze doc\n ' 'factorial' '\n factorial doc\n
Standard library decorator
Python There are three built-in functions for decorating method:
- property
- classmathod
- staticmethod
Another common decoration is functools.wraps .
The standard library there are two interesting decorator (defined in functools) are:
- lru_cache
- singledispatch
functools.lru_cache
functools.lru_cache achieve the memory function. LRU representation Least Recently Logs in Used . We look at how this decorator accelerate fibonacci recursion.
Common methods:
@clock def fibonacci(n): if n < 2: return n return fibonacci(n-2) + fibonacci(n-1) print(fibonacci(6))
Output:
[0.00000000s] fibonacci(0) -> 0 [0.00000310s] fibonacci(1) -> 1 [0.00028276s] fibonacci(2) -> 1 [0.00000095s] fibonacci(1) -> 1 [0.00000000s] fibonacci(0) -> 0 [0.00000167s] fibonacci(1) -> 1 [0.00007701s] fibonacci(2) -> 1 [0.00015092s] fibonacci(3) -> 2 [0.00051212s] fibonacci(4) -> 3 [0.00000095s] fibonacci(1) -> 1 [0.00000000s] fibonacci(0) -> 0 [0.00000095s] fibonacci(1) -> 1 [0.00007415s] fibonacci(2) -> 1 [0.00014782s] fibonacci(3) -> 2 [0.00000095s] fibonacci(0) -> 0 [0.00000095s] fibonacci(1) -> 1 [0.00007510s] fibonacci(2) -> 1 [0.00000119s] fibonacci(1) -> 1 [0.00000095s] fibonacci(0) -> 0 [0.00000000s] fibonacci(1) -> 1 [0.00007606s] fibonacci(2) -> 1 [0.00015116s] fibonacci(3) -> 2 [0.00030208s] fibonacci(4) -> 3 [0.00052595s] fibonacci(5) -> 5 [0.00111508s] fibonacci(6) -> 8 8
Use lru_cache:
@functools.lru_cache() @clock def fibonacci(n): if n < 2: return n return fibonacci(n-2) + fibonacci(n-1) print(fibonacci(6))
[0.00000095s] fibonacci(0) -> 0 [0.00000191s] fibonacci(1) -> 1 [0.00041223s] fibonacci(2) -> 1 [0.00000215s] fibonacci(3) -> 2 [0.00048995s] fibonacci(4) -> 3 [0.00000215s] fibonacci(5) -> 5 [0.00056982s] fibonacci(6) -> 8 8
Generic function: singledispatch
Generic function: function performs a similar set of operations in different ways (first parameter basis).
Code feel:
from functools import singledispatch from collections import abc import numbers import html @singledispatch def htmlize(obj): content = html.escape(repr(obj)) return '<pre>{}</pre>'.format(content) @htmlize.register(str) def _(text): content = html.escape(text).replace('\n', '<br>\n') return '<p>{0}</p>'.format(content) @htmlize.register(numbers.Integral) def _(n): return '<pre>{0} (0x{0:x})</pre>'.format(n) @htmlize.register(tuple) @htmlize.register(abc.MutableSequence) def _(seq): inner = '</li>\n<li>'.join(htmlize(item) for item in seq) return '<ul>\n<li>' + inner + '</li>\n</ul>' htmlize({1, 2, 3}) htmlize(abs) htmlize('Heimlich & Co.\n- a game') htmlize(42) print(htmlize(['alpha', 66, {3, 2, 1}]))
Output:
'<pre>{1, 2, 3}</pre>'
'<pre><built-in function abs></pre>'
'<p>Heimlich & Co.<br>\n- a game</p>'
'<pre>42 (0x2a)</pre>'
<ul>
<li><p>alpha</p></li>
<li><pre>66 (0x42)</pre></li>
<li><pre>{1, 2, 3}</pre></li>
</ul>
Decorator series
As the name implies, a function can be modified plurality decorators.
@d1 @d2 def f(): print('f')
Equivalent to
def f(): print('f') f = d1(d2(f))
Containing ginseng decorator
We know that when the function is decorative decorator, in fact, is passed as an argument to the decorator. To achieve a decorator containing parameters, we need to build a factory decorative function, which receives and returns a decorator. To put it plainly, it is another layer of nested function, write a return decorator function.
Some examples are provided below as a reference.
Registrar
registry = set() def register(active=True): def decorate(func): print('running register(active=%s)->decorate(%s)' % (active, func)) if active: registry.add(func) else: registry.discard(func) return func return decorate @register(active=False) def f1(): print('running f1()') @register() def f2(): print('running f2()') def f3(): print('running f3()') registry register()(f3) registry register(active=False)(f2) registry
Output:
running register(active=False)->decorate(<function f1 at 0x10aef8510>)
running register(active=True)->decorate(<function f2 at 0x10aef8950>)
{<function __main__.f2()>}
running register(active=True)->decorate(<function f3 at 0x10ad791e0>)
<function __main__.f3()>
{<function __main__.f2()>, <function __main__.f3()>}
running register(active=False)->decorate(<function f2 at 0x10aef8950>)
<function __main__.f2()>
{<function __main__.f3()>}
Timer
import time DEFAULT_FMT = '[{elapsed:0.8f}s] {name}({args}) -> {result}' def clock(fmt=DEFAULT_FMT): def decorate(func): def clocked(*_args): t0 = time.time() _result = func(*_args) elapsed = time.time() - t0 name = func.__name__ args = ', '.join(repr(arg) for arg in _args) result = repr(_result) print(fmt.format(**locals())) return _result return clocked return decorate @clock() def snooze(seconds): time.sleep(seconds) for i in range(3): snooze(.123)
[0.12674093s] snooze(0.123) -> None [0.12725592s] snooze(0.123) -> None [0.12320995s] snooze(0.123) -> None
UDP client / server
""" UDP client/server decorator UDP client: to send data. UDP server: to perform operation on the frame it receives. """ import functools import socket import json import time def process_udp_server(ip='0.0.0.0', port=8999, data_size=1024 * 10): """ UDP server decorator :param ip: :param port: :param data_size: :return: """ server = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) server.bind((ip, port)) print(f'UDP server started at {str(ip) + ":" + str(port)}.') def start_server(func): @functools.wraps(func) def processed(*args, **kwargs): while True: data = server.recv(data_size) data = json.loads(data.decode()) res = func(data, *args, **kwargs) if res == -1: break return processed return start_server def camera_udp_client(ip, port): """ UDP client decorator :param ip: :param port: :return: """ client = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) def start_client(func): @functools.wraps(func) def send_data(*args, **kwargs): data = func(*args, **kwargs) client.sendto(str.encode(json.dumps(data)), (ip, port)) return send_data return start_client if __name__ == '__main__': import argparse parser = argparse.ArgumentParser() parser.add_argument('-m', '--mode', type=str, help='server/client mode', default='server') parser.add_argument('-i', '--ip', type=str, help='IP address', default='0.0.0.0') parser.add_argument('-p', '--port', type=int, help='UDP port', default=8999) parser.add_argument('-c', '--camera', type=int, help='camera number', default=0) args = parser.parse_args() if args.mode == 'server': @process_udp_server(args.ip, args.port, 1024 * 1024) def multiply(x): time.sleep(1) print(x * 2) multiply() elif args.mode == 'client': @camera_udp_client(args.ip, args.port) def send_single_data(x): return x while True: send_single_data(8) time.sleep(1) else: print('python udp_decorator.py -m [server|client]')
reference
- 《Fluent Python》by Luciano Ramalho