Higher order functions

First Class Object
Functions are also objects, callable objects
Functions can be used as ordinary variables, parameters, return values, etc.
Mathematical concept y=g(f(x))
In mathematics and computer science, a higher-order function should be a function that satisfies at least one of the following conditions
accepts one or more functions as arguments
output a function

Built-in higher-order functions

sorted(iterable[, key][, reverse])
- sort
filter(function, iterable) --> filter object
- filter data
map(func, *iterables) --> map object
- map

sorted(iterable[, key][, reverse]) 排序

Returns a new list that sorts all elements of an iterable object. The sorting rule is the function defined by key, and reverse indicates whether the sorting is reversed or not.

filter(function, iterable)

Filters the elements of an iterable, returning an iterator
function a function with one argument, returning bool

map(function, *iterables) --> map object

Map the elements of multiple iterable objects according to the specified function and return an iterator

Currying

Refers to the process of changing the original function that accepts two parameters into a new function that accepts one parameter. The new function returns a function that takes the original second argument
z = f(x, y) ==> z = f(x)(y)
Example
of Currying the Addition Function
```
def add(x, y):
return x + y
```
Convert as follows
```
def add(x):
def _add(y):
return x+y
return _add
add(5)(6)
```
Convert functions to curried functions by nesting functions

decorator

In the OOP design pattern, decorators belong to a type of decorator pattern.
The decorator syntax is a syntactic sugar

def logger(func):
    def wrapper(*args, **kwargs):
        print('call ' + func.__name__)
        return func(*args, **kwargs)
    return wrapper

@logger # Equivalent to add = logger(add)
def add(x,y):
return x + y

decorator (no arguments)

it's a function
function as its formal parameter
The return value is also a function
You can use the @functionname method to simplify the call

A decorator is a higher-order function, but a decorator is a decoration (function enhancement) of the function passed into the function

docstring

In the first line of the function statement block, and the habit is multi-line text, so use triple quotation marks more
The convention is to capitalize the first letter, with an overview on the first line, an empty line, and a detailed description on the third line
This document can be accessed using the special attribute doc

Decorator Side Effects

The properties of the original function object have been replaced (the properties of the wrapper are returned after execution), and using the decorator, our requirement is to view the properties of the wrapped function

Solution:

1. Provide a function, encapsulated function attribute ==copy==> wrapper function attribute

def copy_properties(src, dst): # 可以改造成装饰器
    dst.__name__ = src.__name__
    dst.__doc__ = src.__doc__

Overwrite the wrapper function with the properties of the wrapper function through the copy_properties function
All decorated functions need to copy these properties, this function is very general
The function that copies the property can be constructed as a decorator function, with a parameter decorator

2.使用functools.update_wrapper(wrapper, wrapped)

import datetime, time, functools

def logger(duration, func=lambda name, duration: print('{} took {}s'.format(name, duration))):
    def _logger(fn):
        def wrapper(*args,**kwargs):
            start = datetime.datetime.now()
            ret = fn(*args,**kwargs)
            delta = (datetime.datetime.now() - start).total_seconds()
            if delta > duration:
                func(fn.__name__, duration)
            return ret
        return functools.update_wrapper(wrapper, fn)
    return _logger

@logger(5) # add = logger(5)(add)
def add(x,y):
    time.sleep(1)
    return x + y
print(add(5, 6), add.__name__, add.__wrapped__, add.__dict__, sep='\n')

3.使用@functools.wraps(wrapped)

import datetime, time, functools

def logger(duration, func=lambda name, duration: print('{} took {}s'.format(name, duration))):
    def _logger(fn):
        @functools.wraps(fn)
        def wrapper(*args,**kwargs):
            start = datetime.datetime.now()
            ret = fn(*args,**kwargs)
            delta = (datetime.datetime.now() - start).total_seconds()
            if delta > duration:
                func(fn.__name__, duration)
            return ret
        return wrapper
    return _logger

@logger(5) # add = logger(5)(add)
def add(x,y):
    time.sleep(1)
    return x + y

print(add(5, 6), add.__name__, add.__wrapped__, add.__dict__, sep='\n')

With parameter decorator

it is a function function as its formal parameter
The return value is a decorator function that takes no arguments
Use @functionname (parameter list) method to call
It can be seen as adding another layer of functions to the outer layer of the decorator

Extract the recorded functions, so that the output can be flexibly controlled by externally provided functions

parameter annotation

Shaped like:

def add(x:int , y:int) -> int :
'''
:param x: int
:param y: int
:return: int
'''
return x + y

Introduced in Python 3.5
Type annotations on function parameters
Type annotations on the return value of a function
Only an auxiliary description is made for the function parameters, and no type checking is performed on the function parameters.
Provided to third-party tools for code analysis and discovery of hidden bugs
Function annotation information, stored in the annotations attribute
The key is the parameter name and the value is the class type

Variable annotations are used less

inspect module

signature(callable), returns a Signature class object. Get the signature (the function signature contains information about a function, including the function name, its parameter types, its class and namespace, and other information)

sig = signature(fn)
params = sig.parameters

The parameters method of the Signature class object returns an orderedDict whose key value is the formal parameter name of fn, and the value value is the Parameter class object

Parameter object

Stored in a tuple, read-only

name, the name of the parameter
annotation, the annotation of the parameter, may not be defined
default, the default value of the parameter, may not be defined
empty, a special class used to mark the empty value of the default attribute or annotate the annotation attribute
kind, how the actual parameter is bound to the formal parameter, is the type of the formal parameter
- POSITIONAL_ONLY, value must be a positional parameter provided #Python does not implement secondary parameter types
- POSITIONAL_OR_KEYWORD, the value can be provided as a keyword or positional parameter
- VAR_POSITIONAL, variable positional parameters, corresponding to *args
- KEYWORD_ONLY, keyword-only parameter, corresponding or non-variable keyword parameter that appears after args
- VAR_KEYWORD, variable keyword parameter, corresponding to **kwargs

Business Applications

Function parameter type checking
ideas
- The check of function parameters must be outside the function
- The function should be passed as a parameter to the check function
- Check the function to get the actual parameters passed in by the function, compared with the formal parameter declaration
- The annotations property is a dictionary containing declarations of return value types. Assuming that the judgment of positional parameters is to be done, it cannot correspond to the declaration in the dictionary. Use the inspect module
The inspet module provides functions to obtain object information, can check functions and classes, type checking
Code:

import inspect
def check(fn):
    def wrapper(*args,**kwargs):
        sig = inspect.signature(fn)
        params = sig.parameters #是orderedDict
        values = list(params.values()) #可迭代对象转化成列表，处理后也是有序的
        for k,v in enumerate(args): #因为values、args都有序
            if values[k].annotation is not inspect._empty and not isinstance(v,values[k].annotation):
                return '{}`s input is {} type , expected {}'.format(v,type(v),values[k].annotation)
        for k,v in kwargs.items(): #kwargs与params都为字典形式，键值相同
            if params[k].annotation is not inspect._empty and not isinstance(v, params[k].annotation):
                return '{}`s input is {} type , expected {}'.format(k,type(v),params[k].annotation)
        return fn(*args,**kwargs)
    return wrapper

@check
def add(x:int,y:int):
    return x + y

functools module

partial function

Fixing the parameters of the function part is equivalent to adding a fixed default value to the parameters of the part, forming a new function and returning
The new function generated from partial is the encapsulation of the original function

partial source code logic

def partial(func, *args, **keywords):
def newfunc(*fargs, **fkeywords): # 包装函数
newkeywords = keywords.copy() #将原函数关键字参数拷贝后，
newkeywords.update(fkeywords)  #将调用新函数时的关键字参数更新
return func(*(args + fargs), **newkeywords) #调用新函数时的参数是建立偏函数时给的参
                                                                         #数，再各加上调用新函数时的参
newfunc.func = func # 保留原函数,注意：如果原函数已有装饰器，func属性内容有可能不是原函数名，而是装饰器函数名
newfunc.args = args # 保留原函数的位置参数
newfunc.keywords = keywords # 保留原函数的关键字参数参数
return newfunc

partial() returns a new function newfunc, and newfunc returns the original function whose parameters are the parameters given when the partial function was created, plus the parameters when calling the new function.

LRU cache decorator

@functools.lru_cache(maxsize=128, typed=False)
Least-recently-used decorator. lru, least recently used. cache cache
If maxsize is set to None, the LRU feature is disabled and the cache can grow indefinitely. LRU functions perform best when maxsize is a power of two
If typed is set to True, function arguments of different types will be cached separately. For example, f(3) and f(3.0) will be treated as different calls with different results
Cache calls and return values of decorated functions through a dictionary
If they are all positional arguments and the position and value are the same, they are regarded as the same arguments, or if all keyword arguments are passed in different orders, they are considered the same arguments, that is, the cache can be used; otherwise, they are considered different. Arguments
lru_cache uses _make_key to generate parameter cache

Source code:

def _make_key(args, kwds, typed,
         kwd_mark = (object(),),
         fasttypes = {int, str, frozenset, type(None)},
         tuple=tuple, type=type, len=len):
"""Make a cache key from optionally typed positional and keyword arguments

The key is constructed in a way that is flat as possible rather than
as a nested structure that would take more memory.

If there is only a single argument and its data type is known to cache
its hash value, then that argument is returned without a wrapper.  This
saves space and improves lookup speed.

"""
key = args
if kwds:
    key += kwd_mark
    for item in kwds.items():
        key += item
if typed:
    key += tuple(type(v) for v in args)
    if kwds:
        key += tuple(type(v) for v in kwds.values())
elif len(key) == 1 and type(key[0]) in fasttypes:
    return key[0]
return _HashedSeq(key)

lru_cache decorator application

Prerequisites for use
- The same function parameters must yield the same result
- The function execution time is very long and needs to be executed multiple times
- The essence is the parameters of the function call => return value
shortcoming
- Does not support cache expiration, the key cannot expire or become invalid
- Clear operation is not supported
- Does not support distributed, is a stand-alone cache
- Applicable scenarios, where space is needed for time on a single machine, you can use cache to turn calculations into fast queries

Python fifth week study notes (1)