Table of contents
1.2 How to judge iterable objects
3.4 Decorator parameter passing
1. Iterator
1.1 What is iteration
Iteration is the process of processing similar data sets through repeated execution of code, and the processing data of this iteration depends on the result of the previous generation as the initial state of the next generation of results. If there is any pause in the middle, it cannot be regarded as iterate.
Common iterable objects are:
Collection data types, such as list, dict, set, str, etc.
Generators, including generators and generator functions with yield
In python, if given a list, tuple, string..., we can traverse through the for loop, this kind of traversal we call iteration (iteration)
1.2 How to judge iterable objects
Iterable objects have the __iter__() method, and iterable objects can be traversed using the for loop. We import the form collections import Iterable module and use isinstance (variable, Iterable) to determine whether a variable is an iterable object. Returning True indicates that it is an iterable object. , False is not an iterable object.
# 3.6之前的版本collections不需要加.abc，3.7之后会提示添加.abc，3.10版本停用不加.abc的情况。
from collections.abc import Iterable
print(isinstance([1, 2, 3, 4], Iterable))
print(isinstance((1, 2, 3, 4), Iterable))
print(isinstance('python', Iterable))
print(isinstance(12345, Iterable))
1.3 Create an iterator
An iterator is an object that remembers where to traverse
Iterator objects are accessed from the first element of the collection until all elements have been accessed. The iterator can only go forward and not backward
Using a class as an iterator requires implementing two methods __iter__() and __next__() in the class
The __iter__() method returns the object itself, ie: self
The __next__() method returns the next data. If there is no data, a StopIterable exception needs to be thrown
class MyNumbers:
'''
创建一个返回数字的迭代器，初始值为1，逐步递增1，在5次迭代后停止执行
'''
def __iter__(self):
self.a = 1
return self
def __next__(self):
if self.a <= 5:
x = self.a
self.a += 1
return x
else:
raise StopIteration
myclass = MyNumbers()
myiter = iter(myclass)
for x in myiter:
print(x)2. Generator
2.1 What is a generator
Through list generation, we can directly create a list, but due to memory constraints, the capacity of the list must be limited. Moreover, creating a list containing 1 million elements not only takes up a lot of storage space, but if we only need to access the first few elements, the space occupied by most of the elements behind will be wasted. So, if the list elements can be calculated according to a certain algorithm, can we continuously calculate the subsequent elements during the loop? In this way, there is no need to create a complete list, which saves a lot of space. In python, this mechanism of calculating while looping is called a generator generator. This characteristic of the generator provides a solution to the problem of solving the contradiction between infinite variables and limited memory, or provides a way to optimize memory usage efficiency.
Using the yield keyword in a function, calling the function each time is similar to executing the next() method on the iterator, and the generator is actually a special iterator.
2.2 Create a generator
way 1
Change the [] in the list generation to ()
# 创建一个生成器、
G = (x * 2 for x in range(10))
print(G)
print(next(G))
way 2
When a for loop similar to list generation cannot be implemented, a function can be used to implement it
# 利用函数实现生成器
def CreatNum():
a, b = 0, 1
for i in range(5):
print(b)
a, b = b, a+b
CreatNum()If there is yield in the function, it becomes a generator
# 利用函数实现生成器
def CreatNum():
print("-----start-----")
a, b = 0, 1
for i in range(5):
print("-----1-----")
yield b
print("-----2-----")
a, b = b, a+b
print("-----3-----")
print("-----stop-----")
a = CreatNum()
print(a)
print(next(a))
print("-"*30)
print(next(a))
CreatNum()Output result:
<generator object CreatNum at 0x0000011CE7786030>
-----start-----
-----1-----
1
------------------------------
-----2-----
-----3-----
-----1-----
1
a is a generator object. When next(a) is called for the first time, the generator executes from top to bottom. When it reaches yeild b, it stops and returns the value of b. When next(a) is called again, The program continues to execute according to the place where it stopped originally. When the loop executes to yeild b, it stops and returns the value of b.
way 3
In the above example, we keep calling yield during the loop without interruption. Of course, a condition must be set for the loop to exit the loop, otherwise an infinite list will be generated. Similarly, after changing the function to a generator, we basically never use next() to get the next return value, but directly use the for loop to iterate, and don't need to care about the StopIteration error
def creatNum():
print("-----start-----")
a, b = 0, 1
for i in range(5):
yield b
a, b = b, a+b
print("-----stop-----")
a = creatNum()
for num in a:
print(num)But when the generator is called with a for loop, it is found that the return value of the return statement of the generator cannot be obtained. If you want to get the return value, you must capture the StopIteration error, and the return value is included in StopIteration
def creatNum():
print("-----start-----")
a, b = 0, 1
for i in range(5):
yield b
a, b = b, a+b
print("-----stop-----")
a = creatNum()
while True:
try:
x = next(a)
print("value:%d" % x)
except StopIteration as e:
print("生成器返回值:%s" % e.value)
break2.3 Generator use cases
start = time.time()
print(sum([i for i in range(1000000)]))
end = time.time()
print("使用列表推导式的耗时", end-start)
print("-----这是一个分割线-----")
start1 = time.time()
print(sum(i for i in range(1000000)))
end1 = time.time()
print("使用生成器的耗时", end1-start1)It can be seen that using the generator will take less time. If the number continues to increase, there is a high probability that the memory of the list comprehension will explode, because the generator is iterated only after it is used, and it will not be stored in the memory after only one iteration, so the memory will be relatively large. Find robust results.
read large files
Using the generator’s feature of suspending and re-running at the suspending point, you can read files of a specified size on demand each time, avoiding memory overflow problems when reading files because too much content is read at one time
def read_file(fpath):
BLOCK_SIZE = 1024
with open(fpath, 'r', encoding='utf-8') as f:
while True:
block = f.read(BLOCK_SIZE)
if block:
yield block
else:
return
a = read_file('C:/readme')
print(next(a))3. Decorator
3.1 Closures
A closure is a function that can read the internal variables of other functions. It can be understood as a function defined inside a function. In essence, a closure is a bridge connecting the inside of the function with the outside of the function.
def test(number):
print("--1--")
def test_in(number2):
print("--2--")
print(number+number2)
print("--3--")
return test_in
ret = test(100)
print("-"*30)
ret(1)
ret(10)First call the external function to pass a default number value, use ret to point to the reference of the returned internal function, and then call ret will be calculated on the basis of the previous call to the external function.
3.2 Decorators
In the program, the original function will not be changed, and new functions will be added, and the interface will not change when the function is called.
Decorators can be implemented based on functions or classes, using steps:
Define a decorator function (class)
Define business functions
Add @decoration function/class name on business function
def w1(func):
"""装饰器函数"""
def inner():
func()
print("这是添加的新功能")
return inner
@w1
def f1():
"""业务函数"""
print("--f1--")
def f2():
"""业务函数"""
print("--f2--")
f1()
f2()
Using multiple decorators at the same time
def w1(fn):
"""装饰器函数"""
def inner():
print("-----1-----")
return "<b>" + fn() + "<b>"
return inner
def w2(fn):
"""装饰器函数"""
def inner():
print("-----2-----")
return "<1>" + fn() + "<i>"
return inner
@w1
@w2
def f1():
"""业务函数"""
print("-----3-----")
return "aaa---bbb---ccc"
ret = f1()
print(ret)First call w1, then call w2, w2 call ends, w1 calls end again
3.3 Decorator execution time
When the python interpreter executes the code where the decorator decorates, the decorator starts to decorate, and there is no need to wait until the function is called to start decorating
def w1(fn):
"""装饰器函数"""
print("---正在装饰---")
def inner():
print("---正在验证---")
fn()
return inner
@w1
def f1():
"""业务函数"""
print("---2---")
return "aaa---bbb---ccc"When the code executes to @w1, it starts to decorate. We did not call f1() to output---decorating---, what we call is the result after decoration.
def w1(fn):
"""装饰器函数"""
print("正在装饰1")
def inner():
print("---正在验证1---")
fn()
return inner()
def w2(fn):
"""装饰器函数"""
print("正在装饰2")
def inner():
print("---正在验证2---")
fn()
return inner
@w1
@w2
def f1():
"""业务函数"""
print("---3---")
return "aaa---bbb---ccc"
@w1 is at the top, and a function is needed below @w1, but @w2 is below @w1, which needs to be decorated after @w2 is finished.
3.4 Decorator parameter passing
Pass two parameter cases
def w1(fn):
"""装饰器函数"""
print("---正在装饰---")
def inner(a, b):
print("正在验证")
fn(a, b)
return inner
@w1
def f1(a, b):
"""业务函数"""
print(a+b)Indeterminate case
def w1(fn):
"""装饰器函数"""
print("---正在装饰---")
def inner(*args, **kwargs):
print("正在验证")
fn(*args, **kwargs)
return inner
@w1
def f1(a, b):
"""业务函数"""
print(a+b)
@w1
def f2(a, b, c, d):
"""业务函数"""
print(a+b+c+d)
f1(10, 20)
f2(10, 20, 30, 40)3.5 Decorator return value
def w1(fn):
"""装饰器函数"""
print("---正在装饰---")
def inner():
print("---正在验证---")
ret = fn() # 保存返回来的字符串
return ret # 把字符串返回到调用的地方
return inner
@w1
def test():
"""业务函数"""
print("---test---")
return "这是原函数返回值"
ret = test() # 参数接收返回值
print(ret)3.6 Generic decorators
def w1(fn):
"""装饰器函数"""
def inner(*args, **kwargs):
print("---记录日志---")
ret = fn(*args, **kwargs)
return ret
return inner
@w1
def test1():
"""不带返回值"""
print("---test1---")
@w1
def test2():
"""带返回值"""
print("---test2---")
return "这是原函数的值"
@w1
def test3(a):
"""业务函数"""
print("---test3中的数据：%d" % a)
ret1 = test1()
print(ret1)
ret2 = test2()
print(ret2)
ret3 = test3(10)
print(ret3)3.7 Decorator with parameters
def func_arg(arg):
def func(funtionName):
def func_in():
print("输出给装饰器传入的参数：%s" % arg)
if arg == "hello":
funtionName()
funtionName()
else:
funtionName()
return func_in
return func
@func_arg("hello")
def test():
print("---test---")
@func_arg("aaa")
def test1():
print("---test1---")
test()
test1()step:
Execute the func_arg() function first, and the result returned by the function is a reference to the function func
@func
Use @func to decorate test