Python utility library introduction
bisect manages ordered lists
The bisect module contains two main functions, bisect and insort. Both functions use binary search algorithms to find and insert elements in an ordered sequence.
# BEGIN BISECT_DEMO
import bisect
import sys
HAYSTACK = [1, 4, 5, 6, 8, 12, 15, 20, 21, 23, 23, 26, 29, 30]
NEEDLES = [0, 1, 2, 5, 8, 10, 22, 23, 29, 30, 31]
ROW_FMT = '{0:2d} @ {1:2d} {2}{0:<2d}'
def demo(bisect_fn):
for needle in reversed(NEEDLES):
position = bisect_fn(HAYSTACK, needle) # <1>
offset = position * ' |' # <2>
print(ROW_FMT.format(needle, position, offset)) # <3>
if __name__ == '__main__':
if sys.argv[-1] == 'left': # <4>
bisect_fn = bisect.bisect_left
else:
bisect_fn = bisect.bisect
print('DEMO:', bisect_fn.__name__) # <5>
print('haystack ->', ' '.join('%2d' % n for n in HAYSTACK))
demo(bisect_fn)
# END BISECT_DEMO
'''
DEMO: bisect
haystack -> 1 4 5 6 8 12 15 20 21 23 23 26 29 30
31 @ 14 | | | | | | | | | | | | | |31
30 @ 14 | | | | | | | | | | | | | |30
29 @ 13 | | | | | | | | | | | | |29
23 @ 11 | | | | | | | | | | |23
22 @ 9 | | | | | | | | |22
10 @ 5 | | | | |10
8 @ 5 | | | | |8
5 @ 3 | | |5
2 @ 1 |2
1 @ 1 |1
0 @ 0 0
'''import bisect
import random
SIZE = 7
random.seed(1729)
my_list = []
for i in range(SIZE):
new_item = random.randrange(SIZE*2)
bisect.insort(my_list, new_item)
print('%2d ->' % new_item, my_list)
'''
10 -> [10]
0 -> [0, 10]
6 -> [0, 6, 10]
8 -> [0, 6, 8, 10]
7 -> [0, 6, 7, 8, 10]
2 -> [0, 2, 6, 7, 8, 10]
10 -> [0, 2, 6, 7, 8, 10, 10]
'''Among them, bisect and insort functions (essentially bisect_left and insort_left) have similar bisect_right and insort_right. The difference between them is that if there are elements with the same calculation result in the list, then the new element will be inserted on the left or right of these elements.
array efficient numerical storage structure
Although the list is flexible and simple, it is a container structure (it is not a specific value, but a pointer to the corresponding element. Although this makes it more widely used, it requires greater access and storage) , So if the stored content is only a value, you can use an array, which is not a float object, but a machine translation of the number. At the same time, array supports almost all list operations.
from array import array
from random import random
floats = array('d', (random() for i in range(10**7)))
print(floats[-1])
# 0.7030774022684948
fp = open("floats.bin", 'wb')
floats.tofile(fp)
fp.close()
floats = array('d')
fp = open("floats.bin", 'wb')
floats2.fromfile(fp, 10**7)
fp.close()
print(floats2[-1])
# 0.7030774022684948
print(floats == floats2)
# TrueSince the array only allows certain types of values to be stored in it, the data type to be stored needs to be specified when creating it. The specific type names are shown in the table below.
memoryview memory view
memoryview allows users to manipulate different slices of the same array without copying the content.
import array
numbers = array.array('h', [-2, -1, 0, 1, 2])
memv = memoryview(numbers)
print(len(memv)) # 5
print(memv[0]) # -2
memv_oct = memv.cast('B')
print(memv_oct.tolist()) # [254, 255, 255, 255, 0, 0, 1, 0, 2, 0]
memv_oct[5] = 4
print(numbers)
# array('h', [-2, -1, 1024, 1, 2])Another alias memv of data numbers was created, and the data type conversion was realized through memv.cast later. At the same time, if memv is modified, the original data will also be modified.
deque
The collections.deque class is a thread-safe data structure that can quickly add and delete elements from both ends. And if you want a data type to store the "recently used elements", deque is also a good choice. This is because when creating a two-way queue, you can specify the size of the queue. If the queue is full, you can also delete expired elements from the reverse direction, and then add new elements at the end.
from collections import deque
dq = deque(range(10), maxlen=10)
print(dq) # deque([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], maxlen=10)
dq.rotate(3)
print(dq) # deque([7, 8, 9, 0, 1, 2, 3, 4, 5, 6], maxlen=10)
dq.rotate(-4)
print(dq) # deque([1, 2, 3, 4, 5, 6, 7, 8, 9, 0], maxlen=10)
dq.appendleft(-1)
print(dq) # deque([-1, 1, 2, 3, 4, 5, 6, 7, 8, 9], maxlen=10)
dq.extend([11, 22, 33])
print(dq) # deque([3, 4, 5, 6, 7, 8, 9, 11, 22, 33], maxlen=10)
dq.extendleft([10, 20, 30, 40])
print(dq) # deque([40, 30, 20, 10, 3, 4, 5, 6, 7, 8], maxlen=10)deque supports almost all list operations, but deque does not support pop(p) operations.
queue
It provides synchronization (thread-safe) classes Queue, LifoQueue and PriorityQueue. Different threads can use these data classes to exchange information. The construction methods of these three classes have an optional parameter maxsize, which accepts a positive integer as input to limit the size of the queue. But when full, these classes will not throw away old elements to make room. Conversely, if the queue is full, it will be locked, knowing that another thread has removed an element to make room. This feature makes these classes very suitable for controlling the number of active threads.
The multiprocessing package implements its own Queue, which is similar to queue.Queue and is designed for inter-process communication. There is also a special multiprocessing.JoinableQueue type, which can make task management more convenient.
The asyncio package includes Queue, LifoQueue, PriorityQueue and JoinableQueue. These classes provide special convenience for task management in asynchronous programming.