Thread queue event event coroutine

queue

import queue
q = queue.Queue(3)
q.put(1)
q.put(2)
q.put(3)
print(q.get())
print(q.get())
print(q.get())
"""
1
2
3
"""

Stack

t = queue.LifoQueue(4)
t.put(1)
t.put(2)
t.put(3)
t.put(4)
print(t.get())
print(t.get())
print(t.get())
print(t.get())
"""
4
3
2
1
"""

Priority queue

p = queue.PriorityQueue(4)
p.put((5,"oifjanior"))
p.put((-2,"hshts"))
p.put((0,"shtr"))
p.put((3,"hstrhs"))
print(p.get())
print(p.get())
print(p.get())
print(p.get())
"""
(-2,"hshts")
(0,"shtr")
(3,"hstrhs")
(5,"oifjanior")
"""

Event event

If other threads in the program need to determine their next action by judging the state of a thread

A version

from threading import current_thread, Thread
import time

flag = False
def check():
    print(f"{current_thread().name}监测服务器是否开启")
    time.sleep(3)
    global flag
    flag = True
    print("服务器已经开启")

def connect():
    while 1:
        print(f"{current_thread().name}等待连接...")
        time.sleep(0.5)
        if flag:
            print(f"{current_thread().name}连接成功...")
            break
t1 = Thread(target=check)
t2 = Thread(target=connect)
t1.start()
t2.start()
"""
Thread-1监测服务器是否开启
Thread-2等待连接...
Thread-2等待连接...
Thread-2等待连接...
Thread-2等待连接...
Thread-2等待连接...
Thread-2等待连接...
服务器已经开启
Thread-2连接成功...
"""

Version two

from threading import Thread, current_thread, Event
import time

event = Event()
def check():
    print(f"{current_thread().name} 监测服务器是否开启...")
    time.sleep(3)
    print(event.is_set()) 
    # 判断事件是否开启
    event.set()
    print(event.is_set())
    print("服务器已经开启...")

def connect():
    print(f"{current_thread().name} 等待连接...")
    event.wait() 
    # 阻塞, 直到 event.set() 方法之后
    # event.wait(1) 只阻塞1秒, 1秒之后如果还没有进行set 直接进行下一步操作
    print(f"{current_thread().name}连接成功...")

t1 = Thread(target=check)
t2 = Thread(target=connect)
t1.start()
t2.start()

Open two threads, one thread runs into the middle of a stage, triggering another thread executes two threads increases the coupling

Exercise

Whether to start a thread server monitoring, to determine if another thread started, it shows the connection is successful, try to connect this thread only three times, 1s once, if more than three times, yet the connection is successful, it shows the connection fails.

from threading import current_thread,Thread,Event
import time

event = Event()
def check():
    print(f"{current_thread().name}监测服务器是否开启...")
    time.sleep(4)
    event.set()
    print("服务器已开启")

def connect():
    count = 1
    while not event.is_set():
        if count == 4:
            print("连接次数过多, 已断开")
            break
        event.wait(1)
        print(f"{current_thread().name}尝试连接{count}次")
        count += 1
    else:
        print(f"{current_thread().name}连接成功..")

t1 = Thread(target=check)
t2 = Thread(target=connect)
t1.start()
t2.start()

Coroutine

A thread concurrent processing tasks

Serial:

A thread executing a task, after the execution is completed, the next task.

Parallel:

Multiple CPU to perform multiple tasks, four CPU to perform four tasks

Concurrency:

A CPU to perform multiple tasks at the same time looks like a run

Concurrent nature:

Holding state switching +

Multithreading map

3 threading 10, this quest if the thread 1 processing experience blocking, CPU by the operating system to switch to another thread

​

Single-threaded concurrency map

One thread to handle three tasks

Concurrent way:

Single CPU, concurrent execution of tasks 10

1. Turn on the concurrent execution of multiple processes, the operating system is switched on hold +

2. Turn on the concurrent execution of multiple threads, the operating system is switched on hold +

3. Open coroutine execution concurrent with its own procedures to control the CPU, the switching back and forth between the three tasks hold +

Detailed third way

Coroutine switching speed is very fast, blinded by the operating system of the eye, let the operating system believe that this CPU has been running a thread (coroutine)

the best way

Coroutine, because the overhead is small, fast, long-term occupation of CPU coroutine will perform all the tasks I just inside the program

Coroutine process using high efficiency IO intensive, computationally intensive, the use of a serial or

Coroutine

A plurality of single thread concurrent processing tasks, switching program control hold state coroutine +

Switch

def func1():
    print("in func1")

def func2():
    print("in func2")
    func1()
    print("end")
func2()

Hold switching +

def gen():
    while 1:
        yield 1

def func():
    obj = gen()
    for i in range(10):
        next(obj)
func()

Coroutine

import gevent
from gevent import monkey

monkey.patch_all()
def eat(name):
    print(f"{name} eat 1")
    gevent.sleep(2)
    print(f"{name} eat 2")

def play(name):
    print(f"{name} play 1")
    gevent.sleep(1)
    print(f"{name} play 2")

g1 = gevent.spawn(eat, "egon")
g2 = gevent.spawn(play, "egon")

gevent.joinall([g1,g2])

Coroutine features

1. must be implemented concurrently in only a single thread in

2. Modify the shared data without locking

3. The user program context save their stacks plurality of flow control (hold)

4. Additional: a coroutine encountered IO operation is automatically switched to another coroutine

Work

​ 一般在工作中我们都是进程+线程+协程的方式来实现并发，以达到最好的并发效果，如果是4核的cpu，一般起5个进程，每个进程中20个线程（5倍cpu数量），每个线程可以起500个协程，大规模爬取页面的时候，等待网络延迟的时间的时候，我们就可以用协程去实现并发。 并发数量 = 5 * 20 * 500 = 50000个并发，这是一般一个4cpu的机器最大的并发数。nginx在负载均衡的时候最大承载量就是5w个单线程里的这20个任务的代码通常会既有计算操作又有阻塞操作，我们完全可以在执行任务1时遇到阻塞，就利用阻塞的时间去执行任务2。。。。如此，才能提高效率，这就用到了Gevent模块