ZeroMQ配合gevent实现高并发

ZeroMQ (0MQ) [官方文档]是一个漂亮的库,它用一个非常轻便的基于模式的包装器,替换socket层。
另外,0MQ通常还保证一次读取将返回一条消息(或多部分消息的一部分)。

gevent是一个基于协程的第三方网络库,它允许你当发生IO阻塞时(如网络请求)去执行其他的操作,当你并行执行很多IO耗时的操作时能达到非常好的并发效果。

gevent是一种折衷方案,可以以同步的操作效率提高到与多线程相当的程度,典型的应用场景就是请求网络的操作。

曾经有一段时间ZeroMQ不支持这一点(必须使用一个名为gevent zmq的包),但后来zmq与gevent兼容了。最新的版本18.1.0是支持的。

为了实现协程并发效果,需要引用import zmq.green 而不是 import zmq

服务端示例代码:

import gevent
import time
import zmq.green as zmq

_BINDING = 'tcp://127.0.0.1:7000'
context = zmq.Context()

def server():
    server_socket = context.socket(zmq.REP)
    server_socket.bind(_BINDING)

    while True:
        received = server_socket.recv()
        print("Received: [{}]\n".format(received))
        server_socket.send_string('TestResponse')

server = gevent.spawn(server)
server.join()

对应的客户端

import gevent
import time
import zmq.green as zmq

_BINDING = 'tcp://127.0.0.1:7000'
context = zmq.Context()

def client():
    client_socket = context.socket(zmq.REQ)
    client_socket.connect(_BINDING)
    client_socket.send_string("TestMessage")
    response = client_socket.recv()
    print("Response: [{}] at {}".format(response, time.time()))

server = gevent.spawn(server)
clients = [gevent.spawn(client) for i in range(1000)]

分别启动服务端和客户端,服务端输出:

Received: [b'TestMessage']

Received: [b'TestMessage']

Received: [b'TestMessage']
.....

客户端输出:

Response: [b'TestResponse'] at 1565866988.859491
Response: [b'TestResponse'] at 1565866988.859531
Response: [b'TestResponse'] at 1565866988.859568
Response: [b'TestResponse'] at 1565866988.859606
Response: [b'TestResponse'] at 1565866988.859644
Response: [b'TestResponse'] at 1565866988.859683
Response: [b'TestResponse'] at 1565866988.859721
Response: [b'TestResponse'] at 1565866988.860918
Response: [b'TestResponse'] at 1565866988.8609679
Response: [b'TestResponse'] at 1565866988.861006
......

参考文档:
Using ZeroMQ With Coroutines (gevent) Under Python

发布了187 篇原创文章 · 获赞 270 · 访问量 172万+

猜你喜欢

转载自blog.csdn.net/liuchunming033/article/details/99649353