Why is Hyperf able to listen on two ports for WebSocket connections?

From a source code perspective, when multiple Servers are configured, only one Server is actually started.

Note: The code I have been exposed to before starts a service and binds a port. I have read the swoole extension document before, but I did not notice that the service and the listening port are also separated. This inspired me to think that the code can usually be disassembled. If it is divided, continue to split it, so that the code will be more flexible and each function can be expanded. After splitting the service and port, you can bind multiple ports to a server, and each port can be Can have independent events.

/**
 * @param Port[] $servers
 * @return Port[]
 */
protected function sortServers(array $servers): array
{
    $sortServers = [];
    foreach ($servers as $server) {
        switch ($server->getType()) {
            case ServerInterface::SERVER_HTTP:
                $this->enableHttpServer = true;
                if (! $this->enableWebsocketServer) {
                    array_unshift($sortServers, $server);
                } else {
                    $sortServers[] = $server;
                }
                break;
            case ServerInterface::SERVER_WEBSOCKET:
                $this->enableWebsocketServer = true;
                array_unshift($sortServers, $server);
                break;
            default:
                $sortServers[] = $server;
                break;
        }
    }

    return $sortServers;
}

From the source code, the first service configuration will be created, and then the monitoring will be added.

protected function initServers(ServerConfig $config)
{
    $servers = $this->sortServers($config->getServers());

    foreach ($servers as $server) {
        $name = $server->getName();
        $type = $server->getType();
        $host = $server->getHost();
        $port = $server->getPort();
        $sockType = $server->getSockType();
        $callbacks = $server->getCallbacks();

        if (! $this->server instanceof SwooleServer) {
            $this->server = $this->makeServer($type, $host, $port, $config->getMode(), $sockType);
            $callbacks = array_replace($this->defaultCallbacks(), $config->getCallbacks(), $callbacks);
            $this->registerSwooleEvents($this->server, $callbacks, $name);
            $this->server->set(array_replace($config->getSettings(), $server->getSettings()));
            ServerManager::add($name, [$type, current($this->server->ports)]);

            if (class_exists(BeforeMainServerStart::class)) {
                // Trigger BeforeMainServerStart event, this event only trigger once before main server start.
                $this->eventDispatcher->dispatch(new BeforeMainServerStart($this->server, $config->toArray()));
            }
        } else {
            /** @var bool|\Swoole\Server\Port $slaveServer */
            $slaveServer = $this->server->addlistener($host, $port, $sockType);
            if (! $slaveServer) {
                throw new \RuntimeException("Failed to listen server port [{$host}:{$port}]");
            }
            $server->getSettings() && $slaveServer->set(array_replace($config->getSettings(), $server->getSettings()));
            $this->registerSwooleEvents($slaveServer, $callbacks, $name);
            ServerManager::add($name, [$type, $slaveServer]);
        }

        // Trigger beforeStart event.
        if (isset($callbacks[Event::ON_BEFORE_START])) {
            [$class, $method] = $callbacks[Event::ON_BEFORE_START];
            if ($this->container->has($class)) {
                $this->container->get($class)->{$method}();
            }
        }

        if (class_exists(BeforeServerStart::class)) {
            // Trigger BeforeServerStart event.
            $this->eventDispatcher->dispatch(new BeforeServerStart($name));
        }
    }
}

Judging from the makeServer function, if there is SERVER_WEBSOCKET in the service, this will be started as the main service, new SwooleWebSocketServer

protected function makeServer(int $type, string $host, int $port, int $mode, int $sockType): SwooleServer
{
    switch ($type) {
        case ServerInterface::SERVER_HTTP:
            return new SwooleHttpServer($host, $port, $mode, $sockType);
        case ServerInterface::SERVER_WEBSOCKET:
            return new SwooleWebSocketServer($host, $port, $mode, $sockType);
        case ServerInterface::SERVER_BASE:
            return new SwooleServer($host, $port, $mode, $sockType);
    }

    throw new RuntimeException('Server type is invalid.');
}

$this->registerSwooleEvents($this->server, $callbacks, $name); This code will register various events of Websocket, so the main server has various events of websocket, and then the http server mounts port 9501 On, the onrequest event is bound, but if there is a websocket connected to the 9501 port, the server will automatically enable the websocket automatic upgrade by default, and because the main server that listens to the 9501 port is bound to the WebSocketServer, therefore, the default onmessage, onopen of the WebSocketServer events will be used.

It is speculated that if the 9503 WebSocket server is enabled, then theoretically using WebSocket to connect to the 9501 port should be the callback event of the connected 9503. If you do not want the http listening port to automatically open the websocket protocol, set open_websocket_protocol=false

<?php
return [
// 这里省略了该文件的其它配置
'servers' => [
        [
            'name' => 'http',
            'type' => Server::SERVER_HTTP,
            'host' => '0.0.0.0',
            'port' => 9501,
            'sock_type' => SWOOLE_SOCK_TCP,
            'callbacks' => [
                Event::ON_REQUEST => [Hyperf\HttpServer\Server::class,'onRequest'],
             ],
             'settings' => ['open_websocket_protocol' => false,]
         ],
     ]
 ];

Regarding many constants that appear in SWOOLE, these things will be automatically set when executing the script, and can be found by running the actual code.

define('SWOOLE_HTTP2_ERROR_COMPRESSION_ERROR', 9);
define('SWOOLE_HTTP2_ERROR_CONNECT_ERROR', 10);
define('SWOOLE_HTTP2_ERROR_ENHANCE_YOUR_CALM', 11);
define('SWOOLE_HTTP2_ERROR_INADEQUATE_SECURITY', 12);
define('SWOOLE_BASE', 1);
define('SWOOLE_PROCESS', 2);
define('SWOOLE_IPC_UNSOCK', 1);
define('SWOOLE_IPC_MSGQUEUE', 2);
define('SWOOLE_IPC_PREEMPTIVE', 3);

Below, SWOOLE_BASE can be output in a randomly set test.php, and it can output 1. I thought that these constants were set at runtime. It seems that this understanding is wrong.

<?php
echo SWOOLE_BASE."\n";
echo "hello\n";

// 输出
1
hello

The default configuration given by Hyperf-skeleton is the process mode. This is not found, so it is more clear. The PROCESS mode is used, so when the websocket is connected, all connections are controlled by the Manager.

SWOOLE_PRECESS and SWOOLE_BASE two modes

Introduction to the two operating modes of Server

In the third parameter of the Swoole\Server constructor, you can fill in 2 constant values -- SWOOLE_BASE or SWOOLE_PROCESS . The differences, advantages and disadvantages of these two modes will be introduced below

SWOOLE_PROCESS

In the SWOOLE_PROCESS mode Server, all client TCP connections are established with the main process . The internal implementation is relatively complex and uses a large number of inter-process communication and process management mechanisms. Suitable for scenarios with very complex business logic. Swoole provides a complete process management and memory protection mechanism. Even when the business logic is very complex, it can run stably for a long time.

Swoole provides Buffer functionality in the Reactor thread, which can cope with large numbers of slow connections and byte-by-byte malicious clients.

Advantages of process mode:

Connections and data request sending are separated, and the Worker process will not be unbalanced because some connections have a large amount of data and some connections have a small amount of data.
When a fatal error occurs in the Worker process, the connection will not be cut off.
Single connection concurrency can be achieved, only a small number of TCP connections are maintained, and requests can be processed concurrently in multiple Worker processes.

Disadvantages of process mode:

There is the overhead of 2 IPCs. The master process and the worker process need to use unixSocket to communicate.
SWOOLE_PROCESS does not support PHP ZTS, in this case you can only use SWOOLE_BASE or set single_thread to true

SWOOLE_BASE

SWOOLE_BASE This mode is a traditional asynchronous non-blocking Server. It is completely consistent with programs such as Nginx and Node.js.

The worker_num parameter is still valid for BASE mode, and multiple Worker processes will be started.

When a TCP connection request comes in, all Worker processes compete for this connection, and eventually one worker process will successfully establish a TCP connection directly with the client. After that, all data sent and received in this connection will communicate directly with this worker. Forwarded by the Reactor thread of the main process.

There is no role of the Master process in BASE mode, only the role of the Manager process.
Each Worker process simultaneously assumes the responsibilities of the Reactor thread and Worker process in SWOOLE_PROCESS mode .
The Manager process is optional in BASE mode. When worker_num=1 is set and the Task and MaxRequest features are not used, the bottom layer will directly create a separate Worker process without creating a Manager process

Advantages of BASE mode:

BASE mode has no IPC overhead and better performance
BASE mode code is simpler and less error-prone

Disadvantages of BASE mode:

TCP connections are maintained within the Worker process, so when a Worker process hangs up, all connections within this Worker will be closed.
A small number of long TCP connections cannot utilize all Worker processes
The TCP connection is bound to the Worker. Some connections in long-connection applications have a large amount of data, and the load of the Worker process where these connections are located will be very high. However, the data volume of some connections is small, so the load on the Worker process will be very low, and different Worker processes cannot achieve balance.
If there are blocking operations in the callback function, the server will degenerate into synchronous mode, which may easily cause the TCP backlog queue to become full.

Applicable scenarios of BASE mode:

BASE mode can be used if interaction between client connections is not required. Such as Memcache, HTTP server, etc.

BASE mode limitations:

In BASE mode, except for send and close , other Server methods do not support cross-process execution.

Reactor threads and Worker processes

Reactor thread

Reactor thread is a thread created in the Master process
Responsible for maintaining client TCP connections, processing network IO, processing protocols, and sending and receiving data
Does not execute any PHP code
Buffer, splice, and split the data sent from the TCP client into a complete request packet

Worker process

Accept the request packet delivered by the Reactor thread and execute the PHP callback function to process the data
Generate response data and send it to the Reactor thread, which then sends it to the TCP client
It can be asynchronous non-blocking mode or synchronous blocking mode.
Worker runs in multiple processes

The relationship between them can be understood as Reactor is nginx, and Worker is PHP-FPM. The Reactor thread processes network requests asynchronously and in parallel, and then forwards them to the Worker process for processing. Communication between Reactor and Worker is through unixSocket .

In the application of PHP-FPM, a task is often posted asynchronously to a queue such as Redis, and some PHP processes are started in the background to process these tasks asynchronously. TaskWorker provided by Swoole is a more complete solution that integrates task delivery, queue, and PHP task processing process management. The processing of asynchronous tasks can be implemented very simply through the API provided by the underlying layer. In addition, TaskWorker can also return a result to the Worker after the task execution is completed.

Swoole's Reactor, Worker, and TaskWorker can be closely integrated to provide more advanced usage methods.

A more popular metaphor, assuming that Server is a factory, then Reactor is sales, accepting customer orders. The Worker is the worker. When the salesperson receives the order, the Worker goes to work to produce what the customer wants. TaskWorker can be understood as an administrative staff, which can help Worker do some chores and let Worker concentrate on work.

in conclusion

In SWOOLE_PROCESS mode, the Websocket connection object is controlled by the Server. After creating a Reactor, the connection is delivered to the Reactor for management. The Reactor will assign the request to the Worker for processing. After the Worker finishes processing, it will send the message to the Server. Server pushes the message into the queue and waits for the specific Reactor to send it out.
There is an inter-process communication in hyperf. It is not clear why it is necessary to ask other processes for help here, and whether other processes will send more messages after listening to the message.
1. The developer said that if it is in SWOOLE_PROCESS mode, it will not trigger the mechanism for other Workers to send. Only in SWOOLE_BASE mode, when each link is handed over to each Worker for separate processing, it is necessary to cooperate with multiple Workers, because each Worker contends for The connections grabbed are all isolated, so there will be no situation where multiple messages are sent.
The above 2 gives a distributed websocket construction method. With this multi-process method, multiple servers can cooperate to provide long-term connection services, ensuring the access of thousands of users

The following code comes from the onPipeMessage listener, which means that after other workers receive it, they judge that the connection is on their own process and execute it.