NodeJs is a single-process language, and it cannot create multiple threads for concurrent execution like Java. Of course, in most cases, NodeJs does not need to be executed concurrently, because it is event-driven and never blocks. But a single process also has a problem that it cannot make full use of the multi-core mechanism of the CPU. According to previous experience, you can make full use of the multi-core CPU by creating multiple processes, and Node uses the child_process module to create and complete multi-process operations.
The child_process module gives node the ability to create child processes arbitrarily. The official node document gives four methods for the child_process module. Mapping to the operating system is actually creating child processes. But for developers, the APIs of these methods are a bit different
child_process.exec(command[, options][, callback]) Start the child process to execute the shell command, you can get the script shell execution result through the callback parameter
child_process.execfile(file[, args][, options][, callback]) The difference from the exec type is that it executes not a shell command but an executable file
child_process.spawn(command[, args][, options]) only executes a shell command and does not need to obtain the execution result
child_process.fork(modulePath[ , args][, options]) is a .js file that can be executed by node, and there is no need to obtain the execution result. The child process from fork must be a node process
When exec() and execfile() are created, you can specify the timeout attribute to set the timeout period. Once timeout, it will be killed.
If you use execfile() to execute an executable file, the header must be #!/usr/bin/env node
Inter-process communication
Communication between node and child processes is done using the IPC pipeline mechanism. If the child process
is also a node process (using fork), you can listen to the message event and use send() to communicate.
main.js
var cp = require('child_process');
//只有使用fork才可以使用message事件和send()方法
var n = cp.fork('./child.js');
n.on('message',function(m){
console.log(m);
})
n.send({"message":"hello"});
child.js
var cp = require('child_process');
process.on('message',function(m){
console.log(m);
})
process.send({"message":"hello I am child"})
An IPC channel will be created between the parent and child processes, and the message event and send() will use the IPC channel to communicate.
Handle transfer
After learning how to create subprocesses, we create an HTTP service and start multiple processes to jointly
make full use of CPU multi-core.
worker.js
var http = require('http');
http.createServer(function(req,res){
res.end('Hello,World');
//监听随机端口
}).listen(Math.round((1+Math.random())*1000),'127.0.0.1');main.js
var fork = require('child_process').fork;
var cpus = require('os').cpus();
for(var i=0;i<cpus.length;i++){
fork('./worker.js');
}
The above code will create a corresponding number of fork processes according to the number of your cpu cores, and each process listens to a random port to provide HTTP services.
The above completes a typical Master-Worker master-slave replication mode. It is used for parallel processing business in distributed applications and has good shrinkage and stability. It should be noted here that forking a process is expensive, and node single-process event-driven has good performance. The multiple fork processes in this example are to make full use of the CPU cores, not to solve the concurrency problem.
One of the disadvantages of the above example is that it occupies too many ports, so can you use the same port for all the child processes so that the external Only this port is used to provide http services. Try to change the random number of the above port to 8080, and you will find the following exception thrown when you start it.
events.js:72
throw er;//Unhandled 'error' event
Error:listen EADDRINUSE
XXXX
Throw an exception that the port is occupied, which means that only one worker.js can listen to port 8080, and the rest will throw an exception.
If you want to solve the problem of providing a port externally, you can refer to the practice of nginx reverse proxy. For the Master process, port 80 is used to provide external services, while for the fork process, a random port is used. When the Master process receives the request, it forwards it to the fork process.
For the proxy mode just mentioned, because the process will use up a file descriptor every time it receives a connection, so in the proxy mode, the client connects to the proxy process, and the proxy process connects to the fork process and uses two file descriptors. The file descriptors in the OS are limited. In order to solve this problem, node introduces the function of sending handles between processes.
In node's IPC process communication API, the second parameter of send(message, [sendHandle]) is the handle.
A handle is a reference that identifies a resource, and it contains a file descriptor pointing to an object. The handle can be used to describe a socket object, a UDP socket, and a
pipe. Sending a handle from the main process to the worker process means that when the main process receives the socket request from the client, it will directly send the socket to the worker process without further communication. If the worker process establishes a socket connection, the waste of file descriptors can be solved. Let's look at the sample code:
main.js
var cp = require('child_process');
var child = cp.fork('./child.js');
var server = require('net').createServer();
//监听客户端的连接
server.on('connection',function(socket){
socket.end('handled by parent');
});
//启动监听8080端口
server.listen(8080,function(){
//给子进程发送TCP服务器(句柄)
child.send('server',server);
});
child.js
process.on('message',function(m,server){
if(m==='server'){
server.on('connection',function(socket){
socket.end('handle by child');
});
}
});
You can test using telnet or curl:
wang@wang ~/code/nodeStudy $ curl 192.168.10.104:8080
handled by parent
wang@wang ~/code/nodeStudy $ curl 192.168.10.104:8080
handle by child
wang@wang ~/code/nodeStudy $ curl 192.168.10.104:8080
handled by parent
wang@wang ~/code/nodeStudy $ curl 192.168.10.104:8080
handled by parentThe test result is that each connection to the client may be processed by the parent process or processed by the child process. Now we try to only provide http services, and in order to make the parent process more lightweight, only let the parent process pass the handle to the child process without request processing:
main.js
var cp = require('child_process');
var child1 = cp.fork('./child.js');
var child2 = cp.fork('./child.js');
var child3 = cp.fork('./child.js');
var child4 = cp.fork('./child.js');
var server = require('net').createServer();
//父进程将接收到的请求分发给子进程
server.listen(8080,function(){
child1.send('server',server);
child2.send('server',server);
child3.send('server',server);
child4.send('server',server);
//发送完句柄后关闭监听
server.close();
});
child.js
var http = require('http');
var serverInChild = http.createServer(function(req,res){
res.end('I am child.Id:'+process.pid);
});
//子进程收到父进程传递的句柄(即客户端与服务器的socket连接对象)
process.on('message',function(m,serverInParent){
if(m==='server'){
//处理与客户端的连接
serverInParent.on('connection',function(socket){
//交给http服务来处理
serverInChild.emit('connection',socket);
});
}
});
When you run the above code, you will see the following results when checking the possession of port 8080:
wang@wang ~/code/nodeStudy $ lsof -i:8080
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
node 5120 wang 11u IPv6 44561 0t0 TCP *:http-alt (LISTEN)
node 5126 wang 11u IPv6 44561 0t0 TCP *:http-alt (LISTEN)
node 5127 wang 11u IPv6 44561 0t0 TCP *:http-alt (LISTEN)
node 5133 wang 11u IPv6 44561 0t0 TCP *:http-alt (LISTEN)Run curl to see the result:
wang@wang ~/code/nodeStudy $ curl 192.168.10.104:8080
I am child.Id:5127
wang@wang ~/code/nodeStudy $ curl 192.168.10.104:8080
I am child.Id:5133
wang@wang ~/code/nodeStudy $ curl 192.168.10.104:8080
I am child.Id:5120
wang@wang ~/code/nodeStudy $ curl 192.168.10.104:8080
I am child.Id:5126
wang@wang ~/code/nodeStudy $ curl 192.168.10.104:8080
I am child.Id:5133
wang@wang ~/code/nodeStudy $ curl 192.168.10.104:8080
I am child.Id:5126