Scenario: In daily tasks, sometimes you need to perform some log analysis, queue processing and other tasks through the php script. When the amount of data is relatively large, you can use multiple processes to process.
Preparation: php multi-process requires pcntl, posix extended support, can be viewed through php-m, if not installed, you need to recompile php, plus parameters--enable-pcntl，posix一般默认会有。
note:
Multi-process implementation can only be in cli mode, in the web server environment, there will be unexpected results, I test报错：Call to undefined function: pcntl_fork()
An error pcntl_fork causing "errno = 32 Broken pipe" # 474, see https://github.com/phpredis/phpredis/issues/474
Pay attention to two points: If you create a child process in a loop, then the child process must exit at the end to prevent the child process from entering the loop.
The open connection in the child process cannot be copied, and the main process is used. Multiple case mode is required.
pcntl_fork:
One call returns twice, the child process pid is returned in the parent process, 0 is returned in the child process, and -1 is returned on error.
pcntl_wait ( int &$status [, int $options ] )：
Block the current process until any child process exits or receives a signal to end the current process. Note that it is a signal to end the current process. The SIGCHLD sent by the child process does not count. Use $ status to return the status code of the child process, and you can specify the second parameter to indicate whether to call in blocking state
Called in blocking mode, the return value of the function is the pid of the child process, if there is no child process, the return value is -1;
Called in a non-blocking manner, the function can also return 0 when there are child processes running but not ending.
/ ** Make sure this function can only be run in SHELL * /
if (substr (php_sapi_name (), 0, 3)! == 'cli') {
die ("cli mode only");
}
#!/bin/bash
for((i=1;i<=8;i++))
do
/usr/local/bin/php multiprocessTest.php &
done
wait
The above shell program lists a very simple multi-process program, using a for loop, to achieve 8 processes concurrently to run multiprocessTest.php this program. The last wait statement can also make the main process wait for all processes to execute before executing down.
This program is no problem, and many existing codes are also implemented in this way, but the number of concurrent programs of this program is uncontrollable, that is, we cannot schedule the switch of each process according to the number of cores of the machine.
If our machine has 8 cores or more, the above program is no problem, all cores can be fully utilized, and there is no competition for resources between each other.
But what will happen if our machine does not have 8 cores, the number of processes running at the same time is more than the number of cores, then the system will have the problem of process allocation and scheduling, and the contention for resources will also follow accordingly. A process cannot guarantee independence Continuous execution, all processes will follow the system's scheduling, so that more uncertain factors will appear.
<? php
// Maximum number of child processes
$ maxChildPro = 8;
// Current number of child processes
$ curChildPro = 0;
// This function will be triggered when the child process exits, current number of child processes -1
function sig_handler ($ sig)
{
global $ curChildPro;
switch ($ sig) {
case SIGCHLD:
echo 'SIGCHLD', PHP_EOL;
$ curChildPro--;
break;
}
}
// Used with pcntl_signal, to put it simply, it is for the system to generate time clouds, Let the signal capture function capture the semaphore
declare (ticks = 1);
// The function called when the registered child process exits. SIGCHLD: When a process terminates or stops, it sends a SIGCHLD signal to its parent process.
pcntl_signal (SIGCHLD, "sig_handler");
while (true) {
$ curChildPro ++;
$ pid = pcntl_fork ();
if ($ pid) {
// The parent process runs code, when the upper limit is reached, the parent process blocks and waits for any child process to exit while the loop continues
if ($ curChildPro> = $ maxChildPro) {
pcntl_wait ($ status);
}
} else {
// Child process running code
$ s = rand (2, 6);
sleep ($ s);
echo "child sleep $ s second quit ", PHP_EOL;
exit;
}
}
<? php
$ childs = array ();
// Fork 10 child processes
for ($ i = 0; $ i <10; $ i ++) {
$ pid = pcntl_fork ();
if ($ pid == -1)
die (' Could not fork ');
if ($ pid) {
echo "parent \ n";
$ childs [] = $ pid;
} else {
// Sleep $ i + 1 (s). The child process can get the $ i parameter
sleep ( $ i + 1);
// The child process needs to exit to prevent the child process from entering the for loop
exit ();
}
}
while (count ($ childs)> 0) {
foreach ($ childs as $ key => $ pid) {
$ res = pcntl_waitpid ($ pid, $ status, WNOHANG);
//-1 means error, greater than 0 means the child process has exited, the pid of the child process is returned, and 0 means that the child process has not been exited
if not blocked if ( $ res == -1 || $ res> 0)
unset ( $ childs [$ key]);
}
sleep(1);
}
<?php
function _fetchLog()
{
$password = $this->_getPassword();
$online_log_path = NginxConf::getArchiveDir($this->_stat_day);
$task_log_path = QFrameConfig::getConfig('LOG_PATH');
$children = array();
$success = true;
foreach($this->_server_list as $host => $value)
{
$local_dir = $this->_prepareLocalDir($host);
$task_log = "$task_log_path/fetch_log.$host";
$cmd = "sshpass -p $password rsync -av -e 'ssh -o StrictHostKeyChecking=no' $host:$online_log_path/* $local_dir >> $task_log 2>&1";
$pid = pcntl_fork();
if(-1 === $pid)
{
LogSvc::log('stat_pv_by_citycode_error', 'could not fork');
exit('could not fork');
}
else if(0 === $pid)
{
system($cmd, $return_value);
if(0 !== $return_value)
{
LogSvc::log('stat_pv_by_citycode_error', "rsync $host error");
}
exit($return_value);
}
else
{
$children[$pid] = 1;
}
}
while(!empty($children))
{
$pid = pcntl_waitpid(-1, $status, WNOHANG);
if(0 === $pid)
{
sleep(1);
}
else
{
if(0 !== pcntl_wexitstatus($status))
{
$success = false;
}
unset($children[$pid]);
}
}
return $success;
}
posix_kill(posix_getpid(), SIGHUP); 为自己生成SIGHUP信号
declare(ticks = 1); //php < 5.3
pcntl_signal()
function just registered signal and its processing method, the real received signal and call its treatment method is
pcntl_signal_dispatch()
a function must be called in the cycle, in order to detect whether a new signal wait dispatching.
pcntl_signal_dispatch()
declare(ticks = 1);
means that every time a low-level instruction is executed, the signal is checked once, and if the registered signal is detected, its signal processor is called.
kill [PID]
command, did not add any other parameters, the program will receive a SIGTERM signal.
<? php
// Define a processor that only outputs one line of information after receiving the SIGINT signal
function signalHandler ($ signal)
{
if ($ signal == SIGINT) {
echo 'SIGINT', PHP_EOL;
}
}
// Signal registration: when When the SIGINT signal is received, call the signalHandler () function
pcntl_signal (SIGINT, 'signalHandler');
/ **
* PHP <5.3 Use
* Used with pcntl_signal, which means that the signal is checked every time a low-level instruction is executed. If a registered signal is detected Signal, call its signal processor.
* /
if (! function_exists ("pcntl_signal_dispatch")) {
declare (ticks = 1);
}
while (true) {
$ s = sleep (10);
echo $ s, PHP_EOL; // The signal will wake up sleep and return the rest Seconds.
// do something
for ($ i = 0; $ i <5; $ i ++) {
echo $ i. PHP_EOL;
usleep (100000);
}
/ **
* PHP> = 5.3
* Call the installed signal processor
* Must be called in the loop, in order to detect whether there is a new signal waiting for dispatching.
* /
if (! function_exists ("pcntl_signal_dispatch")) {
pcntl_signal_dispatch ();
}
}
<?php
declare(ticks = 1);
function signal_handler($signal) {
print "Caught SIGALRM\n";
pcntl_alarm(5);
}
pcntl_signal(SIGALRM, "signal_handler", true);
pcntl_alarm(5);
for(;;) {
}
<? php
/ **
* The parent process waits for the child process to exit through pcntl_wait
* The child process kills itself through the signal, or it can send a kil signal in the parent process to end the child process
* /
// Generate the child process
$ pid = pcntl_fork ();
if ($ pid == -1) {
die ('could not fork');
} else {
if ($ pid) {
$ status = 0;
// Block the parent process until the end of the child process, not suitable for long-running Script.
// You can use pcntl_wait ($ status, WNOHANG) to implement non-blocking
pcntl_wait ($ status);
exit;
} else {
// End the current child process to prevent the zombie process from being generated
if (function_exists ("posix_kill")) {
posix_kill (getmypid (), SIGTERM);
} else {
system ('kill -9'. getmypid ());
}
exit;
}
}
(1) The parent process waits for the end of the child process through functions such as wait and waitpid, which causes the parent process to hang. It is not suitable for the case where the child process needs to run for a long time (will cause a timeout).
Executing the wait () or waitpid () system call, the child process will immediately return its data in the process table to the parent process after termination, and the system will immediately delete the entry point. In this case, there will be no defunct process.
(2) If the parent process is busy, you can use the signal function to install the handler for SIGCHLD. After the child process ends, the parent process will receive the signal, and you can call wait in the handler to recycle it.
(3) If the parent process does not care when the child process ends, you can use signal (SIGCLD, SIG_IGN) or signal (SIGCHLD, SIG_IGN) to notify the kernel that you are not interested in the end of the child process, then after the child process ends, the kernel will Recycle and no longer send signals to the parent process
(4) Fork twice, the parent process forks a child process, and then continues to work, the child process forks a grandchild process and then exits, then the grandchild process is taken over by init, after the grandchild process ends, init will be recycled. However, the recycling of the child process must be done by yourself.
ps -A -ostat,ppid,pid,cmd | grep -e '^[Zz]'
Command annotations:
-A parameter lists all processes
-o custom output field We set the display field to stat (status), ppid (process parent id), pid (process id), cmd (command) these four parameters
The process with status z or Z is a zombie process, so we use grep to grab the stat status as zZ process
The results are as follows:
At this time, you can use kill -HUP 5255 杀掉这个进程。如果再次查看僵尸进程还存在，可以kill -HUP 5253（父进程）。
如果有多个僵尸进程，可以通过 ps -A -ostat,ppid,pid,cmd | grep -e '^[Zz]'|awk 'print{$2}'|xargs kill -9
deal with.
Multi-process-inter-process communication (IPC)