Scenario Description:
· Shooting scene, first-in, first-out mode
Order:
rpush + blpop or lpush + brpop
rpush : push data to the right of the list
blpop : the client blocks until the queue has a value to output
Simple queue:
simple.php
$stmt = $pdo->prepare('select id, cid, name from zc_goods limit 200000');$stmt->execute();while ($row = $stmt->fetch(PDO::FETCH_ASSOC)) { $redis->rPush('goods:task', json_encode($row));} $redis->close();
Get 200 million items and push the jsonized data into the goods:task queue
queueBlpop.php
// dequeue while ( true ) { // block timeout set to 3 seconds $task = $redis->blPop( array ( 'goods:task' ), 3 ); if ($task) { $redis->rPush ( 'goods:success:task' , $task[ 1 ]); $task = json_decode($task[ 1 ], true ); echo $task[ 'id' ] . ':' . $task[ 'cid' ] . ':' . 'handle success' ; echo PHP_EOL; } else { echo 'nothing' . PHP_EOL; sleep(5); }}
Set the blpop blocking time to 3 seconds. When there is data out of the queue, save it to goods:success:task to indicate successful execution. When there is no data in the queue, the program sleeps for 10 seconds and rechecks whether there is data out of the goods:task.
Execute the command in cli mode:
php simple.phpphp queueBlpop.php
priority queue
Ideas:
When blpop has multiple keys, blpop traverses the keys from left to right, and the client returns as soon as a key pops an element. E.g:
blpop key1 key2 key3 key4
Traverse from key1 to key4. If any key has a value, the value will be popped up. If multiple keys have values at the same time, the key on the left will be popped first.
priority.php
// set priority queue $high = 'goods:high:task' ;
$mid = 'goods:mid:task';$low = 'goods:low:task';
$stmt = $pdo->prepare('select id, cid, name from zc_goods limit 200000');
$stmt->execute();while ($row = $stmt->fetch(PDO::FETCH_ASSOC))
{ // cid less than 100 put in low-level queue if ($row[ 'cid' ] < 100 )
{ $redis->rPush($low, json_encode($row));
} // put between cid 100 and 600 in the intermediate queue elseif ($row[ 'cid' ] > 100 && $row[ 'cid' ] < 600 )
{ $redis->rPush($mid, json_encode($row)); }
// cid greater than 600 is placed in the advanced queue else { $redis->rPush($high, json_encode($row)); }
}$redis->close();
priorityBlop.php
// priority queue $high = 'goods:high:task' ;$mid = 'goods:mid:task' ;$low = 'goods:low:task' ; // dequeue while ( true ){ // priority The high-level queue is placed on the left $task = $redis->blPop( array ($high, $mid, $low), 3 ); if ($task) { $task = json_decode($task[ 1 ], true ); echo $task[ 'id' ] . ':' . $task[ 'cid' ] . ':' . 'handle success' ; echo PHP_EOL; } else { echo 'nothing' . PHP_EOL; sleep(5); }}
The queue with high priority is placed on the left side of the blpop command and sorted in order. The blpop command will pop up the values of the high, mid, and low queues in turn.
Execute the command in cli mode:
php priority.phpphp priorityBlpop.php
delay queue
Ideas:
You can use an ordered set to save delayed tasks, member to save the task content, and score to save (current time + delay time). Use time as score. The program only needs to compare the score of the first task of the sorted set with the current time. If the current time is smaller than the score, it means that all the tasks of the sorted set have not yet reached the execution time.
delay.php
$stmt = $pdo->prepare('select id, cid, name from zc_goods limit 200000');$stmt->execute();while ($row = $stmt->fetch(PDO::FETCH_ASSOC)) { $redis->zAdd('goods:delay:task', time() + rand(1, 300), json_encode($row));}
Import 200,000 tasks into the ordered collection goods:delay:task, and delay all tasks to execute within 1 second to 300 seconds afterward
delayHandle.php
while ( true ) { // Because it is an ordered set, as long as the delay time of the first record is judged, for example, the first record has not yet reached the execution time // Relatively indicates that the other tasks of the set have not yet reached the execution time
$rs = $redis->zRange('goods:delay:task', 0, 0, true);
// The collection has no tasks, the sleep time is set to 5 seconds
if ( empty ($rs)) {
echo 'no tasks , sleep 5 seconds' . PHP_EOL;sleep( 5 ); continue ;}
$taskJson = key($rs);
$delay = $rs[$taskJson];
$task = json_decode($taskJson, true );
$now = time(); // Execute delayed task until time
if ($delay <= $now) {
// Lock the current task to avoid being modified by other clients when moving the delayed task to the task queue
if (!($identifier = acquireLock($task['id']))) {
continue;}
// Move the delayed task to the task queue
$redis->zRem('goods:delay:task', $taskJson);
$redis->rPush('goods:task', $taskJson);
echo $task['id'] . ' run ' . PHP_EOL;
// release the lock
releaseLock($task['id'], $identifier); }
else {
// The delayed task has not reached the execution time
$sleep = $delay - $now;
// The maximum value is set to 2 seconds to ensure that if a new task (delay time of 1 second) enters the collection, it can be processed in time
$sleep = $sleep > 2 ? 2 :$sleep;
echo 'wait ' . $sleep . ' seconds ' . PHP_EOL; sleep($sleep);
}
}
This file processes the delayed tasks in the ordered set. If the delayed task reaches the execution time, the delayed task is moved to the task queue.
queueBlpop.php
// 出队while (true) {
// 阻塞设置超时时间为3秒
$task = $redis->blPop(array('goods:task'), 3);
if ($task) {
$redis->rPush('goods:success:task', $task[1]);
$task = json_decode($task[1], true);
echo $task['id'] . ':' . $task['cid'] . ':' . 'handle success';
echo PHP_EOL; } else {
echo 'nothing' . PHP_EOL;sleep(5);
}
}
Process tasks in the task queue
Execute the command in cli mode:
php delay.phpphp delayHanlde.phpphp queueBlpop.php