shell script multi-threaded implementation

shell script multithreading
in the use of shell scripts, ran too slow implementation of the order, so consider using multiple threads to improve efficiency.
This is done using the & + wait achieve multithreading, see the following comparison.

  1. Execution order
    execution three times no time to wait for 4 seconds, consuming a total of 12 seconds
    Code:
    # / bin / bash
    DATE
    for NUM in seq 1 3
    do
    {
    echo "SLEEP 4"
    SLEEP 4
    }
    DONE
    DATE
    RESULTS:
    November 20, 2019 Wednesday 09: CST 40:56
    SLEEP 4
    SLEEP 4
    SLEEP 4
    2019 Nian 11 Wednesday, 20 September 09:41:08 CST

  2. Use & + wait
    parallel execution, takes a total of 4 seconds
    Code:
    # / bin / the bash
    DATE
    for in NUM seq 1 3
    do
    {
    echo "SLEEP. 4"
    SLEEP. 4
    } &
    DONE
    the wait
    DATE
    Results:
    November 20, 2019 Wednesday, 09:42: CST 44
    SLEEP 4
    SLEEP 4
    SLEEP 4
    2019 Nian 11 Wednesday, 20 September 09:42:48 CST

According to observation can be found & + wait cycle will form the body of all the command into the background, so that when there is a problem when dealing with a large number of transactions, it is possible to achieve control of the number of threads each open according to the following ways

  1. The multilayer loop for
    a total of nine times, three threads send, wait for 4 seconds, totaling 12 seconds takes
    Code:
    # / bin / the bash
    DATE
    for num2 in seq 1 3
    do
    {
    for in NUM seq 1 3
    do
    {
    echo "SLEEP. 4"
    SLEEP. 4
    } &
    DONE
    the wait
    }
    DONE
    DATE
    rESULTS:
    November 20, 2019 CST Wednesday 09:47:23
    SLEEP 4
    SLEEP 4
    SLEEP 4
    SLEEP 4
    SLEEP 4
    SLEEP 4
    SLEEP 4
    SLEEP 4
    SLEEP 4
    November 20, 2019 09:47:35 CST Wednesday

  2. Xargs -P concurrent use of
    a total of nine times, three threads send, wait for 4 seconds, totaling 12 seconds takes
    Code:
    # / bin / the bash

all_num=9
thread_num=3

date

seq 1 ${all_num} | xargs -n 1 -I {} -P ${thread_num} sh -c "sleep 4;echo {}"

date
results:
November 20, 2019 CST Wednesday 09:51:03
1
3
2
4
5
6
8
9
7
20 November 2019 09:51:15 CST Wednesday

These are the shell parallel my summary. There use GNU parallel command concurrency way, because the test environment does not change the order, it did not do the test, will be tested after completion environment.

Guess you like

Origin blog.51cto.com/12120912/2451750