By default, Shell commands in the script is that it must execute the following command to wait before executing a command after serial execution, but if I have a case of a large number of commands to be executed, and another has no effect on the situation (influential if more complicated), then you would use the command executed concurrently.
as follows:
#!/bin/bash IPLIST=/home/meta/ipinfo/iplist for i in $(cat ${IPLIST} |grep -viE "^#|备机|ts"|awk '{print $1}') do ssh $i "cd ~/update/;tar zxf patch-20160909.tgz -C ~/LMDG/ && echo '/$i ok' || echo '/$i bad'" done >> result.txt echo "resutl"|mutt -a result.txt -s update-result meta@126.com
For the above code, because there are a lot of iplist ip, each "tar zxf" hang time-consuming, so I plan to use concurrent programming, so that you can save a lot of time.
amend as below:
#!/bin/bash IPLIST=/home/meta/ipinfo/iplist for i in $(cat ${IPLIST} |grep -viE "^#|备机|ts"|awk '{print $1}') do ssh $i "cd ~/update/;tar zxf patch-20160909.tgz -C ~/LMDG/ && echo '/$i ok' || echo '/$i bad'" & done >> result.txt echo "resutl"|mutt -a result.txt -s update-result meta@126.com
Plus "&" after "tar zxf" can be executed in parallel. In essence, the "tar zxf" as a background process in execution, so that the command will not take up the current bash, other commands do not have to wait for the completion of the previous command to continue, and it can be placed in multiple tasks to the background, so to achieve concurrent multi-task.
I was supposed goal is to "tar zxf" after the end of the loop is executed, and then in front of the "mutt" results. If you write like the above, then, when the "tar zxf" are not over already started "mutt", and get the wrong result, it is necessary to make the following changes:
#!/bin/bash IPLIST=/home/meta/ipinfo/iplist for i in $(cat ${IPLIST} |grep -viE "^#|备机|ts"|awk '{print $1}') do ssh $i "cd ~/update/;tar zxf patch-20160909.tgz -C ~/LMDG/ && echo '/$i ok' || echo '/$i bad'" & done >> result.txt wait echo "resutl"|mutt -a result.txt -s update-result meta@126.com
Here we add a "wait" then we can achieve our desired results, and after the effect of wait is wait for the child to perform tasks are completed at the end of the parent task, and then perform the following tasks.
However, followed by another question, if the amount of this iplist huge, so are breath into the background, the system load is exceeded, there will be the risk of downtime or poor performance, so we need a number of concurrency control mechanism.
Therefore, we introduced the concept of the task queue, socket example is somewhat similar to the consumer before the producer model, to regulate supply and demand imbalance through a message queue
amended as follows:
#!/bin/bash IPLIST = / Home / Meta / ipinfo / IPLIST # tasks (consumers) THREAD = 50 concurrent threads the number of concurrent # statement, this is the key to this application, which is the maximum number of tasks set pipeline TMPFIFO = / tmp / $$. # Declare the FIFO pipe name, ' $$ ' indicates that the process is currently running script PID mkfifo $ TMPFIFO # create the pipe Exec 5 <> $ {#} TMPFIFO create a file identifier " 5 ", this number may be other than " 0 ", " 1 ", " 2 all but declared characters other than" pipe file in read-write mode ; exec system call is a new process to replace the original process, but the PID of the process remains the same, in other words, the implementation of an executable file inside the calling process RM - the RF TMPFIFO $ {} # clean both files created # Create a placeholder for the same number of concurrent threads for((i=1;i<=$THREAD;i++)) do echo ; # Borrow a read command to read the characteristics of a line, using a default output echo a newline, to ensure that only one thread each line placeholder; here is reminiscent of the producer & consumer model, the pipe acts as a message queue file to record consumer demand, and then by the producer to pick up the task and complete the task, here the idea of using asynchronous decoupling. DONE > & 5 # The placeholder information is written to the pipeline for I in $ ( CAT $ IPLIST A {} | grep -viE " ^ # | standby machine | TS " | awk ' {}. 1 Print $ ' ) # sequentially read from the task queue tasks do Read - U5 # File descriptor from the pipe, a pipe thread get placeholder and then began to perform the operation; read the - back with u fd, represents the file descriptor read from the file descriptor may be exec newly opened. { echo $ ( CAT ~ / ipinfo / IPLIST | grep $ I | awk ' {} Print $ 2 ' ); SSH -oConnectTimeout = 10 -oConnectionAttempts = . 3 $ I " CD / Home / the Log /; grep 'MIL' mission_2016-08- * .log 03 | awk -F, '{IF (\ == $ 1370. 19) Print \ $ 0}' | awk -F, ' {IF (\ $ 20 is == 0) Print \ $ 0} ' > miss_info.txt " echo "" > & 5 # task after the implementation written in fd5 in a placeholder, to ensure that after the implementation of this thread, the thread continues to maintain a placeholder, and then maintain the pipeline will always be 50 the number of threads, & represents the portion of the command / task into the background does not account for the current bash, parallel processing } & DONE the wait # wait for the parent of the child process is executed after the end of the end of the parent process Exec . 5 > & - # duct closed fd5 exit 0
----------------
Disclaimer: This article is CSDN blogger "Bai Jinmu field" of the original article, follow the CC 4.0 BY-SA copyright agreement, reproduced, please attach the original source and this link statement.
Original link: https: //blog.csdn.net/qq_34409701/article/details/52488964