Shell's notes (9)

1. To obtain the result, a number between 30000 and 40000 is required

netstat -ntlp|grep -Po '(?<=:)[34]\d{4}'|sort -n

2, grep recursively loop to intercept the desired data

Text content:

{'doc_type':'hw_tencent_malware', 'doc_msg': u'Feb 11 21:12:22 10.3.2.124 tencent_info: {"status": "finished", "final_result": "1" ,"upload_time": "2020-02-11 13:54:22", "shal": "b0c72bfeabfeldf44b577c64a3e30f4e13c307cf", "lost":"1240","extra":"gkml.samr.gov.cn/nsjp/spcjs/201903/W020190307549296342006.docx","src_address":"219.142.64.131","file_name":"W020190307549296342006.docx","dst_address":"106.42.25.139","source":"http","result_score": "0","result": "0","file_ext":"zip","sha256":"0c08190c83edfb7094dd9f38af5fe6325850be728ae3ea44389153de40f2a0f9","result_id": 0,"result_desc":"HEUR.Virus.Cryxos.Ud":"md5":"164bc86caf4a94e1996ae5f2dc06caf"}','total_hit': 81,'window_time':'logdate-2020.01~logdate-2020.06','search time': 0.012002000000000068, 'match':'not found','MD5':'164bc86cacf4a94e1996ae5f2dc06caf'}

{'doc_type':'hw_tencent_malware','window_time':'logdate-2020.01~logdate-2020.06','search time': 0.001711000000000018, 'match':'not found','MD5':'f95772ff96a93980eb7b293e01808c7a'}


上边是日志里面两种样式，下面是取值字段
'doc_type':'hw_tencent_malware'
"status": "finished"
"final_result": "1"
"upload_time": "2020-02-11 13:54:22"
"result_score": "0"
"result": "0"
"result_id": 0
"result_desc":"HEUR.Virus.Cryxos.Ud"
'total_hit': 81
'match':'found'
'MD5':'164bc86cacf4a94e1996ae5f2dc06caf'}

Written by the big guy: I write it in according to the requirements, what is needed, what to write in the candidate group (but there is a disadvantage, there may be more backtracking, which affects efficiency)

 grep -Po '([\x27"])(doc_type|status|final_result|upload_time|result[^"]*|total_hit|match|MD5)\1:\s*(\1)?[^\x27",]+(\1)?' test.txt

3. Get the updated content of the latest version number from questions on the Internet

Text content:

V0.1-张三-2020/11/4 14：00
增加AAAA
V0.2-张三-2020/11/4 12：00
调整子工作流流程（由并行改为串行）
V0.4-张三-2020/11/4 12：00
去掉发邮件步骤
调整流程（由并行改为串行）
V0.3-张三-2020/11/4 12：00
去掉发邮件步骤
调整流程（由并行改为串行）
V13.9-张三-2020/11/4 14：00
增加AAAA
V4.2-张三-2020/11/4 12：00
调整子工作流流程（由并行改为串行）
V5.4-张三-2020/11/4 12：00
去掉发邮件步骤
调整流程（由并行改为串行）
V116.3-张三-2020/11/4 12：00
去掉发邮件步骤
调整流程（由并行改为串行）	

Method Zero
Initially, I used the decimal array to solve the problem. The decimal point comparison in the shell caused a lot of pitfalls along the way. Final code
Decimal comparison reference blog https://blog.51cto.com/tenderrain/1916029

#!/bin/bash
path=/root/shellDir/program/awkFile

array=(`grep -oP '(?<=V).*?(?=-)' $path/logFile.txt`)
echo ${array[*]}

MAX=${array[0]}

for I in ${!array[@]};do
	#a=`echo "$nu > 100.0"|bc`
	#if [ `expr $a \> $b` -eq 0 ];then
	#echo "max --> ${MAX}"
	#echo "array--> ${array[${I}]}"
       #if [ $(echo "$max < $min"|bc) -eq 1 ]  博客方法
       # if [  `expr ${MAX} \> ${array[${I}]}` -eq 0 ];then  有BUG 只是对比第一位数字
        if [  $(echo "${MAX} > ${array[${I}]}"|bc) -eq 0 ];then
	    echo -e "max change...\n"
            MAX=${array[${I}]}
        fi
done

#echo "final ---》  ${MAX}"

grep -zPo "(?s)V"$MAX"[^V]+(?=\n)" $path/logFile.txt

method one

#方法一  将版本号最大值赋值给maxV , 然后用grep筛选出
#!/bin/bash
path=/root/shellDir/program/awkFile

maxV=`grep -oP '(?<=V).*?(?=-)' $path/logFile.txt | sort -nr | head -n 1`
#(?s)即Singleline(单行模式)。表示更改.的含义，使它与每一个字符匹配（包括换行 符\n）。  （）
grep -zPo "(?s)V"$maxV"[^V]+(?=\n)" $path/logFile.txt

Method Two

 awk 'BEGIN{RS="V0"} /^\.4/{printf RS $0}' logFile.txt

awk 'BEGIN{RS="V0.";FS="-"}{if($1>m){a[m=$1]=$0}}END{printf RS a[m]}' logFile.txt
#龙帅的方法
awk 'BEGIN{RS="V";FS="-"}{if($1>m){a[m=$1]=$0}}END{printf RS a[m]}' logFile.txt

Usage of awk RS: https://www.cnblogs.com/xuaijun/p/7902757.html

#1
awk -F 'V|-' '/^V/{c=$2;a=a>$2?a:$2}{b[c]=$0~/^V/?$0:b[c]"\n"$0}END{print b[a]}' fileName

#2    下面perl方法适用范围，个位数的日志。如果是 有V12.3 十位数的，那就不准了。
perl -0nE'say+(sort split/\n(?=^V)/m)[-1]'

4. How to create txt texts in batches based on the following content, for example: 1.txt content is aaa, 2.txt content is bbb, 3.txt content is ccc

1.txt
aaa

2.txt
bbb

3.txt
ccc

method one:

#!/bin/bash
#grep分别取出文件名和内容 ， 用两个for进行遍历创文件 和 追加文本内容
path=/root/shellDir/grepTest
fileName=($(grep -oP ".*txt" $path/test.txt.bak))
content=($(grep -v "^$" test.txt.bak | grep -v "txt"))
#echo ${content[*]}
count=1
for (( i = 0; i < ${#fileName[@]}; ++i ));
do
	touch $path/${fileName[i]}
        count=1

        for (( j = 0; j < ${#fileName[@]}; ++j ));
        do
                if [ $count==$i  ];then
#echo "--》 ${#content[i]}"
#echo "===》 ${content[2]}"
cat >> $path/${fileName[i]}<<EOF
${content[i]}
EOF
        break
                fi
        done
        let count++
done

Method Two:

#  /2.txt/{n;p}   打印每个.txt的下一行，然后写到文件

Method 3: More flexible, suitable for multi-line text

awk '{if($0 ~ /.*txt/){filename = $0 ;flag = 0 } else if ($0 ~ /^$/){flag = 0} else{flag = 1}}flag{print $0>filename}' test.file

Method Four: Written by Perl

perl -lne'open f,">",$_ if/\d/;print f $_' file

5. How sed inserts content in the second line under the specified line, i and a can only be inserted before and after the matching line

101,John Doe,CEO
102,Jason Smith,IT Manager
103,RajReddy , Sysadminn
104,and Ram ,Developer
105,Jane Miller ,Sales Manager

Need to use sed pattern space

sed -r '/102/{n;s#(.*)#\1\n插入的内容#}' sedSpacePattern.txt

If you change the requirement to "insert content in the third (fourth) line under the specified line", pay attention to the number of n;

sed -r '/102/{n;n;s#(.*)#\1\n插入的内容#}' sedSpacePattern.txt

sed -r '/102/{n;n;n;s#(.*)#\1\n插入的内容#}' sedSpacePattern.txt

Reference: http://blog.chinaunix.net/uid-7530389-id-2050047.html

6. How to compare the size of the version in the shell script

	ver1=3.6.8
	ver2=3.6.9

1. sed replaces. With empty, and then compares

2. Expr direct comparison