[shell] 1. Super detailed introduction to bash syntax

bash manual

Modify prefix

reference

export PS1="bash> "

path

dirname

strip last component from file name

dir=$(dirname "$0")

EXAMPLES
       dirname /usr/bin/
              -> "/usr"

       dirname dir1/str dir2/str
              -> "dir1" followed by "dir2"

       dirname stdio.h
              -> "."

set

After runs bash script.sh, a sub-shell will be created, and the running environment parameters of the sub-shell can be viewed and written through the set command. There are many customizable parameters, see gnu shell manual for details

• Executing set directly will display the set environment variables and shell functions
• set -e: Stop running as soon as an error occurs (that is, a non-zero value is returned)
• set -o pipefail: set -e is invalid for commands containing pipelines. After adding a parameter, any command in the pipeline will stop running if there is an error.
  • If$? (indicating the last execution result) is non-0, it indicates an error. If you want to ignore some errors, you can force the output of the return value through || echo is 0
• set -u: Stop running when encountering undefined variables or methods
• set -x: Before executing each command, output the original text of the command

Commonly used functions

ts=$(date +%s) # 获取日期，如1682072409
echo 'a:b' | cut -d ':' -f 2 # 截取第二项（下标从0开始）
tag=$(cat a.txt) # 命令的结果需要用"${}"包围
uri=www."${tag}".com # 字符串拼接时，需要用$包裹变量

parameter

$0 represents the file name
$1 represents the first parameter
if ["x$1" == "x"] The statement is an equality check, check "x$1" (that is, the variable $1 is preceded by whether an x) is equal to "x". If equal, it means that variable $1 is empty.

The reason for using "x$1" is to prevent problems caused by certain special situations. For example, when the value of $1 is a string starting with "-" (such as "-n"), if "x" is not added, the expression will become "-n" == "", which may is parsed by the shell as a command option rather than a string comparison, resulting in an error.

So, ["x$1" == "x"] statement means to check whether the variable $1 is empty, and this method can avoid errors in some special cases.

variable

local

In shell scripts,local is a built-in command used to declare local variables inside a function. This means that the scope of this variable is limited to the function in which it is declared, and once you leave the function, the variable no longer exists.

If a variable is not declared usinglocal inside the function, then the variable is global and can be accessed throughout the script.

For example, consider the following script:

#!/bin/bash

function test_local() {
    
    
    local local_var="I'm local"
    global_var="I'm global"
    echo $local_var
    echo $global_var
}

test_local
echo $local_var
echo $global_var

The output of running this script will be:

I'm local
I'm global
<空行>
I'm global

As you can see, local_var is invisible outside the function, and global_var is visible throughout the script

return value

• if [ -z $plugin ]; thenWhether the string is 0’

regular

LeetCode193 bash question

Print line n

grep -n "" file.txt | grep -w '10' | cut -d: -f2
sed -n '10p' file.txt
awk '{if(NR==10){print $0}}' file.txt
tail -n +10 file.txt | head -1

Get line number

row_num=$(cat file.txt | wc -l)
echo $row_num
if [ $row_num -lt 10 ];then
    echo "The number of row is less than 10"
else
    awk '{if(NR==10){print $0}}' file.txt
fi

Check the number of tables in the database

# input_table.list
table1
table2
table3
table4
table5

#!/bin/bash
start=$(date -j -f "%Y-%m-%dT%H:%M:%S" "2022-02-21T00:00:00" "+%s")
end=$(date -j -f "%Y-%m-%dT%H:%M:%S" "2022-02-22T00:00:00" "+%s")
start=$(bc -e "$start*1000")
end=$(bc -e "$end*1000")
for i in $(cat input_table.list); do
    echo "$i"
    psql -h192.168.2.163 -p5432 -Upostgres -ddeepface_data -c "select count(1) from $i where ts >= $start and ts < $end"
done

jq

Inquire

{
    
    
    "FOO": {
    
    
        "name": "Donald",
        "location": "Stockholm"
    },
    "BAR": {
    
    
        "name": "Walt",
        "location": "Stockholm"
    },
    "BAZ": {
    
    
        "name": "Jack",
        "location": "Whereever"
    }
}

# cat mytxt | jq .[] | jq ."name"
"Jack"
"Walt"
"Donald"

# jq '.[] | select(.location=="Stockholm")' mytxt
{
    
    
  "name": "Donald",
  "location": "Stockholm"
}
{
    
    
  "name": "Walt",
  "location": "Stockholm"
}

# jq -r .FOO.name mytxt   # 查 json 某 key 对应的 value
Donald

Check logs

#!/bin/bash
# 过滤出.level = error, 但不含某些错误的日志
jq 'select(.level=="error")|select(.msg!="pq: no partition of relation found for row")|select(.msg!="json: cannot unmarshal object into Go struct field")' /logs/myprocess.err

sshpass

#!/bin/bash
set -x
app='a.txt b.json'
nodes=$(cat node.list)
password="qqq"
cd PKG
for i in $nodes; do
    ip=$(echo $i| cut -d':' -f1)
    port=$(echo $i| cut -d':' -f2)
    sshpass -p "$password" scp -o StrictHostKeyChecking=no -P $port $app root@$ip:~/data
done
cd -

expect

expect can enter the command non-interactively, set the following /Users/y/login.exp, and then /Users/y/login.exp myname 192.168.2.99 mypass to ssh without entering the password, and sudo -i does not need to be entered Password, simplify operation.

➜  Desktop cat /Users/y/login.exp
#!/usr/bin/expect

set timeout 3
spawn ssh [lindex $argv 0]@[lindex $argv 1]
expect {
    
    
        "(yes/no)?"
        {
    
    send "yes\n";exp_continue}
        "password:"
        {
    
    send "[lindex $argv 2]\n";exp_continue}
        "from"
        {
    
    send "sudo -i\n";}
        {
    
    send "sudo -i\n";exp_continue}
        "password:"
        {
    
    send "[lindex $argv 2]\n";}
}
interact

array

reference

xargs

In the xargs command, {} is a placeholder that indicates where xargs should be inserted. parameter. When using xargs, you can specify a replacement string with the -I option and then use this string anywhere on the command line, < a i=6> will replace this string with each parameter read from the input. xargs

For example, the following command:

echo '1 2 3' | xargs -I {
    
    } echo "Number: {}"

Will output:

Number: 1
Number: 2
Number: 3

If you use two {} in the same command, xargs will replace them both with the same argument. For example, the following command:

echo '1 2 3' | xargs -I {
    
    } echo "Number: {} and again: {}"

Will output:

Number: 1 and again: 1
Number: 2 and again: 2
Number: 3 and again: 3

In this example, each line contains two {}, xargs will replace them both with the same parameter.

Example: Delete docker containers in batches

docker ps | grep unload | awk '{print $11}' | xargs -I {
    
    } docker rm -f {
    
    }
1. `docker ps`：列出所有运行中的 Docker 容器。
2. `grep unload`：从上述列表中找出含有 "unload" 的行。
3. `awk '{print $1}'`：从找到的行中提取出第一个字段（默认情况下，这是容器的 ID）。
4. `xargs -I {
     
     } docker rm -f {
     
     }`：对每一个找到的容器 ID，执行 `docker rm -f` 命令来强制移除容器。

start up

Usually, when starting