Shell script is a program written in Shell language, which can realize automated tasks, such as batch processing files, monitoring system status, and scheduled backup. This article includes:

The definition and function of Shell script: introduce what is Shell script, what are its advantages and disadvantages, and what can it be used for.

The basic grammar of Shell script: introduce the basic elements such as the structure, variables, operators, flow control, and functions of Shell script, and give some simple examples.

Advanced Features of Shell Scripts: Introduce advanced features such as parameter passing, arrays, string processing, regular expressions, file operations, and process management of Shell scripts, and give some complex examples.

Application scenarios of Shell scripts: Introduce some common application scenarios of Shell scripts, such as log analysis, data backup, system monitoring, etc., and give some practical cases.

1. Shell script and shell

Shell is a command line interpreter, an interface for users to interact with the operating system and execute commands.

Shell script is a program written using the functions of Shell. This program uses plain text files, and writes some shell syntax and commands (including external commands) in it, with functions such as regular expressions, pipeline commands, and data flow redirection. (Simply speaking, it is a series of shell commands + control structures)

Shell scripts can automate tasks, such as batch processing files, monitoring system status, and scheduled backups.

The first line of the shell script is usually the path of the specified interpreter, such as #!/bin/bash, which means that Bash is used to execute the script. Shell scripts usually have a .sh extension, but they don't have to. To run a shell script, you need to give it executable permissions, such as chmod +x test.sh, and then use ./test.sh or /bin/bash test.sh to execute.

There are many types of Shell, the common ones are as follows:

Bourne Shell (sh) : Developed by Steve Bourne of AT&T, it is the standard shell on Unix and the basis for the development of other shells. It is excellent in programming, but not as good as other shells in user interaction.
Bourne Again Shell ( bash) : Developed by the GNU organization, it is the default Shell in the Linux system. It maintains compatibility with sh while inheriting the advantages of csh and ksh. It is currently the most popular shell .
C Shell (csh) : Developed by Bill Joy of the University of California, Berkeley, its syntax is similar to the C language. It provides some user interaction features that sh cannot handle, such as command completion, command aliases, historical command substitution, etc. But it is not compatible with sh.
Korn Shell (ksh) : Developed by David Korn of AT&T Bell Labs, it combines the advantages of csh and sh and is fully compatible with sh. It's efficient, and its command interface and programming interface are both good.
Z Shell (zsh) : It is one of the largest shells in Linux, completed by Paul Falstad, with a total of 84 internal commands. It is compatible with bash and provides more functions and extensions.

Second, the basic syntax of Shell script

The basic structure of a shell script is as follows:

Declare the interpreter: The first line of every shell script should specify the interpreter. On Unix systems, the commonly used interpreter is bash (Bourne Again SHell). Therefore, the following statement can be added to the first line of the script: #!/bin/bash
Comments: Comments can be used in Shell scripts to provide explanations and instructions for the script. Comments begin with a pound sign (#) and can appear on a line by themselves or at the end of a line of code.
Variables: Variables are used in shell scripts to store and manipulate data. Variable names must start with a letter or underscore and can be followed by letters, numbers or underscores. Use the equal sign (=) for variable assignment, such as: variable_name=value. When using a variable, you need to add a dollar sign ( ) before the variable name $），如：$ variable_name。
Command Execution: Various system commands can be executed in Shell scripts. To execute a command, simply enter the command in the script. For example, to list the files in the current directory, you can use the ls command: ls.
Input Output: Shell scripts can receive input from the user and display output to the user. Use the read command to get input from the user, and use the echo command to send output to standard output.
Conditional Statements: Conditional statements are used to execute different blocks of code depending on the outcome of the condition. Commonly used conditional statements are if statement and case statement. The if statement executes or skips the code block according to the condition, and the case statement executes different code blocks according to different pattern matching.
Loops: Loops in shell scripts are used to repeatedly execute a piece of code. Commonly used loops are for loop, while loop and until loop. for loops are used to iterate over a list or range, while loops and until loops execute blocks of code while a condition is true or false.
Function: Shell script supports function definition and call. Functions can help organize and reuse code. To define a function, use the keyword function, and to call a function, you only need to use the function name.
Parameter passing: You can pass parameters to scripts or functions in shell scripts. Script parameters are represented using special variables $1, $2, etc. Function parameters are used in a similar way.
File Handling: Shell scripts can handle files and directories. Various commands can be used to read, write, and manipulate files. Commonly used file processing commands include cat, grep, sed, and awk.
Error Handling: Shell scripts can catch and handle errors. You can use conditional statements and command return values to check whether a command executed successfully and take appropriate action.

The following is a detailed introduction to some of the above contents.

2.1 Script header

Every shell script should start with #!/bin/bashor #!/bin/sh, which is called the script header (shebang), which tells the system which shell to use to execute the script. Bash and sh are two common shells that have some minor differences, but are mostly compatible.

2.2 Notes

A comment is text used to explain the meaning or function of the code, it will not be executed. In a shell script, a comment #begins with , and can be on a single line, or it can follow a command. For example:

# 这是一个注释
echo "Hello, world!" # 这也是一个注释

2.3 Variables

Variables are identifiers used to store data, which can be numbers, strings, arrays, etc. In the shell script, the definition and use of variables do not need to add any symbols, as long as there is no space between the variable name and the assignment symbol (=). For example:

name="Alice" # 定义一个变量name，赋值为Alice
echo $name # 使用变量name，需要加$符号

Variable names can consist of letters, numbers, and underscores, but cannot begin with a number. Variable names are case-sensitive, which means name and Name are two different variables.

2.4 parameters

Parameters are data passed to a script or function, which can be accessed using special variables. In Shell scripts, the special variables for parameters are as follows:

$0: Indicates the file name of the script
$1~ $9: Indicates the 1st to 9th parameters
${10}~ ${n}: Indicates the 10th to nth parameters, need to add curly braces
$*: Indicates all parameters, as a whole
$@: Represents all parameters as an array
$#: Indicates the number of parameters

For example:

# 假设这个脚本叫做test.sh，并且执行时传递了三个参数a b c
echo $0 # 输出test.sh
echo $1 # 输出a
echo $2 # 输出b
echo $3 # 输出c
echo ${10} # 输出空白，因为没有第10个参数
echo $* # 输出a b c
echo $@ # 输出a b c
echo $# # 输出3

2.5 Commands

A command refers to an instruction to perform a certain operation, and it can be a system built-in command or a user-defined command. In shell scripts, commands are usually separated by newlines or semicolons (;). For example:

ls # 列出当前目录下的文件和文件夹
pwd; date # 显示当前目录和当前日期

Commands can accept arguments and options to modify their behavior. Parameters usually follow the command, and options usually start with a dash (-) and can have one or more letters. For example:

ls -l # 以长格式列出当前目录下的文件和文件夹
cp -r source destination # 复制source目录及其子目录和文件到destination目录

The execution result of a command can be captured with backticks (`) or dollar signs and parentheses ($()), and then assigned to a variable or used as an argument to another command. For example:

files=`ls` # 把ls命令的结果赋值给变量files
echo $files # 输出变量files的值
today=$(date +%Y%m%d) # 把date命令的结果赋值给变量today，使用+%Y%m%d选项来指定日期格式
echo $today # 输出变量today的值

Ok, here's the second part:

2.6 Process Control

Flow control refers to controlling the execution order of code based on conditions or loops. In Shell scripts, the common flow control statements are as follows:

if...then...else...fi: Execute different code blocks according to the condition. The condition can be expressed by the test command or square brackets ([ ]), and it needs to end with fi. For example:

# 判断参数是否为0
if [ $1 -eq 0 ]; then
  echo "Zero"
elif [ $1 -gt 0 ]; then # elif表示否则如果，可以有多个
  echo "Positive"
else # else表示否则，只能有一个
  echo "Negative"
fi # 结束if语句

case...in...esac: Execute different code blocks according to the value of the parameter. Each code block ends with a right parenthesis ()) and needs to end with esac. For example:

# 根据参数的值输出不同的信息
case $1 in
  start) # 如果参数为start
    echo "Starting..."
    ;; # 双分号表示结束这个代码块
  stop) # 如果参数为stop
    echo "Stopping..."
    ;;
  restart) # 如果参数为restart
    echo "Restarting..."
    ;;
  *) # 如果参数为其他值，*表示匹配任意值
    echo "Invalid argument"
    ;;
esac # 结束case语句

for...do...done: Loop a piece of code according to a list or a range, and need to end with done. For example:

# 遍历一个列表并输出每个元素
for fruit in apple banana orange; do
  echo $fruit
done

# 遍历一个范围并输出每个数字，{1..10}表示从1到10，步长为1
for i in {
    
    1..10}; do
  echo $i
done

# 遍历一个范围并输出每个数字，{1..10..2}表示从1到10，步长为2
for i in {
    
    1..10..2}; do
  echo $i
done

# 遍历一个数组并输出每个元素，数组用小括号（()）来定义，用美元符号和大括号（${}）来引用
array=(red green blue)
for color in ${array[@]}; do
  echo $color
done

# 遍历一个文件中的每一行并输出每一行，使用read命令来读取文件内容，使用<符号来重定向文件输入
while read line; do
  echo $line
done < file.txt

# 遍历一个命令的输出结果并输出每一行，使用read命令来读取命令输出，使用<符号和反引号（`）或美元符号和圆括号（$()）来重定向命令输出
while read line; do
  echo $line
done < `ls`

Ok, here's the 7. function and its rest:

2.7 Functions

A function is a reusable piece of code that accepts parameters and returns a value. In the shell script, there is no need to add any symbols to the definition and call of the function, as long as there is no space between the function name and the curly braces ({}). For example:

# 定义一个函数，名为greet，接受一个参数，表示名字
greet() {
    
    
  echo "Hello, $1!" # 输出问候语，$1表示第一个参数
}

# 调用函数，传递一个参数，表示名字
greet Bob # 输出Hello, Bob!
greet Alice # 输出Hello, Alice!

Functions can return a numeric value that represents the status or result of the execution. The return value can be specified by the return command, or it can be expressed by the execution result of the last command of the function. Return values can be obtained using special variables $?. For example:

# 定义一个函数，名为add，接受两个参数，表示两个数字
add() {
    
    
  return $(($1 + $2)) # 返回两个参数的和，使用$(( ))来进行算术运算
}

# 调用函数，传递两个参数，表示两个数字
add 3 5 # 返回8
echo $? # 输出8

add 10 20 # 返回30
echo $? # 输出30

Functions can be defined and called nestedly, that is, one function can define and call another function. For example:

# 定义一个函数，名为square，接受一个参数，表示一个数字
square() {
    
    
  return $(($1 * $1)) # 返回参数的平方
}

# 定义一个函数，名为sum_of_squares，接受两个参数，表示两个数字
sum_of_squares() {
    
    
  square $1 # 调用square函数，传递第一个参数
  a=$? # 把返回值赋值给变量a
  square $2 # 调用square函数，传递第二个参数
  b=$? # 把返回值赋值给变量b
  return $(($a + $b)) # 返回两个平方的和
}

# 调用函数，传递两个参数，表示两个数字
sum_of_squares 3 4 # 返回25
echo $? # 输出25

sum_of_squares 5 6 # 返回61
echo $? # 输出61

2.8 Script example

#!/bin/bash

# 这是一个示例Shell脚本

# 变量示例
name="John"
age=25

# 输出变量
echo "My name is $name and I am $age years old."

# 输入示例
echo "Please enter your name:"
read input_name
echo "Hello, $input_name!"

# 条件语句示例
if [ $age -ge 18 ]; then
    echo "You are an adult."
else
    echo "You are a minor."
fi

# 循环示例
echo "Counting from 1 to 5:"
for ((i=1; i<=5; i++)); do
    echo $i
done

# 函数示例
say_hello() {
    
    
    echo "Hello, $1!"
}

say_hello "Alice"
say_hello "Bob"

# 参数传递示例
echo "Script name: $0"
echo "First argument: $1"
echo "Second argument: $2"

# 文件处理示例
echo "Contents of the current directory:"
ls

echo "Searching for 'example' in files:"
grep -r "example" .

implement:

insert image description here

3. Advanced Features of Shell Script

In addition to the basic syntax, Shell scripts also have some advanced features that can make your scripts more powerful, flexible and easy to use.

3.1 Arrays

An array is a variable that stores multiple values. It can be defined with parentheses ( ) and referenced ()with dollar signs and braces ( ). ${}Each element in the array has an index, starting from 0. For example:

# 定义一个数组，名为colors，包含三个元素
colors=(red green blue)

# 引用数组中的第一个元素，索引为0
echo ${colors[0]} # 输出red

# 引用数组中的最后一个元素，索引为-1
echo ${colors[-1]} # 输出blue

# 引用数组中的所有元素，使用@或*符号
echo ${colors[@]} # 输出red green blue
echo ${colors[*]} # 输出red green blue

# 引用数组中的部分元素，使用:和数字来指定范围，格式为${array[@]:start:length}
echo ${colors[@]:0:2} # 输出red green
echo ${colors[@]:1:2} # 输出green blue

# 获取数组的长度，使用#符号
echo ${
    
    #colors[@]} # 输出3

Arrays can be modified, elements can be added or removed, and they can be concatenated or cut. For example:

# 修改数组中的第二个元素，索引为1
colors[1]="yellow" # 把green改为yellow
echo ${colors[@]} # 输出red yellow blue

# 添加元素到数组的末尾，使用+=运算符
colors+=("black" "white") # 添加black和white到数组末尾
echo ${colors[@]} # 输出red yellow blue black white

# 删除数组中的第三个元素，索引为2
unset colors[2] # 删除blue
echo ${colors[@]} # 输出red yellow black white

# 拼接两个数组，使用+=运算符
numbers=(1 2 3)
colors+=(${numbers[@]}) # 把numbers数组拼接到colors数组后面
echo ${colors[@]} # 输出red yellow black white 1 2 3

# 切割一个数组，使用:和数字来指定范围，格式为${array[@]:start:length}
first_three=(${colors[@]:0:3}) # 切割出前三个元素
echo ${first_three[@]} # 输出red yellow black
last_three=(${colors[@]:-3:3}) # 切割出后三个元素
echo ${last_three[@]} # 输出1 2 3

3.2 Associative arrays

An associative array is a variable that stores key-value pairs, which can be defined using commands, referenced declare -Awith dollar signs and braces ( ). ${}Each element in an associative array has a key and a value, and the key can be any string. For example:

# 定义一个关联数组，名为users，包含三个键值对
declare -A users
users=([Alice]=20 [Bob]=25 [Cathy]=30)

# 引用关联数组中的某个值，使用键名
echo ${users[Alice]} # 输出20

# 引用关联数组中的所有键，使用!符号
echo ${
    
    !users[@]} # 输出Alice Bob Cathy

# 引用关联数组中的所有值，使用@或*符号
echo ${users[@]} # 输出20 25 30
echo ${users[*]} # 输出20 25 30

# 获取关联数组的长度，使用#符号
echo ${
    
    #users[@]} # 输出3

Associative arrays can modify, add or remove elements, and can also be traversed or sorted. For example:

# 修改关联数组中的某个值，使用键名
users[Alice]=21 # 把Alice的值改为21
echo ${users[Alice]} # 输出21

# 添加元素到关联数组中，使用键名和赋值符号（=）
users[David]=35 # 添加David和35到关联数组中
echo ${users[David]} # 输出35

# 删除关联数组中的某个元素，使用unset命令和键名
unset users[Bob] # 删除Bob和其对应的值
echo ${
    
    !users[@]} # 输出Alice Cathy David

# 遍历关联数组中的所有元素，使用for循环和!符号
for name in ${
    
    !users[@]}; do # 遍历所有的键名
  echo "$name is ${users[$name]} years old" # 输出键名和对应的值
done

# 排序关联数组中的所有元素，使用sort命令和管道符号（|）
echo ${
    
    !users[@]} | sort # 按照键名排序并输出
echo ${users[@]} | sort -n # 按照值排序并输出，使用-n选项表示按照数字排序

3.3 String manipulation

A string is a variable that stores text. It can be defined with double quotes (") or single quotes ('), and quoted with dollar signs and braces (${}). Strings can be manipulated, such as splicing , intercept, replace, match, etc. For example:

# 定义一个字符串，名为greeting，赋值为Hello, world!
greeting="Hello, world!"

# 引用字符串的值，不需要加任何符号
echo $greeting # 输出Hello, world!

# 拼接字符串，使用+运算符或直接相连
name="Alice"
message=$greeting+$name # 使用+运算符拼接字符串
echo $message # 输出Hello, world!+Alice
message=$greeting$name # 直接相连拼接字符串
echo $message # 输出Hello, world!Alice

# 截取字符串，使用:和数字来指定范围，格式为${string:start:length}
echo ${greeting:0:5} # 截取前五个字符并输出，输出Hello
echo ${greeting:7:5} # 截取第七个到第十一个字符并输出，输出world
echo ${greeting: -1} # 截取最后一个字符并输出，输出!

# 替换字符串，使用/和模式来指定替换规则，格式为${string/pattern/replacement}
echo ${greeting/world/China} # 把world替换为China并输出，输出Hello, China!
echo ${greeting//o/O} # 把所有的o替换为O并输出，输出HellO, wOrld!

# 匹配字符串，使用#或%和模式来指定匹配规则，格式为${string#pattern}或${string%pattern}
file="test.sh"
echo ${file#*.} # 从左边开始匹配第一个.及其左边的内容，并删除，输出sh
echo ${file##*.} # 从左边开始匹配最后一个.及其左边的内容，并删除，输出sh
echo ${file%.*} # 从右边开始匹配第一个.及其右边的内容，并删除，输出test
echo ${file%%.*} # 从右边开始匹配最后一个.及其右边的内容，并删除，输出test

3.4 Regular expressions

Regular expressions are a grammar used to describe text patterns, which can be used to perform operations such as string matching, replacement, and segmentation. In shell scripts, regular expressions can be expressed with square brackets ( [[ ]]) or =~operators. Regular expressions have some special symbols, such as:

^: indicates the beginning of the match
$: Indicates the end of the match
.: means match any character
*: means match zero or more preceding characters
+: Indicates to match one or more preceding characters
?: means match zero or one preceding character
[]: means to match any character in square brackets
[^]: means to match any character other than those in square brackets
(): Indicates grouping
|: means or
\: Indicates escape

For example:

# 定义一个字符串，名为text，赋值为This is a test.
text="This is a test."

# 使用双方括号和正则表达式来判断字符串是否匹配某个模式，如果匹配，返回0，否则返回1
[[ $text =~ ^This ]] # 判断字符串是否以This开头，返回0
[[ $text =~ test$ ]] # 判断字符串是否以test结尾，返回0
[[ $text =~ [a-z]+ ]] # 判断字符串是否包含一个或多个小写字母，返回0
[[ $text =~ [0-9]+ ]] # 判断字符串是否包含一个或多个数字，返回1

# 使用=~运算符和正则表达式来判断字符串是否匹配某个模式，如果匹配，返回0，否则返回1
[[ $text =~ "is" ]] # 判断字符串是否包含is，返回0
[[ $text =~ "is$" ]] # 判断字符串是否以is结尾，返回1

# 使用=~运算符和正则表达式来提取字符串中的某个部分，使用BASH_REMATCH变量来获取匹配结果
[[ $text =~ ([a-z]+)\. ]] # 判断字符串是否包含一个或多个小写字母后跟一个点，并提取小写字母部分
echo ${
    
    BASH_REMATCH[0]} # 输出test.
echo ${
    
    BASH_REMATCH[1]} # 输出test

# 使用sed命令和正则表达式来替换字符串中的某个部分，使用s命令和/符号来指定替换规则，格式为sed 's/pattern/replacement/'
echo $text | sed 's/test/example/' # 把test替换为example并输出，输出This is a example.
echo $text | sed 's/[a-z]/X/g' # 把所有的小写字母替换为X并输出，输出XXXX XX X XXXX.

# 使用grep命令和正则表达式来搜索文件中的某个模式，使用-E选项来启用扩展的正则表达式
grep -E '^[A-Z]' file.txt # 搜索文件中以大写字母开头的行并输出
grep -E '[0-9]+$' file.txt # 搜索文件中以数字结尾的行并输出

# 使用awk命令和正则表达式来处理文本数据，使用~运算符来指定匹配规则，使用$符号和数字来引用字段
awk '$1 ~ /^[A-Z]/ {print $1}' file.txt # 输出文件中第一个字段以大写字母开头的行的第一个字段
awk '$2 ~ /is/ {print $2}' file.txt # 输出文件中第二个字段包含is的行的第二个字段

3.5 Signals and traps

A signal is a mechanism for notifying a process that some event has occurred, and it can be sent by the user, the system, or another process. In Shell scripts, you can use the kill command or shortcut keys such as Ctrl+C to send signals. Signals have some predefined names and numbers like:

SIGINT: Indicates an interrupt signal, numbered 2, can be sent with Ctrl+C
SIGTERM: Indicates the termination signal, the number is 15, which can be sent with the kill command
SIGKILL: Indicates a forced termination signal, numbered 9, which can be sent with the kill -9 command
SIGQUIT: Indicates the exit signal, the number is 3, which can be sent with Ctrl+\
SIGSTOP: Indicates the stop signal, numbered 19, can be sent with the kill -19 command
SIGCONT: Indicates the continuation signal, numbered 18, can be sent with the kill -18 command

A trap is a mechanism for capturing and processing signals, which can be defined with the trap command. The format of the trap command is trap 'action' signal, where action indicates the operation to be performed, and signal indicates the signal to be captured. For example:

# 定义一个陷阱，当收到SIGINT或SIGTERM信号时，输出一条信息并退出
trap 'echo "Bye bye!" && exit' SIGINT SIGTERM

# 定义一个陷阱，当收到SIGQUIT信号时，忽略该信号
trap '' SIGQUIT

# 定义一个陷阱，当收到SIGSTOP或SIGCONT信号时，执行一个函数
trap 'myfunc' SIGSTOP SIGCONT

# 定义一个函数，名为myfunc，输出一条信息
myfunc() {
    
    
  echo "Hello, world!"
}

3.6 Debugging and Error Handling

Debugging is a process used to check and fix code errors. In Shell scripts, you can use the set command and the -x option to enable debugging mode, which can display the execution process and results of each command. For example:

# 启用调试模式
set -x

# 执行一些命令
echo "Hello, world!"
name="Alice"
echo $name

# 关闭调试模式
set +x

Error handling is a mechanism for handling code exceptions. In Shell scripts, you can use the set command and the -e option to enable error handling mode, which allows the script to automatically exit when an error is encountered. For example:

# 启用错误处理模式
set -e

# 执行一些命令
echo "Hello, world!"
name="Alice"
echo $name

# 执行一个错误的命令
ls /not/exist # 这会导致脚本退出，并显示错误信息

4. Application Scenarios of Shell Script

4.1 System Management

System management refers to operations such as configuration, maintenance, and monitoring of the system. Shell scripts can be used to implement some common system management tasks, such as:

Backup and restore data, using commands such as tar, cp, rsync
Install and uninstall software, using commands such as apt, yum, and dpkg
Manage users and permissions, use commands such as useradd, userdel, chmod
Monitor system status and performance, use top, ps, vmstat and other commands
Execute tasks regularly, using commands such as crontab and at

4.2 Data Analysis

Data analysis refers to the collection, processing, analysis and display of data. Shell scripts can be used to implement some simple data analysis tasks, such as:

Download and decompress data, use curl, wget, unzip and other commands
Clean and transform data, using commands such as sed, awk, cut
Statistics and calculation data, using commands such as sort, uniq, wc, etc.
Visualize and report data, using commands like gnuplot, Rscript, and more

4.3 Testing and Debugging

Testing and debugging refers to operations such as checking and repairing codes. Shell scripts can be used to implement some testing and debugging tasks, such as:

Run and stop the program, use ./, kill and other commands
Check and modify program output, using commands such as echo and read
Simulate and capture program input, using commands such as cat, tee, etc.
Track and analyze program errors, using commands such as set and trap

4.4 Network communication

Network communication refers to operations such as data transmission and exchange through the network. Shell scripts can be used to implement some network communication tasks, such as:

Send and receive mail, use mail, mutt and other commands
Upload and download files, use ftp, scp and other commands
Browse and crawl web pages, use curl, wget and other commands
Remote login and control, use ssh, telnet and other commands

Write love you forever into the end of the poem ~

[Linux] shell script: basic syntax and advanced features