Shell script learning tutorial (1)

shell script learning

1. What is Shell?

1. shell overview

Shell is a command line interface (CLI) in a computer operating system that allows users to interact with the operating system and perform various tasks and operations. Shell is also a scripting programming language that allows users to write scripts to automate tasks and execute a series of commands

In Linux and Unix operating systems, Shell 是用户与操作系统内核之间的中间层it accepts commands entered by the user and passes these commands to the kernel for execution. Users can enter commands via the keyboard, and then the Shell shells 解释和执行those commands. The Shell is also responsible for processing 输入和输出、环境变量、文件操作etc.

Some common Shells include Bash (Bourne-Again Shell), Zsh (Z Shell), Fish (Friendly Interactive Shell), etc. Each Shell has its own characteristics and functions. Bash is the most common default shell on Linux and macOS

The main uses of Shell include:

  • Execute commands and programs: Users can enter commands in the Shell to run programs, manage files, view system status, etc.
  • Script programming: Users can write Shell scripts to combine a series of commands and operations to achieve automated tasks
  • Pipes and redirection: The shell allows users to pipe the output of commands to other commands, as well as redirect input and output to files or devices
  • Environment variable management: Shell allows users to set and manage environment variables, which affect the running environment of users and programs.
  • Process control: Shell supports conditional statements, loops and branches, allowing users to write complex script logic

2. Classification of Shells

Shell programming, like JavaScript and PHP programming, requires a text editor that can write code and a script interpreter that can interpret and execute it. There are
many types of Shells in Linux, and the common ones are:

  • Bourne Shell (/usr/bin/sh or /bin/sh)
  • Bourne Again Shell(/bin/bash)
  • C Shell(/usr/bin/csh)
  • K Shell(/usr/bin/ksh)
  • Shell for Root(/sbin/sh)

This tutorial focuses on Bash, also known as Bourne Again Shell. Because it is easy to use and free, Bash is widely used in daily work. At the same time, Bash is also the default Shell for most Linux systems.

In general, people do not distinguish between Bourne Shell and Bourne Again Shell, so#!/bin/sh同样也可以改为 #!/bin/bash

#! 告诉系统其后路径所指定的程序即是解释此脚本文件的 Shell 程序

3. The first shell script

# 1. vim helloworld.sh 是打开一个文本编辑器(Vim)并创建或打开名为 "helloworld.sh" 的文件的命令
vim helloworld.sh

# 2. echo "hello, world!" 是一个 Shell 命令。作用是在终端上打印 "hello, world!"
echo "hello,world!"

# 3. 用于为一个文件赋予可执行权限 
chmod +x helloworld.sh

# 4. 运行这个脚本的命令
./helloworld.sh
  • The execution effect is as follows:
    Insert image description here

4. Multiple command execution

Multiple command executors effect Format
; Multiple commands are executed sequentially without any logical connection between the commands. command 1; command 2
&& When command 1 is executed correctly (exit status code is 0), command 2 will be executed; otherwise command 2 will not be executed. Command 1 && Command 2
|| Command 2 will only be executed when command 1 is executed incorrectly (the exit status code is not 0); otherwise command 2 will not be executed. Command 1 || Command 2

Multiple command execution example:

  1. The semicolon (;) is used to execute multiple commands sequentially, and there is no logical connection between the commands:

    $ command1 ; command2 ; command3
    

    Example:

    $ ls ; echo "Hello" ; date
    

    The above command will execute lscommand (list the contents of the current directory), echocommand (print "Hello"), and datecommand (display the current date and time).

  2. The logical AND (&&) operator is used to execute the next command only if the previous command executed correctly:

    $ command1 && command2
    

    Example:

    $ rm file.txt && echo "File deleted successfully"
    

    The above command first attempts to delete file.txtthe file, and if the deletion is successful, execute echothe command to display "File deleted successfully". If the file deletion fails (command 1 was executed incorrectly), echothe command will not be executed.

  3. The logical OR (||) operator is used to execute the next command only if the previous command was executed incorrectly:

    $ command1 || command2
    

    Example:

    $ find /tmp -name "file.txt" || echo "File not found"
    

    The above command first attempts /tmpto find a file named "file.txt" in the directory. If the file is found (command 1 executed correctly), echothe command will not be executed. If the file is not found (command 1 was executed incorrectly), echothe command is executed to display "File not found".

2. Shell variables

3.1 Naming rules for variables

In shell scripts, variables are key elements used to store and manipulate data. Proper variable naming is key to writing clear, readable, and robust scripts. The following are the naming rules for Shell variables:

  1. Variable names can contain:

    • Letters (uppercase or lowercase): a to z or A to Z.
    • Numbers: 0 to 9 (but variable names cannot start with numbers).
    • Underline _.
  2. Variable names must start with a letter or an underscore. Starting with a number is not allowed.

  3. Variable names are case-sensitive, so $myVarand $myvarare two different variables.

  4. Avoid using Shell's reserved keywords (for example, if, while, for, etc.) as variable names

  5. It is recommended to use meaningful, descriptive variable names to improve script readability.

  6. Common naming styles:

    • Camel Case: The first word starts with a lowercase letter, and the first letter of subsequent words is capitalized.
      For example: myVariableName, thisIsCamelCase.

    • Underscore nomenclature (Snake Case): All letters are lowercase, and words are _separated by underscores.
      For example: my_variable_name, this_is_snake_case.

Example:

Here are some examples demonstrating different variable naming rules:

# 驼峰命名法示例
myVariableName="Hello, World!"
thisIsCamelCase="这是一个驼峰命名"
anotherCamelCaseVariable="这是另个驼峰命名"

# 下划线命名法示例
my_variable_name="Hello, World!"
this_is_snake_case="这是下划线命名"
another_snake_case_variable="这是另一个下划线命名"

Correct variable naming rules are one of the foundations of shell script programming, helping to write maintainable and readable code, reducing errors and improving code quality.

3.2 Special symbols for variables

  1. $ : The dollar sign is used to refer to the value of a variable.
    Example:

    myVar="Hello"
    echo $myVar   # 输出:Hello
    

    Insert image description here

  2. * : Asterisk wildcard, used to match zero or more characters.
    Example:

    ls *.txt   # 列出所有以 ".txt" 结尾的文件
    

    Insert image description here

  3. ? : Question mark wildcard, used to match any character.
    Example:

    ls file?.txt   # 列出类似 "file1.txt"、"fileA.txt" 的文件
    

    Insert image description here

  4. [ ] : Square brackets are used to define the character set and can match any character in the square brackets.
    Example:

    ls [aeiou]*   # 列出以元音字母开头的文件
    

    Insert image description here

  5. { } : Braces are used to create command sequences or expand strings.
    Example:

    echo {
          
          A,B,C}   # 输出:A B C
    

    Insert image description here

  6. | : The vertical bar is used to pass the output of one command to another command, forming a pipe.
    Example:

    cat file.txt | grep "pattern"   # 在文件中搜索模式
    

    Insert image description here

  7. ; : Semicolon is used to separate multiple commands, allowing multiple commands to be executed on one line.
    Example:

    command1; command2   # 依次执行两个命令
    

    Insert image description here

  8. && : Logical AND operator, used to execute two commands and execute the second command only after the first command succeeds.
    Example:

    make && make install   # 如果 make 成功,则执行 make install
    
  9. || : Logical OR operator used to execute two commands and execute the second command only if the first command fails.
    Example:

    command1 || echo "command1 failed"   # 如果 command1 失败,则输出提示
    
  10. \> : The greater than symbol is used to redirect the output of a command to a file.
    Example:

    echo "Hello, World!" > output.txt   # 将文本写入文件
    

Insert image description here

  1. >> : The double greater-than symbol is used to append the output of a command to a file.
    Example:
echo "More text" >> output.txt   # 追加文本到文件

Insert image description here

  1. < : The less than symbol is used to redirect the contents of a file as input to a command.
    Example:
command < input.txt   # 使用文件内容作为输入执行命令
  1. ` : Backticks are used to execute the command and output it as a string.
    Example:
currentDate=`date`   # 获取当前日期并存储为字符串

3.3 User-defined variables

Variables are defined by Shell script writers and are used to store intermediate results of specific tasks or calculations. The scope is the current Shell process or script.

1. Define variables:

In shell scripts, you can use the equal sign (=) to define variables. 变量名不应该包含空格,并且通常使用大写字母, to distinguish them from system environment variables.

myVar="Hello, World"

2. Assign variables:

To assign a value to a variable, just use the equal sign (=) for the assignment operation.

myVar="New Value"

3. Use variables:

To use the value of a variable, precede the variable with a dollar sign ($).

echo $myVar

4. String concatenation:

Variables can be concatenated with strings.

greeting="Hello"
name="John"
echo "$greeting, $name!"

Example:

[root@localhost shell]# NAME="cxk"
[root@localhost shell]# DOING="play-basketball"
[root@localhost shell]# echo $NAME $DOING
cxk play-basketball
[root@localhost shell]# echo "$NAME,$DOING!!!"
cxk,play-basketball!!!

5. Export variables:

If you want to export a variable into the current shell's environment so that other child processes can also access it, you can use exportthe command.

export myVar="Exported Value"

6. Read-only variables:

A variable can be made read-only to prevent its value from being modified.

readonly myVar="This variable is read-only"

Example:

[root@localhost shell]# var='cxk'
[root@localhost shell]# echo $var
cxk
[root@localhost shell]# var='rap'
[root@localhost shell]# echo $var
rap
[root@localhost shell]# readonly var='ikun'
[root@localhost shell]# echo $var
ikun
[root@localhost shell]# var='cxk'
-bash: var: 只读变量

7. Delete variables:

Variables can be deleted using unsetthe command.

unset myVar

Example:

[root@localhost shell]# setVar='temp'
[root@localhost shell]# echo $setVar
temp
[root@localhost shell]# unset setVar
[root@localhost shell]# echo $setVar

8. Best Practices:

  • Use meaningful variable names to improve code readability.
  • Use double quotes to quote variables to handle strings containing spaces or special characters.
  • Avoid using all uppercase letters to define non-environment variables to avoid conflicts with system environment variables.
  • Before using a variable, check whether it is defined to avoid unexpected errors.
  • Use readonlyto prevent variables that should not be changed from being accidentally modified.
  • Use exportto make the variable available in the child process.
  • Add comments to explain the purpose of variables to improve code maintainability.

3.4 Environment variables

1. What are environment variables?

Environment variables are key-value pairs stored at the operating system level and are used to store configuration information, paths, user preferences, etc. Shell environment variables are very important for controlling and configuring system behavior

2. Check the environment variables:

Use echothe command and the dollar sign ($) to view the value of a specific environment variable
. For example, output the current user's home directory path:

echo $HOME

Insert image description here

3. Set environment variables:

To set environment variables, you can use exportthe command:

export MY_VARIABLE="Hello"

This will create an MY_VARIABLEenvironment variable named and set its value to "Hello". Note that this variable is available within the current shell session.

4. Persistent environment variables:

To make an environment variable persistent, you usually add it to the shell configuration file, such as .bashrcor .bash_profile(for the Bash shell). This way the variable is automatically set every time a new shell session is started.

5. View all environment variables:

To view all defined environment variables, you can use envthe command:

env

Or use printenvthe command:

printenv

6. Delete environment variables:

To delete environment variables, you can use unsetthe command:

unset MY_VARIABLE

7. Use environment variables:

Environment variables are very useful in scripts and can be used to store configuration information or paths, for example:

# 使用环境变量
echo "home路径的环境变量地址: $HOME"

8. System predefined environment variables:

The operating system and Shell predefine some environment variables, such as PATH, HOME, USERetc., which can be used to access system information or configuration.

9. Import environment variables:

Importing environment variables means loading environment variables from an external file or source into the current shell session so that they are available in the current session. Typically used to ensure that required environment variables are defined before loading configuration information from a file, setting environment variables, or executing other scripts.

In the shell, there are several ways to import environment variables:

  • Use sourcecommand or .(dot): These two commands are used to execute the commands in the specified file and apply the results to the current shell session. Typically used to load environment variable files, such as .env.

    source .env
    # 或者
    . .env
    

    where .envis the file containing environment variable definitions. After executing these commands, .envthe environment variables defined in the file will take effect in the current shell session.

  • Using exportcommand: If you wish to import environment variables into the current shell session and make them available in child processes, you can use exportthe command:

    export MY_VARIABLE="Hello"
    

    This will create an environment variable named in the current shell session MY_VARIABLEand make it available in the current session and its child processes.

  • Pass ~/.bashrcor ~/.bash_profile: If you want the environment variables to be automatically imported every time you start a new Bash Shell session, you can add it to the .bashrcor .bash_profilefile in the user's home directory. This way the environment variables will be set automatically every time you log in.

    For example, you can .bashrcadd the following line to the file to import environment variables:

    source /path/to/myenvfile
    

10. Best Practices:

  • Use meaningful variable names to improve code readability.
  • Avoid overwriting system predefined environment variables to avoid unnecessary problems.
  • When sensitive information (such as passwords) is required, avoid storing them in environment variables, as environment variables are generally not a secure way to store them.

3.5 Positional parameter variables

  • These variables are used to obtain command line arguments in shell scripts.
  • $0Indicates the name of the script, $1indicates the first parameter, $2indicates the second parameter, and so on.
  • Example:
    script.sh arg1 arg2
    # 在脚本内部可以使用 $0、$1、$2 获取参数
    
    Insert image description here

3.6 Predefined variables

  • The Shell provides some special predefined variables for obtaining information about the shell and the system.
  • For example, $HOMEit represents the current user's home directory and $PWDrepresents the current working directory.
  • Example:
    echo "当前用户的主目录是 $HOME"
    

3.7 Accept keyboard input

  • These variables are used to accept keyboard input from the user.
  • For example, reada command is used to store user input in a specified variable.
  • Example:
    echo "请输入您的姓名:"
    read userName
    echo "您输入的姓名是:$userName"
    
    Insert image description here

3. Shell operator

The following is the syntax and examples of common operators in shell scripts

Operator type operator Syntax example Example
arithmetic operators addition result=$((a + b)) 5 + 2The result is7
Subtraction result=$((a - b)) 5 - 2The result is3
multiplication result=$((a * b)) 5 * 2The result is10
division result=$((a / b)) 10 / 3The result is3
Take remainder result=$((a % b)) 15 % 7The result is1
Relational operators equal [ "$a" -eq "$b" ] 5 -eq 5is true
not equal to [ "$a" -ne "$b" ] 5 -ne 2is true
more than the [ "$a" -gt "$b" ] 5 -gt 2is true
less than [ "$a" -lt "$b" ] 5 -lt 10is true
greater or equal to [ "$a" -ge "$b" ] 5 -ge 5is true
less than or equal to [ "$a" -le "$b" ] 5 -le 10is true
Logical Operators AND operation [ "$a" -gt 0 ] && [ "$a" -lt 10 ] 5 > 0and 5 < 10is true
OR operation [ "$a" -eq 0 ] || [ "$a" -eq 10 ] 5 = 0or 5 = 10is false
NOT operation ! [ "$a" -eq 5 ] 5 = 5is false
assignment operator Assignment x=10 xequal10
additive assignment x=$((x + 5)) x5After addition xequals15
subtractive assignment y=$((y - 5)) yminus equal 5toy15
Bit operators Bitwise AND result=$((a & b)) 5 & 3The result is1
Bitwise OR result=$((a | b)) 5 | 3The result is7
Bitwise XOR result=$((a ^ b)) 5 ^ 3The result is6
Bitwise negation result=$((~a)) ~5The result is-6
left shift result=$((a << 2)) 5 << 2The result is20
right shift result=$((a >> 1)) 5 >> 1The result is2

3.1 Arithmetic operators

Note: In the Shell, if you want to perform the addition operation of variables aand b, you need to use specific syntax to achieve it. Typing directly $a+$bis not interpreted as an addition operation by the shell because the shell treats it as a not found command.

To perform addition operations, you can use exprcommands or $((...))expressions. Here are examples of both methods:

  • Method 1: Use exprcommand

    a=1
    b=2
    result=$(expr $a + $b)
    echo "a + b = $result"
    
  • Method 2: Use $((...))expressions

    a=1
    b=2
    result=$((a + b))
    echo "a + b = $result"
    

Used to perform mathematical operations. The supported arithmetic operators are +, -, *, /, %.

# 语法:result=$((expression))

a=5
b=2

# 加法
result=$((a + b))  # 结果是7

# 减法
result=$((a - b))  # 结果是3

# 乘法
result=$((a * b))  # 结果是10

# 除法
result=$((a / b))  # 结果是2

# 取余
result=$((a % b))  # 结果是1

3.2 Relational operators

Used to compare the relationship between two values. The supported relational operators are -eq, -ne, -gt, -lt, -ge, -le.

# 语法:[ expression ]
# 注意:方括号内的空格是必需的!

a=5
b=10

# 相等
if [ "$a" -eq "$b" ]; then
    echo "$a 等于 $b"
else
    echo "$a 不等于 $b"
fi

# 不等于
if [ "$a" -ne "$b" ]; then
    echo "$a 不等于 $b"
else
    echo "$a 等于 $b"
fi

# 大于
if [ "$a" -gt "$b" ]; then
    echo "$a 大于 $b"
else
    echo "$a 不大于 $b"
fi

# 小于
if [ "$a" -lt "$b" ]; then
    echo "$a 小于 $b"
else
    echo "$a 不小于 $b"
fi

# 大于等于
if [ "$a" -ge "$b" ]; then
    echo "$a 大于等于 $b"
else
    echo "$a 小于 $b"
fi

# 小于等于
if [ "$a" -le "$b" ]; then
    echo "$a 小于等于 $b"
else
    echo "$a 大于 $b"
fi

3.3 Logical operators

Used to perform logical operations. The supported logical operators are &&(logical AND), ||(logical OR), !(logical NOT).

# 语法:command1 && command2
# command2 仅在 command1 返回真(退出状态码为0)时执行

# 与运算
if [ "$a" -gt 0 ] && [ "$a" -lt 10 ]; then
    echo "$a 大于0并且小于10"
else
    echo "$a 不满足条件"
fi

# 或运算
if [ "$a" -eq 0 ] || [ "$a" -eq 10 ]; then
    echo "$a 等于0或等于10"
else
    echo "$a 不满足条件"
fi

# 非运算
if ! [ "$a" -eq 5 ]; then
    echo "$a 不等于5"
else
    echo "$a 等于5"
fi

3.4 Assignment operator

Used to assign values ​​to variables. The supported assignment operators are =``+=, -=, *=, /=, %=.

# 语法:variable operator expression

x=10
y=20

# 赋值
x=5
y=10

# 加法并赋值
x=$((x + 5))  # x 现在等于10

# 减法并赋值
y=$((y - 5))  # y 现在等于15

3.5-bit operators

Used to perform bit operations. The supported bit operators are &(bitwise AND), |(bitwise OR), ^(bitwise exclusive OR), (bitwise ~negation), <<(left shift), >>(right shift).

# 语法:result=$((expression))

a=5
b=3

# 按位与
result=$((a & b))  # 结果是1

# 按位或
result=$((a | b))  # 结果是7

# 按位异或
result=$((a ^ b))  # 结果是6

# 按位取反
result=$((~a))     # 结果是-6

# 左移
result=$((a << 2))  # 结果是20

# 右移
result=$((a >> 1))  # 结果是2

4. Process control

The process control structures in Shell scripts include conditional statements (if statements), loop statements (for loop, while loop), case statements, etc. The following is detailed syntax and usage examples for each flow control structure:

4.1 Conditional statement - if statement

  • ifStatements are used to execute different commands under specific conditions. The syntax is as follows:

    if [ condition ]; then
        # 条件为真时执行的命令
    else
        # 条件为假时执行的命令
    fi
    
  • Example:

    #!/bin/bash
    
    x=10
    
    if [ $x -eq 10 ]; then
        echo "x 等于 10"
    else
        echo "x 不等于 10"
    fi
    

4.2 Loop statement - for loop

  • forLoops are used to execute a series of commands over the elements in a list. The syntax is as follows:

    for variable in list; do
        # 在每次迭代中执行的命令
    done
    
  • Example:

    #!/bin/bash
    
    fruits=("apple" "banana" "cherry")
    
    for fruit in "${fruits[@]}"; do
        echo "水果:$fruit"
    done
    

4.3 Loop statement - while loop

  • whileA loop is used to execute a sequence of commands when a condition is true until the condition is false. The syntax is as follows:

    while [ condition ]; do
        # 当条件为真时执行的命令
    done
    
  • Example:

    #!/bin/bash
    
    count=1
    
    while [ $count -le 5 ]; do
        echo "循环次数:$count"
        ((count++))
    done
    

4.4 case statement

  • caseStatements are used to execute different commands based on different conditions. The syntax is as follows:

    case expression in
        pattern1)
            # 匹配 pattern1 时执行的命令
            ;;
        pattern2)
            # 匹配 pattern2 时执行的命令
            ;;
        *)
            # 默认情况下执行的命令
            ;;
    esac
    
  • Example:

    #!/bin/bash
    
    fruit="apple"
    
    case $fruit in
        "apple")
            echo "这是苹果"
            ;;
        "banana")
            echo "这是香蕉"
            ;;
        *)
            echo "这不是苹果或香蕉"
            ;;
    esac
    

4.5 Control flow - break and continue

  • breakUsed to exit the loop.

  • continueUsed to skip the remainder of the current loop and continue with the next iteration.

  • Example:

    #!/bin/bash
    
    for i in {
          
          1..5}; do
        if [ $i -eq 3 ]; then
            continue
        fi
        echo "循环次数:$i"
        if [ $i -eq 4 ]; then
            break
        fi
    done
    

5. Function

Shell functions are powerful tools that allow a set of commands and logic to be encapsulated into a reusable unit to perform a specific task or calculation. In this article, we will delve into the detailed syntax of Shell functions and provide multiple practical examples to better understand and use them.

5.1 Shell function definition

Shell functions can use two different syntax forms, using functionkeywords and using compact form.

5.1.1 function keyword function definition

function function_name {
    
    
    # 函数体,包括命令和逻辑
    # 可以接受参数
    # 可以使用局部变量
    # 可以返回一个值
}

5.1.2 Compact form function definition

function_name() {
    
    
    # 函数体
}

5.2 Function call

Once a function is defined, it can be called wherever needed in the script. When calling a function, simply use its name followed by the required arguments

grammar:

function_name argument1 argument2

Simple function call example

# 定义一个函数,用于打印欢迎消息
welcome() {
    
    
    echo "欢迎来到Shell函数示例!"
}

# 调用函数
welcome

5.3 Function parameters

Functions can accept parameters, which makes them more versatile and flexible. Within the function, you can use variables such $1as , $2, $3and so on to refer to the parameters passed to the function. These variables represent the first parameter, the second parameter, the third parameter, and so on.

Let's look at an example of a function that accepts parameters:

# 定义一个函数,接受两个参数并打印它们
print_arguments() {
    
    
    echo "第一个参数: $1"
    echo "第二个参数: $2"
}

# 调用函数,并传递两个参数
print_arguments "Hello" "World"
print(){
echo "你的名字:$1"
echo "你的爱好:$2"
}
print "蔡徐坤" "打篮球"

Insert image description here

5.4 Function return value

Shell functions can return a value, which enables the function to perform calculations and pass the results back to the main program. To return a value, you can use returna statement and then use it in the main program $?to get the return value.

Example of a function that calculates the sum of two numbers and returns the result:

# 定义一个函数,计算两个数字之和并返回结果
add_numbers() {
    
    
    local sum=$(( $1 + $2 ))
    return $sum
}

# 调用函数,并获取返回值
add_numbers 5 7
result=$?
echo "5 + 7 的和是:$result"

5.5 Local variables

In Shell functions, you can use localkeywords to create local variables. Local variables are only visible inside the function and do not affect variables in the global scope. This is very useful because it allows variables to be used inside a function without worrying about conflicts with other parts of the code.

Function example of local variables:

# 定义一个函数,演示局部变量
my_function() {
    
    
    local local_variable="局部变量"
    echo "在函数内部:$local_variable"
}

local_variable="全局变量"
my_function
echo "在函数外部:$local_variable"
fun(){
    
    

echo "全局变量值:$var"
local var="局部变量"
echo "局部变量赋值后:$var"
}
var="全局变量"
fun

Insert image description here

5.6 Practical examples

5.6.1 Calculating factorial

Computes the factorial of a given number. The factorial is the product of a positive integer and the product of all positive integers from 1 to that integer.

# 定义一个函数,计算给定数字的阶乘
calculate_factorial() {
    
    
    local number=$1
    local result=1

    if [ $number -lt 0 ]; then
        echo "输入必须是非负整数。"
        return 1
    fi

    for (( i=1; i<=number; i++ )); do
        result=$((result * i))
    done

    echo "$number 的阶乘是:$result"
    return 0
}

# 调用函数,计算阶乘
calculate_factorial 5
calculate_factorial 0
calculate_factorial -3

5.6.2 Find files

Finds a specific file in the file system and returns the path to the file

# 定义一个函数,查找文件并返回路径
find_file() {
    
    
    # 接受两个参数:文件名和搜索目录
    local file_name=$1
    local search_dir=$2

    # 检查是否提供了有效的文件名和搜索目录
    if [ -z "$file_name" ] || [ -z "$search_dir" ]; then
        echo "请输入文件名和搜索目录。"
        return 1  # 返回错误代码1表示参数不足或无效
    fi

    # 使用`find`命令在指定目录中查找文件
    local result=$(find "$search_dir" -name "$file_name")

    # 检查是否找到文件
    if [ -z "$result" ]; then
        echo "未找到文件 '$file_name'。"
        return 1  # 返回错误代码1表示未找到文件
    else
        echo "文件 '$file_name' 的路径是:$result"
        return 0  # 返回成功代码0表示找到文件
    fi
}

# 调用函数,查找文件
find_file "example.txt" "/path/to/search"   # 查找存在的文件
find_file "missing.txt" "/path/to/search"   # 查找不存在的文件
find_file "" "/path/to/search"              # 无效的参数:文件名为空

6. Character interception, replacement and processing commands

6.1 Common character processing

6.1.1 String length

grammar:

${
    
    #string}

Example:

string="Hello, World!"
length=${
    
    #string}
echo "字符串的长度为:$length"  # 输出 "字符串的长度为:13"

6.1.2 Intercepting substrings

grammar:

${string:起始位置:长度}

Example:

string="Hello, World!"
substring="${string:0:5}"  # 从位置0开始截取5个字符
echo "$substring"  # 输出 "Hello"

6.1.3 Delete prefix

grammar:

${string#前缀}

Example:

string="/path/to/file.txt"
without_prefix="${string#/path/}"  # 删除前缀 "/path/"
echo "$without_prefix"  # 输出 "to/file.txt"

6.1.4 Delete suffix

grammar:

${string%后缀}

Example:

string="/path/to/file.txt"
without_suffix="${string%.txt}"  # 删除后缀 ".txt"
echo "$without_suffix"  # 输出 "/path/to/file"

6.1.5 Convert to lowercase

grammar:

${string,,}

Example:

string="Hello, World!"
lowercase="${string,,}"  # 转换为小写
echo "小写: $lowercase"  # 输出 "小写: hello, world!"

6.1.6 Convert to uppercase

grammar:

${string^^}

Example:

string="Hello, World!"
uppercase="${string^^}"  # 转换为大写
echo "大写: $uppercase"  # 输出 "大写: HELLO, WORLD!"

6.1.7 String replacement (first match)

grammar:

${string/要替换的子串/替换为的字符串}

Example:

string="姓名:name,age:20,height:159cm,艺名:name"
replaced="${string/name/cxk}"  # 替换第一个 "name" 为 "cxk"
echo "$replaced"  # 输出 "姓名:cxk,age:20,height:159cm,艺名:name"

Insert image description here

6.1.8 String replacement (match all)

grammar:

${string//要替换的子串/替换为的字符串}

Example:

string="姓名:name,age:20,height:159cm,艺名:name"
replaced="${string//name/cxk}"  # 替换所有的 "apples" 为 "bananas"
echo "$replaced"  # 输出 "I love bananas, bananas are great!"

Insert image description here

6.1.9 Extract substring matching

grammar:

[[ $string =~ 正则表达式 ]]
匹配结果:${
    
    BASH_REMATCH[n]}

Example:

string="我的邮箱 [email protected]"
# 使用正则表达式来匹配字符串中的邮箱地址,并将匹配结果存储在`BASH_REMATCH`数组中,然后提取出匹配的邮箱地
if [[ $string =~ [a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{
    
    2,4} ]]; then
    email="${
     
     BASH_REMATCH[0]}"
    echo "匹配到的邮箱地址是:$email"
fi

Insert image description here

6.2 Regular expressions and wildcards

6.2.1 The difference between regular expressions and wildcards

  1. Regular Expressions :

    • Regular expressions are a powerful 模式匹配tool for matching strings in text based on patterns. It is an include match, so it can match anywhere in the string.
    • In Linux, commands such as grep, awk, sedetc. usually support regular expressions. You can use regular expressions to search, replace, filter text, and more.
    • The rich syntax of regular expressions allows you to perform advanced text operations such as capturing, grouping, repeat matching, etc.

    Example:

    • Use grepto find all rows containing numbers:grep '[0-9]' file.txt
  2. Wildcard :

    • Wildcards are used to match file names. It is an exact match that only matches file names that match a specific pattern, regardless of file content.
    • In Linux, commands such as ls, find, cpetc. usually support wildcards. Wildcard characters are used to find or manipulate files, not the text within the files.
    • Wildcard characters include *(match zero or more characters), ?(match one character), [...](match a range of characters), etc.

    Example:

    • Use to lslist all .txtfiles ending with :ls *.txt

Summarize:

  • Regular expressions are used for text processing and support more advanced matching and operations.
  • Wildcards are used for file name matching and are a basic file selection tool.

6.2.2 Commonly used regular expressions

Regular expression pattern describe Usage example
^pattern Matches patternlines starting with . grep '^Error' file.txtMatch lines starting with "Error".
pattern$ Matches patternlines ending with. grep 'end$' file.txtMatches lines ending with "end".
. Matches any single character except newline. grep 'a.b' file.txtMatches "aab", "axb", etc.
* Matches zero or more instances of the previous character or subexpression. grep 'go*gle' file.txtMatches "ggle", "google", "gooogle", etc.
+ Matches one or more instances of the previous character or subexpression. grep 'go+gle' file.txtMatches "google", "gooogle", etc., but not "ggle".
? Matches zero or one instance of the previous character or subexpression. grep 'colou?r' file.txtMatches "color" and "colour".
[abc] Matches any character in the character set. grep '[aeiou]' file.txtMatches lines containing vowels.
[0-9] Match any number. grep '[0-9]' file.txtMatches lines containing numbers.
[^abc] Matches any character that is not in the character set. grep '[^0-9]' file.txtMatches lines that do not contain numbers.
\d Matches any numeric character, equivalent to [0-9]. grep '\d' file.txtMatches lines containing numbers.
\D Matches any non-numeric character, equivalent to [^0-9]. grep '\D' file.txtMatches lines that do not contain numbers.
\w Matches any single word character (letter, number, or underscore), equivalent to [a-zA-Z0-9_]. grep '\w' file.txtMatches lines containing word characters.
\W Matches any non-word character, equivalent to [^a-zA-Z0-9_]. grep '\W' file.txtMatches lines that do not contain word characters.
\s Matches any whitespace character (space, tab, newline, etc.). grep '\s' file.txtMatches lines containing whitespace characters.
\S Matches any non-whitespace character. grep '\S' file.txtMatches lines that do not contain whitespace characters.
(pattern) Create a capture group that captures matching text. grep 'a\(bc\)*d' file.txtMatches "ad", "abcd", "abcbcd", etc.
` ` Logical OR operator, matches either of two patterns.
.* Matches zero or more arbitrary characters, usually an entire line of text. grep '.*pattern.*' file.txtMatches lines containing "pattern".

6.3 Character interception, replacement and processing commands

6.3.1 cut command

cut命令用于从文本行中剪切或提取字段。默认情况下,cut使用制表符(Tab)作为字段分隔符,但可以通过-d选项指定其他分隔符。它通常用于提取或删除文本文件中的特定列。

基本用法:

cut [OPTIONS] [FILE]
  • 剪切文件中的第2列:

    cut -f2 input.txt
    

    Insert image description here

  • 使用逗号作为分隔符剪切第1和第3列:

    cut -d',' -f1,3 input.csv
    

    Insert image description here

  • 以特定字符作为字段分隔符,剪切第1列:

    cut -d',' -f1 /etc/passwd
    

    Insert image description here

  • 剪切每行的前5个字符:

    cut -c1-5 input.txt
    

    Insert image description here

6.3.2 awk命令

awk是一个强大的文本处理工具,它可以执行复杂的文本处理操作,包括文本提取、计算、条件筛选等。awk将文本分割成字段,并允许对字段进行操作。它的灵活性使其成为处理结构化文本数据的理想工具。

基本用法:

awk 'pattern { action }' [FILE]
  • 显示文件的第2列和第3列:

    awk '{print $2, $3}' input.txt
    

    Insert image description here

  • 计算文件中所有数字的总和:

    awk '{ sum += $1 } END { print sum }' input.txt
    

    Insert image description here

  • 使用逗号作为分隔符,打印第1列和第3列:

    awk -F',' '{print $1, $3}' input.csv
    

    Insert image description here

  • 打印包含特定模式的行:

        awk '/pattern/ {print}' input.txt
    

    Insert image description here

6.3.3 sed命令

sed(流编辑器)用于对文本进行基本的编辑和转换,如替换、删除、插入和替换。sed可以通过正则表达式来匹配和操作文本,通常在管道中与其他命令一起使用。

基本用法:

sed [OPTIONS] 's/pattern/replacement/' [FILE]
  • 替换文本文件中的所有"old"为"new":

    sed 's/old/new/g' input.txt
    

    Insert image description here

  • 删除包含特定字符串的行:

    sed '/pattern/d' input.txt
    

    Insert image description here

  • 在每行的开头插入文本:

    sed 's/^/Prefix /' input.txt
    

    Insert image description here

  • 将文件中的所有字母转换为大写:

    sed 's/[a-z]/\U&/g' input.txt
    

    Insert image description here

6.4 sort、uniq、wc 命令

sortuniqwc是在Linux和Unix系统上常用的命令,用于排序、去重和统计文本数据。

6.4.1 sort命令

sort命令用于对文本文件的行进行排序。默认情况下,它按照字典顺序对文本行进行排序,但可以使用不同的选项来进行数字排序、逆序排序等。

基本用法:

sort [OPTIONS] [FILE]
  • 对文件进行排序并将结果输出到标准输出:

    sort input.txt
    

    Insert image description here

  • 对文件进行数字排序(例如,按数值大小而不是字典顺序):

    sort -n input.txt
    

    Insert image description here

  • 对文件进行逆序排序:

    sort -r input.txt
    

    Insert image description here

6.4.2 uniq命令

uniq命令用于从已排序的文本数据中去重重复的行。它通常与sort命令结合使用,以确保重复行相邻。

基本用法:

uniq [OPTIONS] [FILE]
  • 从文件中去重重复的行:

    sort input.txt | uniq
    

    Insert image description here

  • 显示去重后的行以及它们重复的次数:

    sort input.txt | uniq -c
    

    Insert image description here

6.4.3 wc命令

wc命令用于统计文本文件的行数、字数和字符数。

基本用法:

wc [OPTIONS] [FILE]
  • 统计文件的行数、单词数和字符数:

    wc input.txt
    

    Insert image description here

  • 仅统计行数:

    wc -l input.txt
    

    Insert image description here

  • 仅统计单词数:

    wc -w input.txt
    

    Insert image description here

  • 仅统计字符数:

    wc -c input.txt
    

    Insert image description here

6.5 管道(Pipeline)

管道(Pipeline)是在Unix和类Unix操作系统中广泛使用的强大工具,它允许将一个命令的输出直接传递给另一个命令,以便进行复杂的数据处理和筛选。

管道(Pipeline)语法

command1 | command2
  • command1:第一个命令的输出将被传递给第二个命令。
  • command2:接收来自command1的输入并进行处理的第二个命令。

6.6 grep命令

grep命令用于在文本中搜索匹配指定模式的行

grep命令语法

grep [选项] 模式 [文件...]
  • 选项:可以是一些可选的标志,用于控制搜索的行为。
  • 模式:要搜索的文本模式或正则表达式。
  • 文件:要在其中搜索模式的一个或多个文件。如果省略文件参数,则grep将从标准输入中读取数据。
  1. 搜索文件中包含特定字符串的行:
grep "search_term" filename.txt
  1. 搜索多个文件中包含特定字符串的行:
grep "search_term" file1.txt file2.txt
  1. 搜索目录中所有文件包含特定字符串的行:
grep "search_term" /path/to/directory/*
  1. 搜索多个文件并显示匹配行的行号:
grep -n "search_term" file1.txt file2.txt
  1. 忽略大小写进行搜索:
grep -i "search_term" filename.txt
  1. 使用正则表达式进行高级搜索(例如,查找以"abc"开头的行):
grep "^abc" filename.txt
  1. 逆向搜索,即只显示不匹配模式的行:
grep -v "search_term" filename.txt
  1. 递归搜索目录中的文件:
grep -r "search_term" /path/to/directory/
  1. 搜索匹配模式的行并统计匹配的次数:
grep -c "search_term" filename.txt

7. 输入输出重定向

7.1 Linux标准输入

类型 设备 设备名 文件描述符 类型
标准输入(stdin) 键盘 /dev/stdin 0 标准输入
标准输出(stdout) 显示器 /dev/stdout 1 标准输出
标准错误输出(stderr) 显示器 /dev/stderr 2 标准错误输出

7.2 输入重定向

类型 符号(语法) 功能
标准输入 命令 < 文件1 命令将文件1的内容作为标准输入设备
标识符限定输入 命令 << 标识符 命令将标准输入中读入内容,直到遇到“标识符”分隔符为止
输入输出重定向 命令 < 文件1 > 文件2 命令将文件1的内容作为标准输入,将文件2作为标准输出。

7.3 输出重定向

类型 符号 作用
Standard output redirection (override) Command > File In overwrite mode, output the correct output content of the command to the specified file or device
Standard output redirection (append) Command>>File In append mode, output the correct output content of the command to the specified file or device
Standard error output redirection (override) Error command 2> file Outputs the error output of the command to the specified file or device in overwriting mode
Standard error output redirection (append) Error command 2>> file In append mode, output the error output of the command to the specified file or device
Save both stdout and stderr (overwrite) Command>File2>&1 In overlay mode, save both correct and error output to the same file
Save both standard output and standard error output (append) Command>>File2>&1 In append mode, save both correct output and error output to the same file
Save both stdout and stderr (overwrite) Command&>File In overlay mode, save both correct and error output to the same file
Save both standard output and standard error output (append) Command &>> File In append mode, save both correct output and error output to the same file
Save correct output and error output separately Command > File 1 2 > File 2 Save correct output to file 1 and error output to file 2

7.4 /dev/null

If you want to execute a command without displaying the output on the screen, you can /dev/nulldiscard the output entirely by redirecting it to a file.

Example:

command > /dev/null

This command redirects the command's standard output to /dev/nulla file, discarding the output entirely.

Guess you like

Origin blog.csdn.net/qq_29864051/article/details/132650005