Shell script entry to in-depth tutorial: Getting Started

Benpian is a Quick Start tutorial, and then later in the article for relevant content depth.

Shell Script Basics

Bash Comment

Bash supports only single-line comments, use #the beginning of the statement are interpreted as comments:

# 整行注释
echo hello world # 行尾注释

Bash through some of the features can be tricky to achieve multi-line comments:

: '
注释1
注释2
'

: <<'EOF'
注释1
注释2
EOF

____='
注释1
注释2
'

However, do not pain free eggs tricky to use multi-line comments, peace of mind #to comment.

Bash basic data types

Only basic data types Bash string type, numeric type are not even ( declare -ito force values declared type).

such as:

# 都会当作字符串
echo haha
echo 1234

Bash string concatenation

Bash operating in series strings directly connected to the two pieces of data, without any operator.

E.g:

echo "junma""jinlong"
echo 1234 5678

Command Basics

Variable assignment and reference variable

a=3
echo $a

a="www.junmajinlong.com"
echo $a

a='hello world'
echo $a

Shell can reference an undefined variable:

echo $xyzdefabc

Empty variables can be defined:

a=
echo $a

Variable Substitution

Variable substitution means at a position before the start of execution, the value of the variable Shell will replace the first reference variable in the command.

E.g:

a="hello"
echo $a world

Before the echo command begins execution, Shell will get a variable value hello, and replace it with the command line $aat. So, when the echo command execution, command line has become:

echo hello world

In addition to variable substitution, Shell will make other alternatives:

• Command substitution
• Process substitution
• Replace Arithmetic
• Brace expansion
• Tilde expansion
• Path extension

These extensions and replacements are Shell command before the call is complete, this and other language parsing code in a different way.

The back will explain in detail how Shell is doing command-line parsing, if you do not master the command line parsing, when faced with the command-line syntax error is likely to spend a lot of unnecessary time to debug commands. And the mastery of command line parsing, command will be in charge of the life cycle of such, I can not say one will be able to write all the command line, but it can save a lot of debugging time, to write the command line and the ability to write the script will rise a level .

Command substitution

Backticks or $()can execute command substitution.

`cmd`
$(cmd)

First substitution means performs command cmd, cmd and outputs the result to replace $()or at a position back quotes.

E.g:

echo `id root`
echo $(id root)

Before the echo command is executed, the first execution will result id command, the command id:

$ id root
uid=0(root) gid=0(root) groups=0(root)

So the result will be uid=0(root) gid=0(root) groups=0(root)replaced $(id root). So, when the echo command execution, command line has become:

echo uid=0(root) gid=0(root) groups=0(root)

Arithmetic

$[]Or $(())or let command can do the arithmetic.

let alone command, the other can not write the command line.

a=3
let a=a+1
echo $a

$[]And $(())can be written on the command line inside, Shell when parsing the command line, they will do the arithmetic, and then replace the computation result to the command line.

a=33
echo $[a+3]
echo $((a+3))

Because variable substitution to replace in arithmetic, therefore, use a variable name or reference variable methods can be:

a=333
echo $[$a+3]
echo $(($a+3))

Exit status codes

After each command execution there will be a corresponding process exit status codes that indicate whether or not the process is normal exit.

Therefore, the command line, Shell script, often using special variables $?determine whether a recent reception command normal exit.

Typically, if $?the value of:

• 0 indicates successful execution of the process, that normal exit
• Non-zero, it indicates that the process is not successfully executed, that is, non-normal exit
• But non-zero exit status does not necessarily indicate an error, it may be normal logical exit

In addition, Shell script, all conditional (such as an if statement, while statement) expressed True 0 exit status to a non-zero exit status to False.

exit command

exit command can be used to exit the current Shell process, such as Shell exits the current terminal, exit Shell scripts, and so on.

exit [N]

exit exit status codes may specify N, omitted, if N, the default exit status is 0, it means that the correct exit.

Background execution Command &

At the end use the command &symbol, this command can be placed in the background.

After the command into the background, the process will immediately return to Shell, Shell process will execute immediately the next command (if any) or to exit.

Use $!can get a PID recent background process.

sleep 20 &
echo $!

Use waitthe command can wait for a background process (the current child process Shell process) is completed:

wait [n1 n2 n3 ...]

When not given any parameters to, we will wait for all child processes (that is, all background process) is completed.

sleep 5 &
wait
echo haha

Multi-command combination

Shell There are several ways of combining a plurality of commands.

After 1.cmd1 quit, do cmd2

cmd1;cmd2

After 2.cmd1 correct exit (exit status 0), execution cmd2

cmd1 && cmd2

After 3.cmd1 does not exit properly, perform cmd2

cmd1 || cmd2

4. logics: &&and ||may be combined arbitrarily

# cmd1正确退出后执行cmd2，cmd2正确退出后执行cmd3
cmd1 && cmd2 && cmd3...

# cmd1正确退出则执行cmd2，cmd1不正确退出会执行cmd3
# cmd1正确退出，但cmd2不正确退出，也会执行cmd3
cmd1 && cmd2 || cmd3

# cmd1正确退出会执行cmd3
# cmd1不正确退出会执行cmd2，cmd2正确退出会执行cmd3
cmd1 || cmd2 && cmd3

5. The plurality of command packet: parentheses or brackets plurality of commands may be combined

# 小括号组合的多个命令是在子Shell中执行
# 即会先创建一个新的Shell进程，在执行里面的命令
(cmd1;cmd2;cmd3)

# 大括号组合的多个命令是在当前Shell中执行
# 大括号语法特殊，要求：
#   1.开闭括号旁边都有空白，否则语法解析错误(解析成大括号扩展)
#   2.写在同一行时，每个cmd后都要加分号结尾
#   3.多个命令可分行书写，不要求分号结尾
{ cmd1;cmd2;cmd3; }
{
  cmd1
  cmd2
  cmd3
}

Basic redirection

Software Design believes that the program should have a data source, data export and reporting the wrong place. In the Linux system, each program will be open three default file descriptor (file descriptor, fd):

• fd = 0: standard input, indicates where the data read from the default program
• fd = 1: standard output, the program indicates where the data is output to the default
• fd = 2: standard error, indicates that the program will output an error message to default where

File descriptor, it means the file system to open track allocated to it a number, file number and the corresponding relationship: reading data from a file descriptor, it means that the corresponding data is read from a file, the file write data descriptor, it means that data is written to the corresponding file.

Linux, everything is a file, the file descriptor is a file. default:

• fd = 0 is a standard input / dev / stdin file
• fd = 1 standard output is / dev / stdout file
• fd = standard error 2 is / dev / stderr file

The default file is a soft link to the file each terminal:

$ ls -l /dev/std*
lrwxrwxrwx 1 root root 15 Jan  8 20:26 /dev/stderr -> /proc/self/fd/2
lrwxrwxrwx 1 root root 15 Jan  8 20:26 /dev/stdin -> /proc/self/fd/0
lrwxrwxrwx 1 root root 15 Jan  8 20:26 /dev/stdout -> /proc/self/fd/1

$ ls -l /proc/self/fd/
lrwx------ 1 root root 64 Jan 16 10:40 0 -> /dev/pts/0
lrwx------ 1 root root 64 Jan 16 10:40 1 -> /dev/pts/0
lrwx------ 1 root root 64 Jan 16 10:40 2 -> /dev/pts/0
lr-x------ 1 root root 64 Jan 16 10:40 3 -> /proc/75220/fd

Therefore, by default all read and write data terminals, for example:

# 数据输出到终端
$ echo haha
$ cat /etc/fstab

# 从终端读取数据
$ cat
hello   # 在终端输入
hello   # 在终端输出
world   # 在终端输入
world   # 在终端输出
^C

Changing the target file descriptor corresponding to the data flow can be changed. Fd = 1 such as a standard default input flow is a terminal device, which if changed /tmp/a.log, it allows the file data is written /tmp/a.log rather than the terminal device.

In the Shell, this change of behavior is called the target file descriptor to redirect that redefine the flow of data.

In fact, there are many types of file descriptor operations, including the redirection fd, fd is assigned (Open, i.e., open the file), fd copy (Duplicate), fd movement (Move), fd is closed (close). Now only describes the basic redirection.

Shell, based redirection in the following ways:

• [N]> file: overlay output redirection, fd = n output to change to the output data file to file, create file does not exist, the presence of the file data is written again to empty
  • Omitted when n> file, equivalent to 1> file, i.e., the standard output is redirected to the file file cover
• [N] >> file: Output additional redirect output to fd = n to the output data changes to the file tail file, create file does not exist, the additional file is present directly in the end of the file
  • When omitted, n, >> file equivalent to 1 >> file, i.e., the standard output file to file append redirection
• [N] <file: input redirection, open the file in read mode, and the file allocation fd = n, file does not exist error
  • When omitted, n <file, is equivalent to 0 <file, i.e., the data is read directly from the file
  • Generally only the program data read from the fd = 0, so that when n is not equal to 0, the operation requires one more step 3 <file <& 3, read skip
• &> File: This is a special redirection methods, represents the standard output and standard error are redirected to the file file, equivalent to> file 2> & 1
• & >> file: This is a special redirection methods, represent the standard error and standard output are appended to the file file, equivalent to >> file 2> & 1

Furthermore, a special object file is often used for the output / dev / null, which device is empty, it may lose all write data directly.

echo www.junmajinlong.com >/dev/null
curl -I www.junmajinlong.com 2>/dev/null >/tmp/a.log

cat </etc/fstab

A frequently used technique is the way to empty file:

$ cat /dev/null >file
$ >file

Distinguish cat <file cat file and

cat is a command, the command's source code to write some code to handle options and parameters.

cat -n /etc/fstab

After the cat command execution, identifies -noptions, while the output line numbers, while the cat will recognize this option will make cat output /etc/fstabparameters, cat reads the parameters specified file and output.

If the file does not specify the parameters of the cat, the cat will default to read data from standard input. The default standard input is a terminal, so there is no change in the flow of standard input, reads data from the terminal, which is what the user input characters, reads what character, what character then output:

$ cat
junmajinlong   # 在终端输入
junmajinlong   # 在终端输出
junma   # 在终端输入
junma   # 在终端输出
^C

But the user can change the standard input sources. such as:

$ cat </etc/fstab

It represents the standard input source to the / etc / fstab file, then reads data from the cat / etc / fstab in.

Moreover, convention, would use a -to represent standard input or standard output.

# 下面是等价的，都表示从标准输入中读取数据
cat -
cat /dev/stdin
cat

Note: This is not infallible, but most of the code in the program of the convention defines -the relevant code processing. Refer to the relevant command man page. As man cat in a line:

With no FILE, or when FILE is -, read standard input.

here doc

Input redirection is <, in addition to << <<<.

<< notation here doc. In other words, it is followed with a document, like a file, but this is a temporary file is defined in the << symbol. here doc used to specify multiple data input.

Since it is a document, there is a beginning of the document symbol indicates the document from the beginning and termination document symbol indicates the document concludes. Start and end in the middle of all the contents of that document content. Content of the document will be read as a data standard input.

Document start and end characters can be arbitrarily defined, but must be the same both before and after. Common symbols are:

• EOF：end of file
• EOL：end of line
• EOB：end of block

E.g:

# here doc作为标准输入被读取，然后被cat输出
cat <<EOF
hello
world
EOF

# here doc的内容还会被cat覆盖式输出到指定文件
cat <<eof >/tmp/file
hello
world
eof

# here doc的内容还会被cat追加式输出到指定文件
cat <<eof >>/tmp/file
hello
world
eof

# here doc和重定向符号的前后位置随意
cat >>/tmp/file<<eof
...
eof

Further, if the start code value is quoted, is not variable substitution, command substitution, replacement, etc. arithmetic. If not quoted start character, it will be replaced.

a=333
cat <<eof
$a
eof

cat <<"eof"
$a
eof

Output:

333
$a

here string

<<< expressed here string. That is behind the symbol is a string that will be entered as content standards.

cat <<<"www.junmajinlong.com"

When using single quotes here string, not variable substitution, replacement, etc. command, use can be replaced when surrounded by double quotes.

$ a=3333
$ cat <<<$a            
3333
$ cat <<<"hello world$a"
hello world3333
$ cat <<<'hello world$a' 
hello world$a

here string can often replace the pipe before the echo command echo xxx|. E.g:

# 下面是等价的
echo hello world | grep "llo"
grep "llo" <<<"hello world"

pipeline

Pipeline usage:

cmd1 | cmd2 | cmd3...

Each vertical bar represents a pipeline. The above command will be represented cmd1 standard output into the duct, cmd2 reads from the pipeline processing, cmd2 standard output will be placed in another conduit, cmd3 reads data from the pipeline processing. Back can also take any number of pipes.

Shell Shell pipeline is one of the most commendable features, it implements inter-process communication pipes in a very simple form, I think half of Shell processing text data comes from the form of vertical pipes. Like other programming languages, open the pipeline should write to distinguish which process pipeline, which process reading the pipe, for safety, but also to shut down each process not read or write terminal end, in short, it is in trouble, and Shell pipeline is very simple, vertical line on the left is to write pipeline, the right of the vertical line is read pipe.

E.g:

ps aux | grep 'sshd'

Data (Standard Output) PS command generation will be written into the pipe, as long as there is data in the pipeline, grep command to read data for processing.

That the following command, grep reads data from which it?

ps aux | grep '#' /etc/fstab

That both want to make grep / fstab to read data from / etc, also read data from the pipe it?

ps aux | grep '#' /etc/fstab /etc/stdin
ps aux | grep '#' /etc/fstab -

The tee command

tee command can be copied as a standard input and standard output to 0 or more files. In other words, tee role of multiple data redirection.

NAME
    tee - read from standard input and write to standard output and files

SYNOPSIS
    tee [OPTION]... [FILE]...

DESCRIPTION
    Copy standard input to each FILE, and also to standard output.

    -a, --append
        ppend to the given FILEs, do not overwrite

Figure:

E.g:

$ echo hello world | tee /tmp/file1 /tmp/file2 | cat
$ echo hello world | tee -a /tmp/file3 >/dev/null

Process substitution

Bash also supports the process of replacing (Note: Some Shell does not support the process of replacement).

Process substitution syntax:

<(cmd)
>(cmd)

Similar processes and command substitution, let cmd command is executed first, because they are in the Shell command line parsing stage of implementation.

Process substitution is performed asynchronously let cmd into the background and does not wait for cmd executed.

In fact, each process is replaced by a virtual file, but this file is (produced by cmd command <(cmd)) or cmd command read ( >(cmd)).

$ echo <(echo www.junmajinlong.com)
/dev/fd/63

Since the process of replacing the file, then it can be operated like a file. Such as read, it is redirected as a standard input data sources, and so:

# cmd做数据产生者
$ cat <(echo www.junmajinlong.com)   # 等价于cat /dev/fd/63
$ cat < <(echo www.junmajinlong.com) # 等价于cat </dev/fd/63

# cmd做数据接收者
$ echo hello world > >(grep 'llo')
$ echo hello world | tee >(grep 'llo') >(grep 'rld') >/dev/null

Test conditions statement

test command or functionally equivalent Bash built-in commands [ ]can do a conditional test, if the test result is True, the exit status is 0.

In addition, it can also be used [[]]to do the testing conditions, even let、$[]、$(())can do a conditional test, but it would not introduce here.

These conditions are commonly used in the tests if, while statement, also commonly used in cmd1 && cmd2 || cmd3command line format.

Example usage:

sh_file=test.sh
[ -x "$sh_file" ] && ./$sh_file || { echo "can't execute,exit...";exit 1; }
test -x "$sh_file" && ./$sh_file || { echo "can't execute,exit...";exit 1; }

[]The test conditions required for opening and closing brackets separated by spaces, otherwise a parser error.

No test content

[  ]
test

The absence of any test content, direct returns false.

true and false command

Returns true true commands directly, i.e., as exit status 0.

false commands directly returns false, i.e., non-zero exit status.

$ true
$ echo $?  # 0
$ false
$ echo $?  # 1

File type test

Conditional expression	meaning
-e file	If a file exists (exist)
-f file	If a file exists and is a regular file (file)
-d file	If a file exists and is a directory (directory)
-b file	If the file exists and block device to a block device
-c file	If a file exists and is a character device character device
-S file	If the file exists and is a socket file Socket
-p file	If a file exists and is a named pipe file FIFO (pipe)
-L file	If the file exists and is a file link (Link)

File attribute class test

Conditional expression	meaning
-r file	If the file exists and the current user-readable
-w file	If the file exists and the current user can write
-x file	If the file exists and the current user can perform
-s file	If the file size is greater than 0 and bytes exists, i.e., whether the document is detected non-empty file
-N file	If the file exists, and since it was last read whether modify

Comparison between the two documents

Conditional expression	meaning
file1 file2 -nt	If (newer than) than file1 file2 new judge
file1 file2 -ot	If (older than) judgment file1 is older than file2
-If file1 file2	(Equal file) to determine whether file1 file2 with the same file

Numerical comparison size

Conditional expression	meaning
int1 -eq int2	Two values ​​are equal (equal)
int1 -ne int2	Two numerical ranges (not equal)
int1 -gt int2	n1 is greater than n2 (greater than)
int1 -lt int2	n1 is smaller than n2 (less than)
int1 -ge int2	greater than or equal n1 n2 (greater than or equal)
int1 -le int2	n1小于等于n2(less than or equal)

字符串比较

条件表达式	含义
-z str	(zero)判定字符串是否为空？str为空串，则true
str <br>-n str	判定字符串是否非空？str为串，则false。注：-n可省略
str1 = str2 <br>str1 == str2	str1和str2是否相同，相同则返回true。"=="和"="等价
str1 != str2	str1是否不等于str2，若不等，则返回true
str1 > str2	str1字母顺序是否大于str2，若大于则返回true
str1 < str2	str1字母顺序是否小于str2，若小于则返回true

逻辑运算符

条件表达式	含义
-a或&&	(and)两表达式同时为true时才为true。<br>"-a"只能在test或[]中使用，&&只能在[[]]中使用
-o或||	(or)两表达式任何一个true则为true。<br/>"-o"只能在test或[]中使用，||只能在[[]]中使用
!	对表达式取反
( )	改变表达式的优先级，为了防止被shell解析，应加上反斜线转义( )

if语句

if test-commands; then
  consequent-commands;
[elif more-test-commands; then
  more-consequents;]
[else alternate-consequents;]
fi

test-commands既可以是test测试或[]、[[]]测试，也可以是任何其它命令，test-commands用于条件测试，它只判断命令的退出状态码是否为0，为0则为true。

例如：

if [ "$a" ];then echo '$a' is not none;else echo '$a' undefined or empty;fi

if [ ! -d ~/.ssh ];then
  mkdir ~/.ssh
  chown -R $USER.$USER ~/.ssh
  chmod 700 ~/.ssh
fi

if grep 'junmajinlong' /etc/passwd &>/dev/null;then
  echo 'User "junmajinlong" already exists...'
elif grep 'malongshuai' /etc/passwd &>/dev/null;then
  echo 'User "malongshuai" already exists...'
else
  echo 'you should create user,exit...'
  exit 1
fi

case

case常用于确定的分支判断。比如：

while [ ${#@} -gt 0 ];do
  case "$1" in
    start)
      echo start;;
    stop)
      echo stop
      ;;
    restart)
      echo restart
      ;;
    reload | force-reload)
      echo reload;;
    status)
      echo status;;
    *)
      echo $"Usage: $0 {start|stop|status|restart|reload|force-reload}"
      exit 2
  esac
done

case用法基本要求：

• 除最后一个分支外，每个分支都以;;结尾，否则出现分支穿透(所以;;不是必须的)
• 分支条件可以使用通配符号
• 分支条件中可使用竖线隔开多个条件，表示只要匹配其中之一就执行该分支
• 最后一般会定义一个能匹配其它任意条件的默认分支，即*)

下面是为ping命令自定义设计选项的脚本：

#!/bin/bash
while [ $1 ];do
  case "$1" in 
    -c|--count)
      count=$2
      shift 2
      ;;
    -t|--timeout)
      timeout=$2
      shift 2
      ;;
    -h|--host)
      host=$2
      shift 2
      ;;
    *)
      echo "wrong options or arguments"
      exit 1
  esac
done

ping -c $count -W timeout $host

执行：

$ chmod +x ping.sh
$ ./ping.sh -c 5 -t 2 

for循环

有两种for循环结构：

# 成员测试类语法
for i in word1 word2 ...;do cmd_list;done

# C语言for语法
for (( expr1;expr2;expr3 ));do cmd_list;done

成员测试类的for循环中，in关键字后是使用空格分隔的一个或多个元素，for循环时，每次从in关键字后面取一个元素并赋值给i变量。

例如：

$ for i in 1 2 3 4;do echo $i;done
1
2
3
4
$ for i in 1 2 "3 4";do echo $i;done
1
2
3 4

C语言型的for语法中，expr1是初始化语句，expr2是循环终点条件判断语句，expr3是每轮循环后执行的语句，一般用来更改条件判断相关的变量。

for ((i=1;i<=3;++i));do echo $i;done
1
2
3

while循环

while test_cmd_list;do cmd_list;done

while循环，开始时会测试test_cmd_list，如果测试的退出状态码为0，则执行一次循环体语句cmd_list，然后再测试test_cmd_list，一直循环，直到测试退出状态码非0，循环退出。

例如：

let i=1,sum=0;
while [ $i -le 10 ];do 
  let sum=sum+i
  let ++i
done

还有until循环语句，但在Shell中用的很少。

while循环经常会和read命令一起使用，read是Bash的内置命令，可用来读取文件，通常会按行读取：每次读一行。

例如：

cat /etc/fstab | while read line;do
  let num+=1
  echo $num: $line
done

上面的命令行中，首先cat进程和while结构开始运行，while结构中的read命令从标准输入中读取，也就是从管道中读取数据，每次读取一行，因为管道中最初没有数据，所以read命令被阻塞处于数据等待状态。当cat命令读完文件所有数据后，将数据放入到管道中，于是read命令从管道中每次读取一行并将所读行赋值给变量line，然后执行循环体，然后继续循环，直到read读完所有数据，循环退出。

但注意，管道两边的命令默认是在子Shell中执行的，所以其设置的变量在命令执行完成后就消失。换句话说，在父Shell中无法访问这些变量。比如上面的num变量是在管道的while结构中设置的，除了在while中能访问该变量，其它任何地方都无法访问它。

如果想要访问while中赋值的变量，就不能使用管道。如果是直接从文件读取，可使用输入重定向，如果是读取命令产生的数据，可使用进程替换。

while read line;do
  let num1+=1
  echo $num1: $line
done </etc/fstab
echo $num1

while read line;do
  let num2+=1
  echo $num2: $line
done < <(grep 'UUID' /etc/fstab)

Shell函数

Shell function can perform the same as a command, a structure which is a combination of one or more commands. Typically, a function can be defined for each function, the function contains the logic to implement all functions and commands related.

Since a plurality of commands may be combined, and thereafter can directly call defined in the current Shell, the function call having a plurality of times and the defined function code reuse.

Shell function defined style are the following categories:

function func_name {CMD_LIST}
func_name() {CMD_LIST}
function func_name() {CMD_LIST}

After the function definition, function name can be used directly to call a function, but also can transfer zero or more parameters to functions.

# 不传递参数
func_name
# 传递多个参数
func_name arg1 arg2 arg3

In function, the position of those variables will have a special significance:

• $1、$2、$3...: The first parameter passed to the function is stored in $1the second parameter stored in $2the ...
• $@And $*: Save all the parameters, the parameters separated by spaces
  • When not surrounded by double quotes, no difference between the two
  • When surrounded by double quotation marks, $@the respective elements are surrounded by double quotation marks, $*all the elements of the disposable is enclosed in double quotes

For example, define a function designed to set the variables and related agents:

proxy_addr=127.0.0.1:8118
function proxy_set {
  local p_addr=$1
  export http_proxy=$p_addr
  export https_proxy=$p_addr
  export ftp_proxy=$p_addr
}

# 调用函数
proxy_set $proxy_addr

# 各代理变量已设置
echo $http_proxy
echo $https_proxy
echo $ftp_proxy

The above code using a defined function local, it can be used inside a definition of the function represented by the local variables, local variables disappear after the function is executed, it does not affect the function of the external environment.

Further, the function return statement may be used to define the function's return value, to return each time the execution of the function, the function is terminated and the exit function. In Shell, the function's return value is actually exit status.

return [N]

If not specified N, then the default exit status 0.

E.g:

function sayhello {
  [ "$1" ] || { echo "give me a name please!"; return 1; }
  echo hello $1
}

sayhello "junma" ; echo $?
sayhello "jinlong" ; echo $?
sayhello ; echo $?