1. Local variables and environment variables
Variable Types | Defined form | Position statement | Display command | Scope |
---|---|---|---|---|
Local variables | VARNAME=value | Command line or shell scripts | set (display all variables) | This user, the process |
Environment Variables | export VARNAME=value | ~ / .Bash_profile (valid user currently logged in) / etc / profile (for all user login valid) | env (display only environment variables) | All processes for all users |
2. Script Debugging
- A law
#!/bin/bash -x
- Act II
set -x
要调试的几行代码
set +x
3. Exit state
exit n # n=0正常,n!=0出错,n~[0-255]
脚本中没有exit命令则以最后一条命令的退出码为整个脚本的退出码
$? 上一条命令的退出码
4. Variable connection
Linux in the $ {} variables considered as a whole. $ {} Will content subsequent connection processing.
BABY="cerana" 注意赋值的=号前后不要留白,否则变量将被当做shell命令执行,而后边的内容解释为参数。
BABY_svn="hello"
echo ${BABY}_svn 显示 cerana_svn
echo ${BABY}:babala 显示cerana:babala
echo ${BABY_svn} 显示 hello
5. shell script commands in common
:与true语句功能相同,产生测试结果为真的结果
如:
while :
do
......
done
echo -n "Enter your name: " -n不输出换行符
read name 将控制台输入赋给变量name
或者
read -p "Enter your name: " name
expr 整数运算
expr 1 + 1
expr 1 - 1
expr 3 \* 4
expr 3 / 4 为0,是整除
echo 2^3 | bc 指数运算
expr "abc" = "Abc" 返回0
expr "abc" = "abc" 返回1
echo "scale=2;1.2*7.8" | bc scale控制小数位数
echo "scale=2;13/2" | bc
n=1
let n+=3 支持+=, -=, *=, /=, %=
6. numerical constant
0 represents the beginning of an octal number
0x or 0X opened hexadecimal number
7. Replace command
` ... ` 或者$( ... )
8. test statement
test expression
[ expression ] 注意方括号内部前后要留白
[[ expression ]] 注意方括号内部前后要留白(扩展形式)
9. File test
Testing Statement | description |
---|---|
-e file | A given file exists |
-r file | Given file exists, and the current user-readable |
-w file | A given file exists and the current user can write |
-x file | A given file exists and the current user can perform |
-s file | Given the existence of the file and the file is not empty |
-f file | A given file exists and is a regular file |
-d file | A given file exists and is a directory |
-L file | Given file exists and is a symbolic link |
-c file | Given file exists and is a character special file |
-b file | Given file exists, and is a block special file |
-p file | Given file exists and is a named pipe file |
BACKUPDIR=/data/backup
[ -d ${BACKUPDIR} ] || mkdir -p ${BACKUPDIR}
[ ! -d ${BACKUPDIR}/bin ] && rm -rf ${BACKUPDIR}/bin
10. String Test
Testing Statement | description |
---|---|
-with page | Given string length is 0, the string enclosed in double quotes |
-n str | Given string length is not zero, the string enclosed in double quotes |
s1=s2 | Strings are equal |
s1!=s2 | Ranging from string |
s1<s2 | String comparison |
s1>s2 | String comparison |
11. Numerical Comparison
Testing Statement | description |
---|---|
-eq | equal |
-born | not equal to |
-gt | more than the |
-lt | Less than |
-give | greater or equal to |
-the | Less than or equal |
12. The logical operation
Testing Statement | description |
---|---|
( ) | combination |
! exp | non- |
-a 或 && | versus |
-o or ││ | or |
13. Functions
function fname(){
cmd;
[return 数值;] 如果不加此句则将最后一条命令的执行结果作为退出码
}
example
#!/bin/bash
function traverse(){
dir=$(echo $1 | sed "s|\/$||")
for file in `ls $dir`
do
if [ -d $dir"/"$file ]
then
traverse $dir"/"$file
else
echo $dir"/"$file
fi
done
}
traverse $1
14. The string interception
command | Explanation |
---|---|
$ {Var ## * str} | From left to right after the last interception string str |
$ {Var # * str} | 从左向右截取第一个str后的字符串 |
${var%%str*} | 从右向左截取最后一个str后的字符串 |
${var%str*} | 从右向左截取第一个str后的字符串 |
${var:n1:n2} | 截取变量var从n1开始的n2个字符(n1从0开始) |
15. 控制结构
- if分支
if condition1
then
cmd1
else
cmd2
fi
或者
if condition1
then
cmd1
elif condition2
cmd2
else
cmd3
fi
- case分支
case value in
mode1)
cmd1
;;
mode2|mode3)
cmd2
;;
mode4)
cmd3
;;
*)
cmd4
;;;
esac
- 循环
for var in list
do
cmd
done
list可以为:
列表,字符串,文件名,{100..200}
while conditions
do
cmd
done
循环常用形式:
for LINE in `cat file`
或者
cat $FILE | while read LINE
16. sed基本用法
sed [-nefr] [n1, n2] 'actions'
例子:
删除file的2~5行后再显示
cat -n file | sed '2,5d'
在文件file的第二行的下一行添加“Hello, world”字符串
cat -n file | sed '2a Hello, world'
注意:行末使用 \ 转义回车换行符达到续行的效果。
将2~5行的内容替换为baby girl
cat -n file | sed '2,5c baby girl'
只显示5~7行的内容
cat -n file | sed -n '5,7p'
实现字符串函数 trim() 的效果
sed 's/^\s*//' totrim.txt |sed 's/\s*$//'>trimed.txt
sed 's/^\s*//;s/\s*$//' totrim.txt>trimed.txt
sed -e 's/^\s*//' -e 's/\s*$//' totrim.txt>trimed.txt
设置去除,不要注释行
cat /etc/man.config | grep 'MAN' | sed 's/#.*$//g' | sed '/^$/d'
下加一行(&指代pattern)
sed 's/pattern/&\n/g' file
上加一行
sed 's/pattern/\n&/g' file
变量替换
sed -e "s/$var1/$var2/g" file
在第一行前插入文本---BEGIN---,在最后一行插入文本---END---。
cat test4.txt | sed '1 i\---BEGIN---' | sed '$ a\---END---'
在正则匹配行前插入string
sed -i '/pattern/ i "string"' file
在正则匹配行后插入string
sed -i '/pattern/ a "string"' file
将家目录~下所有文件中的abc都改为cba(g指一行中的每一列都要处理)
sed -i 's/abc/cba/g' `grep abc -rl ~`
其中:-r表示递归,但不包括符号连接,-l表示只列出匹配的文件名(绝对路径)。
使用其他分隔符文本替换
sed -i 's@http://[^.]*.1234.com@/home/html/www.2321.com@g' file
行首添加注释#
sed -i '1,20s/^/#/' file 只处理第一个匹配
取消行首的#
sed -i '1,20s/^#//' file 只处理第一个匹配
提取日志记录
sed -n '/2018-11-08 10:15:00/, /2018-11-11 10:15:00/p' logfile
17. awk基本用法
awk [-F field-seperator] 'pattern {action}' file
BEGIN 初始化
数据处理
END 收尾处理
注意:需使用单引号,以防被shell解释。
pattern和{action}可以省略其中之一,但不能同时省略。
action 中的语句以 ; 分隔。
awk -F[:" "] ... -F[:" "]表示以:和空格作为分隔符
默认分隔符是空格
内置变量
FS 输入字段分隔符
RS 输入记录分隔符
OFS 输出字段分隔符
ORS 输出记录分隔符
NF 输入字段个数
NR 输入当前记录编号
$0 整条记录
$n 第n条记录,n=1,2,3,...
pattern{action}模式
/^$/ {print "blank line"}
$5~/MA/ {print $1" "$3}
例子:
抓取时间范围是Jun 30 10:30:00到Jun 30 11:00:00的日志记录
cat /var/log/secure | grep "Jun 29" | awk '$3>="22:00:00" && $3<="22:30:00"'
类似地有
cat log1 | awk -F'\t' '$1$2>="2013/08/16 01:16:11.111" && $1$2<="2013/08/16 01:25:22.222"'
转换时间格式
date -d "Jun 23 12:22:21" "+%F %T" 转换时间格式
2018-06-23 12:22:21 回显
echo "Jun 23 12:22:21" | awk '{NF=3;cmd="date -d \""$0"\" \"+%F %T\"";cmd|getline dt;print dt;close(cmd)}'
统计
cat file | awk -F: 'BEGIN{count=0} $2>0{count+=$9} END{print count}'
- 常用运维脚本
- ftp备份
ftp -i -n -v << !
open ${HOST}
user ${USERNAME} ${PASSWORD}
bin
cd ${OLDDATE}
mdelete *
cd ..
rmdir ${OLDDATE}
mkdir ${DATE}
cd ${DATE}
mput *
bye
!
- 批量添加/删除用户
用户信息文件users.txt
cerana1:13888298736
cerana2:13888298737
cerana3:13888298738
cerana4:13888298739
cerana5:13888298740
批量添加用户的脚本
#!/bin/bash
#把本shell脚本和账号文件放在同一个目录下
for line in `cat users.txt` ; do
username=$(echo $line | awk -F: '{print $1}')
password=$(echo $line | awk -F: '{print $2}')
useradd $username
echo "User $username was added successfully"
#passwd --stdin表示不交互,直接输入密码
echo $password | passwd --stdin $username
#强制用户第一次登陆就必须更改密码
chage -d 0 $username
#定义密码有效期30天
chage -M 30 $username
done
echo "Finish!"
批量删除用户的脚本
#!/bin/bash
for line in `cat users.txt` ; do
username=$(echo $line | awk -F: '{print $1}')
#同时删除用户home目录,邮箱等资源
userdel -r $username
echo "User $username was deleted successfully"
done
echo "Finish!"