02-shell文本处理三剑客之sed

版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.csdn.net/qq_29622761/article/details/51603101

sed 是流编辑器，但是它不会修改源文件。

sed (流文本编辑器)
用法：sed OPTIONS… [SCRIPT] [INPUTFILE…]
-r: 使用扩展的正则表达式;
-n, –quiet, –silent: 不输出模式空间的内容；
-i：直接编辑原文件；
-e: -e ” -e ” -e ”
‘CMD1;CMD2;CMD3’
-f /path/to/sed_script：从指定的文件读取处理脚本

1 模式空间是什么？
模式空间其实就是内存中一个空间，归sed所有s。ed读取流文本文件的时候，每次读取一行，然后载入内存中，这个内存就成为模式空间。那么我们编辑的内容是属于模式空间中的内容，而不是源文件内容。

[root@hadoop1 shelltest]# sed '1,10d' /etc/fstab 
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap                    swap    defaults        0 0
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
[root@hadoop1 shelltest]# cat /etc/fstab 

#
# /etc/fstab
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 /                       ext4    defaults        1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot                   ext4    defaults        1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap                    swap    defaults        0 0
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
[root@hadoop1 shelltest]# 

sed ‘1,10d’ /etc/fstab 这一句语句的执行逻辑是，sed把内容读到模式空间中，判断是不是1——10的内容，是的话执行删除命令d,所以删除了1——10行内容，11行以后的内容就显示出来了。
sed的工作流程可以总结为：
第一步：提取一行内容到模式空间
第二步：针对模式空间中的内容执行编辑命令。
第三步：将模式空间中的内容进行输出。
第四步：清空模式空间。

2 地址定界：
#（行号）: 指定行
$: 最后一行
/regexp/：任何能够被regexp匹配到的行
\%regexp%: 任何能够被regexp匹配到的行；换用%为边界符号；
/regexp/I
\%regexp%I：匹配时忽略字符大小写；
addr1, addr2: 指定范围内的所有行：
1·,/regexp/
#（行号）,#（行号）
/regexp/,/regexp/
#(行号),+#
first~step：指定起始位置，以及步长 1~3

[root@hadoop1 shelltest]# sed '$d' /etc/fstab 

#
# /etc/fstab
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 /                       ext4    defaults        1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot                   ext4    defaults        1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap                    swap    defaults        0 0
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
[root@hadoop1 shelltest]# 

以上功能是删除了最后一行。

[root@hadoop1 shelltest]# sed '/dev/d' /etc/fstab 

#
# /etc/fstab
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 /                       ext4    defaults        1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot                   ext4    defaults        1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap                    swap    defaults        0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
[root@hadoop1 shelltest]# sed '\%/dev/mapper%d' /etc/fstab 

#
# /etc/fstab
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 /                       ext4    defaults        1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot                   ext4    defaults        1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap                    swap    defaults        0 0
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
[root@hadoop1 shelltest]# sed '1+2d' /etc/fstab 
sed: -e expression #1, char 2: unknown command: `+'
[root@hadoop1 shelltest]# sed '1,+2d' /etc/fstab 
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 /                       ext4    defaults        1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot                   ext4    defaults        1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap                    swap    defaults        0 0
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
[root@hadoop1 shelltest]# sed '1~2d' /etc/fstab 
#
# Created by anaconda on Tue Jan 26 09:32:28 2016
# Accessible filesystems, by reference, are maintained under '/dev/disk'
#
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot                   ext4    defaults        1 2
tmpfs                   /dev/shm                tmpfs   defaults        0 0
sysfs                   /sys                    sysfs   defaults        0 0
[root@hadoop1 shelltest]#

基本正则表达式元字符：
    字符：.  []  [^]
    次数：*, \+, \?, \{i,j\}
    分组：\(\)
    锚定：^, $, \<, \>
    或者：|
    后向引用：\1, \2, ...

编辑命令：
    注意：命令之前可以使用!表示取反操作；

    =: 显示行号
        # sed -n '\%/dev/mapper%=' /etc/fstab

    d: 删除匹配到的行；
    p: 打印模式空间中的内容；
        通常与-n选项一同使用；
    a \text: append，
        a \text1\ntext2\ntext3: 附加多行文本
    i \text：insert, 
        i \text1\ntext2: 插入多行文本
    c \text: change，把匹配到的行替换为给定的文本
    s///: 替换命令
        g: 全局替换
        i: 不区分大小写

[root@hadoop1 shelltest]# sed -n '\%/dev/%=' /etc/fstab 
6
12
13
[root@hadoop1 shelltest]# sed -n '5,7p' /etc/fstab
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
[root@hadoop1 shelltest]# 

追加：

[root@hadoop1 shelltest]# sed '1a \xielaoshi' /etc/fstab 

xielaoshi
#
# /etc/fstab
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 /                       ext4    defaults        1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot                   ext4    defaults        1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap                    swap    defaults        0 0
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
[root@hadoop1 shelltest]# 

还可以同时执行多条命令：

[root@hadoop1 shelltest]# sed '1d;2p;3=' /etc/fstab 
#
#
3
# /etc/fstab
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 /                       ext4    defaults        1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot                   ext4    defaults        1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap                    swap    defaults        0 0
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
[root@hadoop1 shelltest]# 

替换：

[root@hadoop1 shelltest]# sed '3c \bbbb' /etc/fstab 

#
bbbb
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 /                       ext4    defaults        1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot                   ext4    defaults        1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap                    swap    defaults        0 0
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
[root@hadoop1 shelltest]#

[root@hadoop1 shelltest]# sed 's/dev/xielaoshi/' /etc/fstab 

#
# /etc/fstab
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# Accessible filesystems, by reference, are maintained under '/xielaoshi/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 /                       ext4    defaults        1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot                   ext4    defaults        1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap                    swap    defaults        0 0
tmpfs                   /xielaoshi/shm                tmpfs   defaults        0 0
xielaoshipts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
[root@hadoop1 shelltest]#

全局替换：

[root@hadoop1 shelltest]# sed 's/dev/xielaoshi/g' /etc/fstab 

#
# /etc/fstab
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# Accessible filesystems, by reference, are maintained under '/xielaoshi/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 /                       ext4    defaults        1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot                   ext4    defaults        1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap                    swap    defaults        0 0
tmpfs                   /xielaoshi/shm                tmpfs   defaults        0 0
xielaoshipts                  /xielaoshi/pts                xielaoshipts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
[root@hadoop1 shelltest]#

3 练习
练习1：删除/boot/grub/grub.conf文件中所以空白开头的行行首的空白；

[root@hadoop1 shelltest]# cat /boot/grub/grub.conf 
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE:  You have a /boot partition.  This means that
#          all kernel and initrd paths are relative to /boot/, eg.
#          root (hd0,0)
#          kernel /vmlinuz-version ro root=/dev/sda2
#          initrd /initrd-[generic-]version.img
#boot=/dev/sda
default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title CentOS (2.6.32-431.el6.x86_64)
    root (hd0,0)
    kernel /vmlinuz-2.6.32-431.el6.x86_64 ro root=UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 rd_NO_LUKS rd_NO_LVM LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto  KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet
    initrd /initramfs-2.6.32-431.el6.x86_64.img
[root@hadoop1 shelltest]# sed 's/^[[:space:]]\+//g' /boot/grub/grub.conf
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE:  You have a /boot partition.  This means that
#          all kernel and initrd paths are relative to /boot/, eg.
#          root (hd0,0)
#          kernel /vmlinuz-version ro root=/dev/sda2
#          initrd /initrd-[generic-]version.img
#boot=/dev/sda
default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title CentOS (2.6.32-431.el6.x86_64)
root (hd0,0)
kernel /vmlinuz-2.6.32-431.el6.x86_64 ro root=UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 rd_NO_LUKS rd_NO_LVM LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto  KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet
initrd /initramfs-2.6.32-431.el6.x86_64.img
[root@hadoop1 shelltest]# 

练习2：删除/etc/fstab文件中所有以#开头，后跟至少一个空白字符的行的行首的#和空白字符；

[root@hadoop1 shelltest]# sed 's/^#[[:space:]]\+//g' /etc/fstab 

#
/etc/fstab
Created by anaconda on Tue Jan 26 09:32:28 2016
#
Accessible filesystems, by reference, are maintained under '/dev/disk'
See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 /                       ext4    defaults        1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot                   ext4    defaults        1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap                    swap    defaults        0 0
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
[root@hadoop1 shelltest]# 

练习3：echo一个文件路径给sed命令，取出其基名；取出其目录名；

[root@hadoop1 shelltest]# echo "/etc/sysconfig/" | sed 's@[^/]\{1,\}/\?$@@g'
/etc/
[root@hadoop1 shelltest]# 

练习4：替换/etc/inittab文件“id:数字:initdefault:”一行中的那个数字为6；

[root@hadoop1 shelltest]# cat /etc/inittab 
# inittab is only used by upstart for the default runlevel.
#
# ADDING OTHER CONFIGURATION HERE WILL HAVE NO EFFECT ON YOUR SYSTEM.
#
# System initialization is started by /etc/init/rcS.conf
#
# Individual runlevels are started by /etc/init/rc.conf
#
# Ctrl-Alt-Delete is handled by /etc/init/control-alt-delete.conf
#
# Terminal gettys are handled by /etc/init/tty.conf and /etc/init/serial.conf,
# with configuration in /etc/sysconfig/init.
#
# For information on how to write upstart event handlers, or how
# upstart works, see init(5), init(8), and initctl(8).
#
# Default runlevel. The runlevels used are:
#   0 - halt (Do NOT set initdefault to this)
#   1 - Single user mode
#   2 - Multiuser, without NFS (The same as 3, if you do not have networking)
#   3 - Full multiuser mode
#   4 - unused
#   5 - X11
#   6 - reboot (Do NOT set initdefault to this)
# 
id:5:initdefault:
[root@hadoop1 shelltest]# sed 's@id:[0-9]:initdefault:@id:6:initdefault@' /etc/inittab 
# inittab is only used by upstart for the default runlevel.
#
# ADDING OTHER CONFIGURATION HERE WILL HAVE NO EFFECT ON YOUR SYSTEM.
#
# System initialization is started by /etc/init/rcS.conf
#
# Individual runlevels are started by /etc/init/rc.conf
#
# Ctrl-Alt-Delete is handled by /etc/init/control-alt-delete.conf
#
# Terminal gettys are handled by /etc/init/tty.conf and /etc/init/serial.conf,
# with configuration in /etc/sysconfig/init.
#
# For information on how to write upstart event handlers, or how
# upstart works, see init(5), init(8), and initctl(8).
#
# Default runlevel. The runlevels used are:
#   0 - halt (Do NOT set initdefault to this)
#   1 - Single user mode
#   2 - Multiuser, without NFS (The same as 3, if you do not have networking)
#   3 - Full multiuser mode
#   4 - unused
#   5 - X11
#   6 - reboot (Do NOT set initdefault to this)
# 
id:6:initdefault
[root@hadoop1 shelltest]# 

4 高级编辑命令：
h: 用模式空间中的内容覆盖保持空间中的内容；
H：把模式空间中的内容追求至保持空间；
g: get, 从保持空间取回数据至模式空间，覆盖模式空间中原有的内容；
G：GET, 从保持空间取回数据至模式空间，追加于模式空间中现有的内容后面；
x: 把模式空间和保持空间中的内容进行交换；
n: 读取匹配到的行下一行至模式空间；
N: 追加读匹配到的行下一行至模式空间；
d: 删除模式空间中的内容；
D：删除多行模式空间中首行；
sed ‘1!G;h;$!d’ FILE

[root@hadoop1 shelltest]# sed '1!G;h;$!d' /etc/issue

Kernel \r on an \m
CentOS release 6.5 (Final)
[root@hadoop1 shelltest]# cat /etc/issue
CentOS release 6.5 (Final)
Kernel \r on an \m

[root@hadoop1 shelltest]# 

    显示文件的偶数行：sed -n 'n;p' FILE;     sed -n '2~2p' FILE
        1
        2
        3
        4

    显示文件的奇数行：sed -n 'p;n' FILE;     sed -n '1~2;p' FILE

练习：用sed将文件的奇数行与偶数行合并为一行，中间用“&&”隔开；

[root@hadoop1 shelltest]# sed '$!N;s@\n@\&\&@g' /etc/issue
CentOS release 6.5 (Final)&&Kernel \r on an \m

[root@hadoop1 shelltest]# 

5 课后练习：说明以下sed命令的意义：
(1) sed ‘$!N;$!D’ /etc/issue

[root@hadoop1 shelltest]# sed '$!N;$!D' /etc/issue
Kernel \r on an \m

[root@hadoop1 shelltest]# cat /etc/issue
CentOS release 6.5 (Final)
Kernel \r on an \m

[root@hadoop1 shelltest]#

(2) sed ‘$!d’ /etc/issue
删除非尾行。
(3) sed ‘G’ /etc/issue

[root@hadoop1 shelltest]# sed 'G' /etc/issue
CentOS release 6.5 (Final)

Kernel \r on an \m



[root@hadoop1 shelltest]# 

因为保持空间里面都是空的，所以追加到模式空间后面全是空行。
(4) sed ‘/^$/d;G’ /etc/issue

[root@hadoop1 shelltest]# sed '/^$/d;G' /etc/issue
CentOS release 6.5 (Final)

Kernel \r on an \m

[root@hadoop1 shelltest]# 

(5) sed ‘n;d’ /etc/issue

[root@hadoop1 shelltest]# sed 'n;d' /etc/issue
CentOS release 6.5 (Final)

[root@hadoop1 shelltest]# 

(6) sed -n ‘1!G;h;$p’ /etc/issue

[root@hadoop1 shelltest]# cat /etc/issue
CentOS release 6.5 (Final)
Kernel \r on an \m

[root@hadoop1 shelltest]# 

6 参考资料
模式空间和保持空间的理解：
什么是模式空间和保持空间

O(∩∩)O嗯!O(∩∩)O嗯!O(∩_∩)O嗯!今天就写到这里，休息一下。如果你看到此文，想进一步学习或者和我沟通，加我微信公众号：名字：五十年后 。