Shell Programming - [07] of Linux sed stream editor sed command

Shell Programming [series] https://blog.csdn.net/ilo114/article/category/8961526

Introduction

• sed (Stream Editor), stream editor. Or the standard output file are processed line by line.

Syntax

• Standard output processing

sed [选项]... {脚本} [文件]...

• File Handling

stdout | sed [选项]... {脚本}

file

• The following command will be carried out under the file containing the document reads as follows directory.

i love python
lovelove python
loooove
I LOVE PYTHON
I LOVE pYtHoN
py.*

sed options Use Case

No parameters directly using p (print) command

sed "p" file

i love python
i love python
lovelove python
lovelove python
loooove
loooove
I LOVE PYTHON
I LOVE PYTHON
I LOVE pYtHoN
I LOVE pYtHoN
py.*
py.*

-n cancel the automatic print mode empty

• It may be a bit mask what is meant by this sentence is in centOS7 sed to enter the world of tips.
• First explain, when using the sed command will first automatically prints the original text, text processing and then output the matching results.
• -n it can simply not understand the original text output, only output.
• Example:

sed -n "p" file

i love python
lovelove python
loooove
I LOVE PYTHON
I LOVE pYtHoN
py.*

• Print Matches

sed -n "/python/p" file

i love python
lovelove python

-e script, add the "script" to the list of running programs

• That is, when a progressive process of matching a variety of circumstances, not only a write -e
• Such as matching uppercase and lowercase

sed -n -e '/python/p' -e '/PYTHON/p' file

i love python
lovelove python
I LOVE PYTHON

-r expand the use of regular expressions

sed -nr "/python|PYTHON/p" file
sed -nE "/python|PYTHON/p" file
# 两种都可以

i love python
lovelove python
I LOVE PYTHON

-f script file, add the "script" to the program run list

-i operation source file

sed -i 's/love/like/g' file
# s是替换 love变成like  g是每行的所有匹配字符，
# 如果不加g 则会匹配到第一个符合的字符替换了就结束当前行的替换

• No output

sed common usage Pattern

Matching a row

sed -n "5p" file

I LOVE pYtHoN

A row from the start to the match in a row

sed -n '2,5p' file

lovelove python
loooove
I LOVE PYTHON
I LOVE pYtHoN

A match starting line number, and a set offset row

sed -n '2,+2p' file

• Number of rows processed from the beginning cardinality offset + 1 row

lovelove python
loooove
I LOVE PYTHON

Processing line where the regular expression

sed -n '/^I LOVE/p' file

I LOVE PYTHON
I LOVE pYtHoN

The number of rows to start the matching condition 1 to condition 2 is completed row

sed -n "/lovelove/,/py\.\*/p" file

lovelove python
loooove
I LOVE PYTHON
I LOVE pYtHoN
py.*

A row from the start to the end of line 2 of the matching condition

sed -n "4,/pYtHoN/p" file

I LOVE PYTHON
I LOVE pYtHoN

The number of rows to start a match condition to a line end

sed -n "/pYtHoN/, 6s/LOVE/love/gp" file

I love pYtHoN

Note: If the number of lines range processing, the end of the file if the condition is not treated it would have been to the last line

sed -n "3,/abc/p" file

loooove
I LOVE PYTHON
I LOVE pYtHoN
py.*

sed in editing commands

p print print

After a row append the

sed '/loooove/a bbbbbbbbbb' file

i love python
lovelove python
loooove
bbbbbbbbbb
I LOVE PYTHON
I LOVE pYtHoN
py.*

i row before adding

sed '/loooove/i aaaaaaaaa' file

i love python
lovelove python
aaaaaaaaa
loooove
I LOVE PYTHON
I LOVE pYtHoN
py.*

r external file read line after adding

• list file

1 line
2 line

sed '/lovelove/r list' file 

i love python
lovelove python
1 line
2 line
loooove
I LOVE PYTHON
I LOVE pYtHoN
py.*

w matching lines written to the external file

sed -n '/I LOVE/w aaa' file  && cat aaa```

• Note aaa can not exist if there is content inside will be cleared

I LOVE PYTHON
I LOVE pYtHoN

d delete

sed '1，3d' file

• Printing is executed does not modify the source file to modify the source file needs -i

I LOVE PYTHON
I LOVE pYtHoN
py.*

The s / old / new old replace a line

s / old / new / 2 rows in the second alternative

The s / old / new / g replace all old row

s / old / new / 2g Alternatively the second and all subsequent

All the old s / old / new / ig Alternatively row, ignoring case

• Replace command demo

sed 's/o/AA/i2g' file

i love pythAAn
lovelAAve pythAAn
loAAAAAAve
I LOVE PYTHAAN
I LOVE pYtHAAN
py.*

sed 's/py/&OOO/ig' file

• Ampersand is a reference to the content that matches

i love pyOOOthon
lovelove pyOOOthon
loooove
I LOVE PYOOOTHON
I LOVE pYOOOtHoN
pyOOO.*

sed applications

• Process a configuration file my.conf Mysql text, enter the article has several buckets ([] of segments), each segment has several configurations,

#!/bin/bash
#

FILE_NAME=./my.cnf

function get_all_segments
{
	echo "`sed -n '/\[.*\]/p' $FILE_NAME  | sed -e 's/\[//g' -e 's/\]//g'`"
}

function count_items_in_segment
{
	items=`sed -n '/\['$1'\]/,/\[.*\]/p' $FILE_NAME | grep -v "^#" | grep -v "^$" | grep -v "\[.*\]"`
	
	index=0
	for item in $items
	do
		index=`expr $index + 1`
	done

	echo $index

}

number=0

for segment in `get_all_segments`
do
	number=`expr $number + 1`
	items_count=`count_items_in_segment $segment`
	echo "$number: $segment  $items_count"
done

• Text my.conf

# this is read by the standalone daemon and embedded servers
[client]
port=3306
socket=/tmp/mysql.socket

#ThisSegmentForserver
[server]
innodb_buffer_pool_size=91750M
innodb_buffer_pool_instances=8
innodb_buffer_pool_load_at_startup=1
innodb_buffer_pool_dump_at_shutdown=1
innodb_data_file_path=ibdata1:1G:autoextend
innodb_flush_log_at_trx_commit=1
innodb_log_buffer_size=32M
innodb_log_file_size=2G
innodb_log_files_in_group=2
innodb_max_undo_log_size=4G
innodb_undo_directory=undolog
innodb_undo_tablespaces=95

#thisisonlyforthemysqldstandalonedaemon
[mysqld]
port=3306
socket=/tmp/mysql.sock
basedir=/usr/local/mysql
datadir=/data/mysql
pid-file=/data/mysql/mysql.pid
user=mysql
bind-address=0.0.0.0
sort_buffer_size=16M
join_buffer_size=16M
thread_cache_size=3000
interactive_timeout=600
wait_timeout=600

#ThisSegmentFormysqld_safe
[mysqld_safe]
log-error=/var/log/mariadb/mariadb.log
pid-file=/var/run/mariadb/mariadb.pid
max_connections=1000
open_files_limit=65535
thread_stack=512K
external-locking=FALSE
max_allowed_packet=32M

#thisisonlyforembeddedserver
[embedded]
gtid_mode=on
enforce_gtid_consistency=1
log_slave_updates
slave-rows-search-algorithms='INDEX_SCAN,HASH_SCAN'
binlog_format=row
binlog_checksum=1
relay_log_recovery=1
relay-log-purge=1


#usethisgroupforoptionsthatolderserversdon'tunderstand
[mysqld-5.5]
key_buffer_size=32M
read_buffer_size=8M
read_rnd_buffer_size=16M
bulk_insert_buffer_size=64M
myisam_sort_buffer_size=128M
myisam_max_sort_file_size=10G
myisam_repair_threads=1
lock_wait_timeout=3600
explicit_defaults_for_timestamp=1
innodb_file_per_table=1