[Shell script] Process text files line by line && spaces to convert lines

Reference: http://www.cnblogs.com/dwdxdy/archive/2012/07/25/2608816.html

 

Stylistic files are often processed line by line. How to get each line of data in the Shell, then process the line of data, and finally read the next line of data, and process it in a loop. There are multiple workarounds as follows:

1. Completed by the read command.

The read command accepts input from standard input, or other file descriptors. After getting the input, the read command puts the data into a standard variable.

When using read to read a file, each invocation of the read command will read a "line" of text in the file.

When the file has no readable lines, the read command will exit with a non-zero status.

1 cat data.dat | while read line
2 do
3     echo "File:${line}"
4 done
5 
6 while read line
7 do 
8     echo "File:${line}"
9 done < data.dat

2. Use the awk command to complete

Awk is an excellent text processing tool that provides extremely powerful functions.

Use awk to read each line of data in the file, and you can do some processing on each line of data, and you can also process each column of data in each line of data separately.

1 cat data.dat | awk '{print $0}'
2 cat data.dat | awk 'for(i=2;i<NF;i++) {printf $i} printf "\n"}'

The first line of code outputs each row of data in data.dat, and the second line of code outputs the data from the second column in each row.

If it is a simple data or text file to read and display line by line, it is more convenient to use the awk command.

 

3. Use the for var in file command to complete

for var in file means that the variable var is looped in the file. The value separator is determined by $IFS.

1 for line in $(cat data.dat)
2 do 
3     echo "File:${line}"
4 done
5 
6 for line in `cat data.dat`
7 do 
8     echo "File:${line}"
9 done

如果输入文本每行中没有空格，则line在输入文本中按换行符分隔符循环取值．

如果输入文本中包括空格或制表符，则不是换行读取，line在输入文本中按空格分隔符或制表符或换行符特环取值．

可以通过把IFS设置为换行符来达到逐行读取的功能．

demo：

假设现需要读取如下的文件rollback_config.txt：

ROLLBACK_SERVICES:upserv  checkserv

ROLLBACK_VERSION:v1.1

使用   for line in `cat rollback_config.txt`; do echo "${line}"; done  读取的结果会是：

ROLLBACK_SERVICES:upserv 

checkserv

ROLLBACK_VERSION:v1.1

Obviously not what we want.

 

Solution:

IFS_old=$IFS
IFS=$'\n'
for line in  `cat  rollback_config`;do
echo "$line"
done;
IFS=$IFS_old

That's it!

 

 

 

The default value of IFS is: blank (including: spaces, tabs, newlines).