[Make it clear, to say it! Linux] essential tool for employees (two) - AWK text-processing tool to combat

Contents:
(ix) merge two files
(X) to set permissions on NFS
(XI) intersection of
(xii) the number of characters that appear Statistics
(xiii) sum
(xiv) the real case

---

(Ix) merge two files
(9.1) First we define a1 and a2 two files in the system, and now we need to use the second field of a1 to a2 than the second field, if the same, then a1 of the first field to the third field is added file a2, a2 eventually to print out information corresponding to the file as a template, if a plurality of files can be combined to consider an array
# awk -F "[,]" 'NR == the FNR {a [$ 2] = $. 1} NR! = the FNR {Print $ 0, a [$ 2]} 'A1 A2



(9.2) we create two files b1 and b2, and now we want to two files into the following format, we find the second row and first column b2, b1 is a common item, then we are not at the time when NR FNR will operate printing, remember to use between fields ":" separated by colons.
The results we want to print as follows:
Tom: 001: AA
Tom: 001: BB
Bob: 002: CC
Bob: 002: dd
# awk -F:! 'NR == the FNR {A [$ 2] = $ 0} NR = the FNR { Print a [$ 1] ":" $ 2} 'B1 B2



(9.3) we create a c1 and c2 file, the same if c1 $ 1 and c2 of $ 3, the same as the last three characters c1, $ 2 and c2 is $ 4, c1, c2 the same $ 3 and $ 6, then c2 in qualifying rows of print, if the demand is more common screening columns, as long as we put multiple columns merge together to compare. We found that in column 1 c1 plus the first two after the first three plus c2 file plus the first four of the six plus three characters of the three, if it is the same, then it is in line with demand, you can be c2 files in the current line printed.
# Awk 'NR == FNR {a [$ 1] = $ 1 $ 2 $ 3} NR = FNR {X = substr ($ 4,3);! Y = $ 3X $ 6; if (Y == a [$ 3]) {print} } '



(9.4) we create a file d1 and d2, d2 at this time we want to replace the file in the second field to the second field d1 files, we must first pay attention to the first with d1 d2 then use when using the file name, specify input and output for the delimiter ":" colon, and then perform the replace operation in the second document d2 field d1 to the file of the second field.
# Awk -F: 'BEGIN {FS = OFS = ":"}! NR == FNR {a [$ 1] = $ 2} NR = FNR {$ 2 = a [$ 1]; print}' d2 d1

---

(X) in the same row as a single line
(10.1) we create a text file aa1, we now need is to merge a file with the same line in the field in a row, for example, the first line of the first field is 27, the second line 27 is the first field, then information of the second line of the second field of direct printing positions of the first row in the three fields, the first field of the third row is 26, so the next line to print directly to forth. First, the use of NR is equal to 1 is determined, if NR is a current row and a first print line of the first field is assigned to the variable x, then the second row, NR found not equal to 1, but the second row a field equal to the value of the variable x, so at this time Print "% s" format using printf, and then deal with the third row, we found that in line with "else if ($ 1! = x)" this condition, so this time we will first Print a newline character "\ n" and then print the entire line, and assign the current line of a field to the variable x, so that we can achieve the basic format we need a.
# Awk 'NR == 1 {printf "% s", $ 0; x = $ 1}! NR = 1 {if ($ 1 == x) {printf "% s", $ 2} else if (! $ 1 = x) { printf "\ n% s", $ 0; x = $ 1}} END {printf "\ n"} 'aa1

---

（十一）求交集
(11.1)现在我们创建了两个文件a1和a2，现在要求打印a1文件中的一些行，a1的第2字段出现在a2的第1个字段中，现在如果要求a的某部分是否出现在b的某部分里的时候，那么我们就把b的那部分设置为数组的下标，然后判断a的那部分是否是下标的一个元素。此时则应该为“awk -F”[,/]” ‘’ a2 a1”
# awk -F"[,/]" 'NR==FNR{a[$1]=$0}NR!=FNR{if($2 in a){print $0}}' a2 a1



(11.2)现在我们创建两个文件b1和b2，现在我们的需求是打印b1的一些行，b1的$1出现在b2的第1个字段里，找到那些行之后，并把b2里对应的这些行的$2部分进行求和，我们可以先在b2文件中将所有的第1字段的第2个字段值进行相加操作，最后在b1文件行打印出来时也把相加的值一同打印出来。
# awk -F, 'NR==FNR{a[$1]+=$2}NR!=FNR{if($1 in a){printf "%s,%f\n",$0,a[$1]}}' b2 b1

---

（十二）统计字符出现的次数
(12.1)我们有一个文件a1，现在我们需要统计每个字符在文本文件中出现的次数，此时我们的思路是遇到一个字段就将其设置为一个元素的下标，当我们已经定义了一个元素时，如果下次再遇到这个下标的元素的时候，则加上1次。我们首先是使用循环遍历每一行，使用遍历的字段作为数组的下标，如果后面出现了相同的字段，则将统计的结果加1，然后在END中去遍历数组中每个元素当前的数值，此时就可以得到每个字符在文本文件中出现的次数。
# awk '{for(i=1;i<=NF;i++){a[$i]++}}END{for(x in a){print x"一共出现了"a[x]"次"}}' a1

---

（十三）求和
(13.1)在我们的内存中正常是会将内存的数据写入到硬盘中的，因此内存的数据和硬盘的数据是同步的，我们称之为干净数据，但是有时候在我们的内存中有些数据修改完后并没有写入到硬盘中去，内存中的数据和硬盘数据并不同步，此时这类数据我们称之为脏数据，如果脏数据没有及时写入到硬盘，那么当计算机重启之后没有写入到硬盘的数据就会丢失了。现在我们的需求是计算出某个进程一共有多少个干净页，有多少个脏页，我们是可以从/proc/1/smaps的系统映射文件中查看到我们所需的内存相关信息的，我们可以使用模式匹配的方式，将干净页和脏页分别进行统计，在之前的基础上进行连加操作。
# cat /proc/1/smaps
# cat /proc/1/smaps | awk '/Shared_Clean/{AA+=$2}/Shared_Dirty/{BB+=$2}END{print "干净页总合是"AA"KB";print "脏页的总合是"BB"KB"}'


(13.2)现在我们有两个文件a1和a2，因为我们发现a1文件和a2文件的列数是一样的，所以现在我们的需求是希望将a1文件和a2文件中对应的列数进行相加得到一个相加后的一行数组。解决这个问题的思路我们是把一个文件里所有的数字放在一个数组里保存，然后读取第二个文件，把第二个文件里所有的数组保存，然后分别相加（图13-5）。当然我们也可以使用getline函数解决问题，首先使用for循环获取a1文件的所有值，然后使用getline读取a2文件读取的行替换到当前所在行的信息，最后使用for循环去遍历数组a与当前a2文件中的每个字段，并进行相加的操作，此时也可以得到我们所需的结果（图）。
# awk 'NR==FNR{for(i=1;i<=NF;i++){a[i]=$i}}NR!=FNR{for(i=1;i<=NF;i++){b[i]=$i;printf "%s ",a[i]+b[i]}}END{printf "\n"}' a1 a2
# awk '{for(i=1;i<=NF;i++){a[i]=$i};getline < "a2";for(i=1;i<=NF;i++){printf "%s ",a[i]+$i}printf "\n"}' a1

---

(Xiv) the real case
(14.1) achieved card IP
# ifconfig | grep 'inet' | grep Broadcast | awk '{Print $ 2}'

(14.2) made on system memory size
# cat / proc / meminfo | awk '/ MemTotal / Print $} 2 { '

(14.3) change the order of fields in the CSV file, data.csv we create a file in the system wants the second field and the fourth field to be exchanged.
# Cat data.csv | awk -F, ' BEGIN {OFS = ","} {print $ 1, $ 4, $ 3, $ 2, $ 5, $ 6, $ 7}'

------ This concludes the article, thanks for reading ------