Linux shell programming study notes 37: readarray command and mapfile command

Table of contents

0 Preface

In interactive programming, the values of array elements sometimes need to be input from outside the program. For example, if it is input by the user through the keyboard, we can use the read -a command to achieve this. However, when there is a lot of data that needs to be entered repeatedly, it is not convenient and efficient enough to use the read -a command. And for some frequently used fixed data, we can store the data in a file, and then read the data from the file when using the data.

For this purpose, Linux specifically provides the readarray command.

1 Format and function of readarray command

We can use the command help readarray to view the help information of the readarray command.

purleEndurer @ bash ~ $ help readarray
readarray: readarray [-n count] [-O origin] [-s count] [-t] [-u fd] [-C callback] [-c quantum] [array]
Read lines from a file into an array variable.

A synonym for `mapfile'.

purleEndurer @ bash ~ $ readarray --help
bash: readarray: --: invalid option
readarray: usage: readarray [-n count] [-O origin] [-s count] [-t] [-u fd] [-C callback] [-c quantum] [array]
purleEndurer @ bash ~ $

Unfortunately, the help readarray command displays not much help information. We tried the readarray --help command again, but the readarray command does not support the --help option.

1.1 Command format

readarray [-n maximum number of lines] [-O starting index] [-s skip number of lines] [-t] [u file descriptor] [-C callback program] [-c number of lines] [array name]

Options	illustrate	Remark
-c number of lines	Call the callback program specified by the "-C callback program" option every time the specified number of lines are read. The default is to call the callback every 5000 rows	count
-C callback routine	Execute the callback routine every time the number of lines specified by the "-c number of lines" option is read.	callback
-n maximum number of lines	Only copies the specified maximum number of rows of data to the array. The default is 0, which means all rows are copied.	number
-O starting index	Specify which index to start storing data from, the default is 0. For two-dimensional arrays, the starting row number is specified.	origin
-s skip lines	Ignore data in the specified number of skipped rows, starting after the number of skipped rows	skip
-t	Remove trailing line separator, default is newline Mainly used with the -u option	trim
-u file descriptor	Specifies that data is read from a file descriptor instead of standard input	use

1.2 Command functions

Reads data from standard input or the specified file and stores it in the specified array.

1.3 Things to note

When entering data in the standard, press the Enter key to change the line. After entering all the data, press Ctrl+D to end the input (Ctrl+D is not displayed on the screen).
If the specified array variable originally stored a value, and there is no -O option when using the readarray command, the original data in the array variable will be cleared first, and then the newly read data will be stored.
If the array name is not specified, the data will be saved in the MAPFILE array.

2 Command application examples

2.1 When reading data from standard input without specifying an array name, the data will be saved in the MAPFILE array.

Example 2.1

purpleEndurer @ bash ~ $ readarray
1 1 1
2 2 2
purpleEndurer @ bash ~ $ echo $REPLY

purpleEndurer @ bash ~ $ echo $MAPFILE
1 1 1
purpleEndurer @ bash ~ $ echo ${MAPFILE[*]}
1 1 1 2 2 2
purpleEndurer @ bash ~ $ echo ${MAPFILE[0]}
1 1 1
purpleEndurer @ bash ~ $ echo ${MAPFILE[1]}
2 2 2
purpleEndurer @ bash ~ $

After we input the two lines of data 1 1 1 and 2 2 2, press Ctrl+D to end the input.

For the read command, if you do not specify the variable name used to store data, the data will be saved in the variable REPLY.

But for the readarray command, if the array variable name used to store data is not specified, the data will be saved in the MAPFILE array.

2.2 Read data from standard input and store it in the specified array

Example 2.2 Read two lines of data from standard input and store them in the specified array variable a

purpleEndurer @ bash ~ $ readarray a
1 2 3
4 5 6
purpleEndurer @ bash ~ $ echo $a
1 2 3
purpleEndurer @ bash ~ $ echo ${a[*]}
1 2 3 4 5 6
purpleEndurer @ bash ~ $ echo ${a[0][*]}
1 2 3

purpleEndurer @ bash ~ $ echo ${a[1][*]}
4 5 6
purpleEndurer @ bash ~ $

We entered two lines of data, 1 2 3 and 4 5 6, and you can see that the data is stored in the array variable a.

By default, the system starts storage from array index 0, so the results of command execution are as follows:

a[0][0]=1 a[0][1]=2 a[0][2]=3

a[1][0]=4 a[1][1]=5 a[1][2]=6

2.3 Use the -O option to specify the starting index

Example 2.3.1 Based on Example 2.2, we continue to read two lines of data from the standard input and store them in the specified array a, with the starting index being 1

purpleEndurer @ bash ~ $ readarray -O1 a
a b c
d e f
purpleEndurer @ bash ~ $ echo ${a[*]}
1 2 3 a b c d e f
purpleEndurer @ bash ~ $ echo ${a[1][*]}
a b c
purpleEndurer @ bash ~ $ echo ${a[2][*]}
d e f
purpleEndurer @ bash ~ $

We entered two lines of data, abc and def. Since we specify starting from index 1,

Therefore, the first row of data in the two-dimensional array a has not changed.

The second row of data in the two-dimensional array a becomes [abc]

[def] becomes the data in the third row of the two-dimensional array a.

At this time, the value of the two-dimensional array a is:

a[0][0]=1 a[0][1]=2 a[0][2]=3

a[1][0]=a a[1][1]=b a[1][2]=c

a[2][0]=d a[2][1]=e a[2][2]=f

It can be seen that for a two-dimensional array, -O specifies the starting row number.

So, what about one-dimensional arrays? What does -O specify?

Let's take a look at the following example.

Example 2.3.2 First define the one-dimensional array a and initialize its value to 1 2 3, then use the readarray command to read the data abc, and specify to start storing from subscript 2 of the array a.

purpleEndurer @ bash ~ $ a=( 1 2 3)
purpleEndurer @ bash ~ $ echo $a
1
purpleEndurer @ bash ~ $ echo ${a[*]}
1 2 3
purpleEndurer @ bash ~ $ readarray -O2 a
a b cpurpleEndurer @ bash ~ $ echo ${a[*]}
1 2 a b c
purpleEndurer @ bash ~ $

Notice:

After entering abc, press Ctrl+D twice to keep array a as a one-dimensional array.

If the Enter key is pressed, array a will become a two-dimensional array.

As you can see, for a one-dimensional array, the -O option specifies the subscript of the element.

Example 2.3.3 Without using the -O option, the original data in the specified array name will be cleared first.

purpleEndurer @ bash ~ $ readarray a
1
2
3
4

purpleEndurer @ bash ~ $ echo ${a[*]}
1 2 3 4
purpleEndurer @ bash ~ $ readarray a
a
b
purpleEndurer @ bash ~ $ echo ${a[*]}
a b
purpleEndurer @ bash ~ $

When the readarray a command is executed for the first time , the data 1, 2, 3, and 4 we entered are stored in the data variable a.

When the readarray a command is executed for the second time , the data a and b we entered are stored in the data variable a, and the original data 1, 2, 3, and 4 are cleared.

2.4 Use -n to specify the number of valid lines

Example 2.4 Read 2 lines of data from standard input and store them in array variable a.

purpleEndurer @ bash ~ $ echo $a

purpleEndurer @ bash ~ $ readarray -n 2 a
1 1 1
2 2 2
purpleEndurer @ bash ~ $ echo ${a[*]}
1 1 1 2 2 2
purpleEndurer @ bash ~ $ echo ${a[1]}
2 2 2
purpleEndurer @ bash ~ $ echo ${a[0]}
1 1 1
purpleEndurer @ bash ~ $

As you can see, after we enter two lines of data, the readarray command automatically stops inputting and stores the data we entered into the array variable a.

2.5 Use -s to pass through some data

Example 2.5 Skip the first 2 lines of data in the standard input and store the subsequent data in the array variable a.

purpleEndurer @ bash ~ $ echo $a

purpleEndurer @ bash ~ $ readarray -s 2 a
1 1 1
2 2 2
3 3 3
4 4 4
purpleEndurer @ bash ~ $ echo ${a[*]}
3 3 3 4 4 4
purpleEndurer @ bash ~ $ echo ${a[1]}
4 4 4
purpleEndurer @ bash ~ $ echo ${a[0]}
3 3 3
purpleEndurer @ bash ~ $

We input four lines of data 1 1 1 , 2 2 2 , 3 3 3 , and 4 4 4 . Due to the -s 2 option, the first two lines of data 1 1 1 and 2 2 2 are skipped. The data stored in array variable a is 3 3 3, 4 4 4, that is:

a[0][0]=3 a[0][1]=3 a[0][2]=3

a[1][0]=4 a[1][1]=4 a[1][2]=4

2.6 Using the callback routine with the -c and -C options

Example 2.6 Read data from the standard input, and call the echo command to display the string every time 2 lines of data are read ---

purpleEndurer @ bash ~ $ readarray -c 2 -C "echo ---"
a
b
--- 1 b

c
d
--- 3 d

e
f
--- 5 f

purpleEndurer @ bash ~ $ echo ${MAPFILE[*]}
a b c d e f
purpleEndurer @ bash ~ $

2.7 Reading data from disk files using output redirection and the -t option

Example 2.7.1 Use the seq command to create the data file d.txt, and then use readarray and input redirection to store the contents of the data file d.txt into the array variable a

purpleEndurer @ bash ~ $ seq 5 > d.log
purpleEndurer @ bash ~ $ cat d.log
1
2
3
4
5
purpleEndurer @ bash ~ $ readarray a < d.log
purpleEndurer @ bash ~ $ echo ${a[*]}
1 2 3 4 5
purpleEndurer @ bash ~ $ echo ${#a[*]}
5
purpleEndurer @ bash ~ $ echo ${#a[1]}
2
purpleEndurer @ bash ~ $ echo ${#a[1][1]}
2
purpleEndurer @ bash ~ $ echo ${#a[1][2]}
2

Example 2.7.2 Using input redirection and the readarray -t command to read data from the d.txt file created in Example 2.7.1 and store it in the array variable a

purpleEndurer @ bash ~ $ readarray -t a < d.log
purpleEndurer @ bash ~ $ echo ${a[*]}
1 2 3 4 5
purpleEndurer @ bash ~ $ echo ${#a[*]}
5
purpleEndurer @ bash ~ $ echo ${#a[1][1]}
1
purpleEndurer @ bash ~ $ echo ${#a[1][2]}
1

purpleEndurer @ bash ~ $ echo ${#a[1]}
1
purpleEndurer @ bash ~ $

Judging from the command execution results of echo ${a[*]} and echo ${#a[*]}, the execution results of readarray a < d.log and readarray -ta < d.log seem to be the same.

But judging from the execution results of the echo ${#a[1]}, echo ${#a[1][1]}, and echo ${#a[1][2]} commands, readarray a < d.log and The results of readarray -ta < d.log execution are different.

This is because readarray a < d.log does not filter newlines.

3 mapfile command

The mapfile command is not only functionally the same as the readarray command, but also has the same command format as the readarray command.

However, the help information of the mapfile command is much more detailed than that of the readarray command.

purpleEndurer @ bash ~ $ help mapfile
mapfile: mapfile [-n count] [-O origin] [-s count] [-t] [-u fd] [-C callback] [-c quantum] [array]
Read lines from the standard input into an indexed array variable.

Read lines from the standard input into the indexed array variable ARRAY, or
from file descriptor FD if the -u option is supplied. The variable MAPFILE
is the default ARRAY.

Options:
-n count Copy at most COUNT lines. If COUNT is 0, all lines are copied.
-O origin Begin assigning to ARRAY at index ORIGIN. The default index is 0.
-s count Discard the first COUNT lines read.
-t Remove a trailing newline from each line read.
-u fd Read lines from file descriptor FD instead of the standard input.
-C callback Evaluate CALLBACK each time QUANTUM lines are read.
-c quantum Specify the number of lines read between each call to CALLBACK.

Arguments:
ARRAY Array variable name to use for file data.

If -C is supplied without -c, the default quantum is 5000. When
CALLBACK is evaluated, it is supplied the index of the next array
element to be assigned and the line to be assigned to that element
as additional arguments.

If not supplied with an explicit origin, mapfile will clear ARRAY before
assigning to it.

Exit Status:
Returns success unless an invalid option is given or ARRAY is readonly or
not an indexed array.
purpleEndurer @ bash ~ $