[Linux] Process Concept (Part 2)

1. Environment variables

1. Command line parameters

What are command line parameters? First of all, we must first know that the main function can pass parameters! The parameters passed to the main function are command line parameters.

We can create a main function and receive command line parameters in the main function. Print the command line parameters and observe them, as follows:

argc and argv in the picture are to receive commands For the formal parameters of the row parameter, let’s observe the printed data:

We see that the printed data is actually the name of our executable program, so what does 0 represent? Let's try to add some more data after the executable program, as follows:

Finally we came to the conclusion: the command we entered, using spaces as delimiters, was divided into 4 substrings, and This4 substring will eventually be passed into the main function to be received, argc is the number of command line parameters, argv is the divided substring; among them 0 The substring number must be our executable program. What follows can be said to be options. Why is it called an option? Because we can execute different codes through different substrings, for example:

		  1 #include <stdio.h>
		  2 #include <string.h>
		  3 
		  4 int main(int argc, char* argv[])
		  5 {
		  6     if(argc != 2)
		  7     {
		  8         printf("error!\n");
		  9         return 1;                                                                                                                 
		 10     }
		 11     if(strcmp(argv[1], "-a") == 0)
		 12         printf("aaa\n");
		 13     else if(strcmp(argv[1], "-b") == 0)
		 14         printf("bbb\n");
		 15     // ........
		 16 
		 17     return 0;
		 18 }

As shown in the above code, we can support the setting of command line options at various command levels through command line parameters!

2. Common environment variables

1. Environment variables generally refer to some parameters used in the operating system to specify the operating environment of the operating system;
2. For example, when we write C/C++ code, we never know where our linked dynamic and static libraries are when linking, but we can still link successfully and generate an executable program. The reason is that there are relevant environment variables to help the compiler search;
3. Environment variables usually have some special purposes, and they usually have global characteristics in the system;

Common environment variables include:

• PATH: Specify the search path for the command
• HOME: Specify the user's home working directory (that is, the default directory when the user logs in to the Linux system)
• SHELL: The current Shell, its value is usually /bin/bash

（1）PATH

What is PATH? We usually write a code in Linux. If we want to run it, we first need to find the path of the executable program, so if the executable program is currently Under the path, you need to add ./ in front, for example, as shown below:

Then through the above study of command line parameters, we know that, Linux< The instructions in a i=4> are also executable programs, so why can their instructions run normally without adding ./? This is related to our environment variable PATH, PATH is< /span> to run! ./ , our program does not need to add PATH, just add the path of our program toSystem default search path

We can use the command which + 指令 to view the path where this command is located. For example, we need to view the path where the ll command is located:

Next let’s check the contents of PATH and use the command echo $PATH to view; where; as follows: shell is equivalent to the meaning of dereference view. This is the syntax of $

What if we want to add our current path to PATH? We first use pwd to view our current path:

Then we copy our path and put it in the following location. You can use the following command:

		PATH=$PATH:/home/lmy/.mygitee/Linux_Study/study9

At this point our path has been added to PATH. We can check it out:

As shown above, we have indeed added the path toPATH; then when we execute the executable program under the current path, we do not need to Add ./ in front; as shown below:

What if we want to delete the current path? It’s also very simple. You just need to remove the unnecessary parts. Suppose we remove the current path in PATH and you can copy other parts except the current path. path, and then execute the following command:

	PATH=/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/lmy/.local/bin:/home/lmy/bin

will remove the current path; note that PATH starts with :< a i=4> is used as a separator to separate different paths.

The above is one method, and there is another method, which is to directly use the cp command to copy our executable program to PATH< It can also be implemented under a file in a path in /span> means to delete the executable program from the path. Program uninstallation. The essence is to copy the executable program to a path that can be found by the system; Program installationsudo. This will not be demonstrated here. This method is called . Note that copying here requires elevated permissions. Just use

If we set PATH to an empty string, let’s see what happens, as shown below:

We can see that most of the commands are no longer available; at this time we can restart Xshell to recover. So we conclude that changing environment variables by default is limited to this login, and the environment variables are automatically restored when you log in again. Why is this? We’ll talk about that later.

（2）PWD

When we use the pwd command, how does the system know where our path is? The reason is because there is also an environment variable in Linux: PWD.

PWD is for our current correspondingbash built-in one specially used to record what we are currently doing. An environment variable for the path at .

For example, let's first check our current path:

Let'scd Go to the upper level and check againPWD Environment variables:

As shown above, PWD environment variables will also change accordingly; so pwd< The command of a i=4> is actually to read the contents of the PWD environment variable and print it out.

（3）HOME

When we log in by defaultXshell, the path/directory we are in is/home/xxx, for example:

But if we log in as the root user, the default directory we are in will be/root ;Why is this?

This is because when we log in, we first need to enter the user name and password and wait for system authentication; after the authentication is completed, environment variables will be formed, and there must be more than one environment variable at this time (PATH, PWD, HOME, etc.) ; Then HOME will be initialized according to the user name, that is, HOME = /root, HOME = /home/xxx a>. cd $HOME; Finally, you only need to execute

(4) env View all environment variables

We can use theenv command to view all environment variables. Linux has many environment variables. You can check it out yourself. Each environment variable has its own special purpose and is used to complete specific system functions.

3. Get environment variables

(1) Obtain environment variables through code

We have an interface that can directly obtain environment variables through code, which isgetenv. We can use man Check out the instructions:

We can use getenv to check the PATH environment variable, as shown in the following code:

		  1 #include <stdio.h>
		  2 #include <stdlib.h>
		  3 
		  4 int main()
		  5 {
		  6     printf("%s\n", getenv("PATH"));
		  7     return 0;                                                                                                                     
		  8 }   

The execution results are as follows:

Every environment variable we mentioned above has its own special purpose. How to reflect it? Let's simply use it in combination with getenv, as follows:

		#include <stdio.h>
		#include <stdlib.h>
		#include <string.h>
		
		int main()
		{
		    char* s = getenv("USER");
		    if(strcmp(s, "root") != 0)
		    {
		        printf("%s 是非法用户!\n", s);
		        return 1;
		    }
		    printf("Hello!\n");
		    printf("Hello!\n");
		    printf("Hello!\n");
		    return 0;
		}

We use environment variablesUSER to determine whether the current user isroot, we only let root execute the corresponding code; the following results:

We are here root Usage:

When we are ordinary users:

(2) Obtained through command line parameters

1. Through the command parameters we learned above, we know that there can be two command line parameters, but in fact there can also be a third command line parameter, that isenv !

env is actually an environment variable table. When the system starts our program, we can choose to provide two tables for our process (main): 1. Command line parameter table 2. Environment variable table; the third parameter env is the environment variable table, env is a pointer, pointing to an array of function pointers. You can refer to the following figure to understand:

The large list on the right is environment variables; usually the last element of env points to empty;

We can also print out the code to observe; we can print the pid of the current program running, as shown in the following code:

		    1 #include <stdio.h>
		    2 #include <stdlib.h>
		    3 #include <string.h>
		    4 #include <unistd.h>
		    5 #include <sys/types.h>
		    6 
		    7 int main(int argc, char* argv[], char* env[])
		    8 {
		    9     int i = 0;
		   10     for(; env[i]; i++)
		   11     {
		   12         printf("pid: %d, env[%d]: %s\n", getpid(), i, env[i]);                                                                  
		   13     }
		   14 
		   15     return 0;
		   16 }

You can view the results of running by yourself; then why do we print out its pid? So we might as well think about it, who passed the env of this process to us? That's bash! The processes started by our command line are all child processes ofshell(bash). The command line parameters and environment variables of the child process are all the parent process a> passed! bash

Then the question comes again, where do the environment variables of the parent process bash come from? In fact, environment variables exist in the form of script configuration files; we can also find this file. For example, when we return to our home directory, there is a hidden file called .bash_profile, as shown below:

Every time you log in, ourbash process will read the contents of this file.bash_profile , forming a new environment variable table information for ourbash process! This can also explain one of our problems above: we changed the path of PATH to empty and restarted Xshell It will return to normal after that.

2. Another question, we can also create our own environment variables, as shown below, just enter it directly on the command line:

At this point we check in the environment variable table:

It was found that it was not imported into our environment variable table; maybe we can find it directly in our executable program:

is also not available, because we have not imported it into the environment variable table. We need to import it into the environment variable table, using the command export MYENV_LMY, as follows:

We can also import it directly when creating environment variables, using the command

export 环境变量名称=内容,As shown below:

We can see that both of them appear in the environment variable table; if we log out and log in again, will they still exist? The answer is no more! Because the above two environment variables are not written into the configuration file! What has changed is only the current bash internal environment variable table. When we log out and log in again, bash will re-read the .bash_profile file to re-obtain the environment variable table! The above is only valid inside the bash process and is called local variable!

So we want our own environment variable to take effect forever, we need to add it to the .bash_profile configuration file, as shown below:

At this point we save and exit, and after logging in again, we can find our corresponding environment variables:

Inheriting the environment variable table through child process inheritance is called environment variable!

We know that both the command line parameter table and the environment variable table can be inherited by the child process, which shows that the system environment variables haveglobal attributes a>!

(3) Obtained through extern char** environ

The second way to obtain environment variables requires us to write command line parameters and pass them in. There is also a way to obtain the environment variable table without writing command line parameters, which is provided by the system< a i=1>environ pointer; when we use this pointer, we need to use extern to declare it, because it is not our own The definition is provided to us by the system. In fact, environ points to env pointers, their relationship is as follows:

We can also print out the environment variable table through environ, as shown in the following code:

		  1 #include <stdio.h>
		  2 #include <stdlib.h>
		  3 #include <string.h>
		  4 #include <unistd.h>
		  5 #include <sys/types.h>
		  6 
		  7 int main()
		  8 {
		  9     extern char** environ;
		 10     int i = 0;
		 11     for(; environ[i]; i++)
		 12     {
		 13         printf("pid: %d, environ[%d]: %s\n", getpid(), i, environ[i]);                                                            
		 14     }
		 15 
		 16     return 0;
		 17 }

You can try and observe the running results yourself. The results are the same as the second method.

4. Local variables and environment variables

Above we also briefly introduced local variables and environment variables. Next, we further analyze the differences between them:

• Local variables: Local variables are only valid within the bash process and will not be inherited by child processes.
• Environment variables: Environment variables realize their global nature by letting all child processes inherit them! Environment variables are naturally inherited by all child processes!

5. Classification of Linux commands

We first recall that when we setPATH to empty, are some commands able to run and some commands unable to run? ? Let’s try it again, as shown below:

We can see that some commands cannot be run, but why can commands such as pwd and echo still run? The reason is because the commands in Linux can be divided into two categories:

1. standing orders

The regular command isshell command line interpreterfork Let the child process execute.

2. built-in commands

The built-in command is a function of theshell command line, built on shell , so you can directly read the local variables defined inside shell!

6. Commands related to environment variables

• echo

echo is to display the value of an environment variable; we have also used this above, for example, we need to view PATHEnvironment variable value:

• export

export imports local variables into the environment variable table, that is, sets a new environment variable, which we have also used above. For example, we first set a local variable. At this time, we can use echo to view it, because echo is a built-in command; but there is no such local variable in environment variablesenv, as shown below:

We can import this local variable into the environment variable and see it in env:

• unset

unset clears environment variables and deletes an environment variable. For example, we want to delete the environment variable MYENV that we defined above, as follows:

At this time, there are no local variables and environment variables.

• env

env displays all environment variables, which we also introduced above.

• set

set displays locally defined shell variables and environment variables. For example, we define a local variablelocal_env and an environment variablemyenv, this When we check the environment variables, we only have myenv, as shown below:

But we can see both when we use set to view, as shown below:

2. Program address space

1. Program address space distribution

We have roughly understood the spatial distribution diagram in C/C++ such as the following figure:

Among themThe heap and the stack are relative. The heap area grows upwards and the stack area grows downwards. In fact, the heap area and the stack There are other spaces in the middle of the area, we will learn about it later; static data variables also exist in the data segment. In fact, static variables will be modified into global variables by the compiler, so they will be placed in the data segment.

How to prove that the addresses of our program are distributed according to the above space? Let’s verify it using code, as shown below:

		    1 #include <stdio.h>
		    2 #include <stdlib.h>
		    3 
		    4 int init_val = 1;
		    5 int uninit_val;
		    6 
		    7 int main()
		    8 {
		    9     char* str = "Hello";
		   10     char* heap1 = (char*)malloc(10);
		   11     char* heap2 = (char*)malloc(10);
		   12     char* heap3 = (char*)malloc(10);
		   13     char* heap4 = (char*)malloc(10);
		   14     static int static_val1 = 2;
		   15     static int static_val2;
		   16 
		   17     printf("栈区地址1:%p\n", &heap1);
		   18     printf("栈区地址2:%p\n", &heap2);                                                                                           
		   19     printf("栈区地址3:%p\n", &heap3);
		   20     printf("栈区地址4:%p\n", &heap4);
		   21 
		   22     printf("堆区地址4:%p\n", heap4);
		   23     printf("堆区地址3:%p\n", heap3);
		   24     printf("堆区地址2:%p\n", heap2);                                                                                            
		   25     printf("堆区地址1:%p\n", heap1);
		   26 
		   27     printf("未初始化静态变量地址:%p\n", &static_val2);
		   28     printf("未初始化全局数据区:%p\n", &uninit_val);
		   29     printf("已初始化静态变量地址:%p\n", &static_val1);
		   30     printf("已初始化全局数据区:%p\n", &init_val);
		   31     printf("字符常量区地址:%p\n", str);
		   32     printf("代码区地址:%p\n", main);
		   33 
		   34     return 0;
		   35 }

The running results are as follows, and the results are indeed like this:

We take the stack area separately for analysis. The arrays and structures we create locally grow downward to open up space in the stack area. Suppose we have a a[ 10] array, a struct A = {x, y, z} structure, then Who has the bigger address? Let's write a program to verify it, as follows:&obj.z and &obj.x, A obj? a[9] and a[0]

		   #include <stdio.h>
		   #include <stdlib.h>
		   
		   typedef struct A                                                                                                                  
		   {
		       int x;
		       int y;
		       int z;
		   }A;
		  
		  int main()
		  {
		      int a[10];
		      A obj;
		      printf("%p\n", &a[0]);
		      printf("%p\n", &a[9]);
		      printf("%p\n", &obj.x);
		      printf("%p\n", &obj.y);
		      printf("%p\n", &obj.z);
		      
			  return 0;
		 }

The result is as follows:

So we come to the conclusion that the stack area opens up the application space downwards, but when used, it is used partially upwards. It can be summarized by the following figure:

As shown above, if we define a variable int b on the stack, then we know that int occupies four bytes, and each space in our space above occupies one byte, then we Which address did you get? In fact, the lowest address is taken, and then accessed through the starting address + offset. &b

In fact, in addition to the areas in the spatial distribution above, there are some areas that we have not learned yet, so we will introduce them later. However, there are two areas in the stack area that we have just learned, which are< a i=1>Command line parameters and environment variables.

So what exactly is the spatial distribution we learned above? Is it memory? Let’s start learning below.

2. Process address space

First of all, let’s review how the parent and child processes ran when we learnedfork before. One problem that has not yet been solved is that when the child process modifies the code, copy-on-write will occur. But why do different values ​​of a variable have the same address? This is what we need to learn next; first we type out the code, as follows:

		  1 #include <stdio.h>
		  2 #include <stdlib.h>
		  3 #include <unistd.h>
		  4                                                                                                                                   
		  5 int g_val = 100;
		  6 
		  7 int main()
		  8 {
		  9     pid_t id = fork();
		 10     if(id == 0)
		 11     {
		 12         //child
		 13         int cnt = 5;
		 14         while(1)
		 15         {
		 16             printf("i am child, g_val: %d, &g_val=%p\n", g_val, &g_val);
		 17             sleep(1);
		 18             if(cnt == 0)
		 19             {
		 20                 g_val=200;
		 21                 printf("child change g_val: 100->200\n");
		 22             }
		 23             cnt--;
		 24         }
		 25     }
		 26     else
		 27     {                                                                                                                             
		 28         //father
		 29         while(1)
		 30         {
		 31             printf("i am father, g_val: %d, &g_val=%p\n", g_val, &g_val);
		 32             sleep(1);
		 33         }
		 34     }
		 35     sleep(100);
		 36     return 0;
		 37 }

We defined a global variableg_val. When the child process modifies it, the parent process remains the same< a i=3>g_val, because the data between the parent and child processes do not affect each other and are independent; when we observe the running results, we will find that their addresses are actually the same! As shown below:

So why does reading the same address result in different contents? So from the above figure we draw the conclusion that the addresses we usually see in C/C++ are definitely not physical addresses, but are actually virtual addresses/linear addresses!

Actually, the picture of space distribution we learned above is process address space, and all the addresses in it are virtual addresses! As shown below:

But our process needs to be scheduled by the CPU. For the data in the process to be read and recognized by the CPU, it must be loaded into memory, that is, physical memory. So what exactly does this process look like? In fact, the addresses printed by our code above are all the addresses of its process address space, that is, virtual addresses, and this executable program is bash's child process! And this parent process creates its own child process in the code, and also has its own process address space, so we believe that after every program runs, there will be a process address space!

Then we now have three important roles, namelypcb(task_struct), process address space, and physical memory, among which < There must be some fields in /span>; so their current relationship is as follows:0x11111111 that point to the address space of its own process; and some data of the process must also be stored in physical memory. In physical memory The address where this data is stored in the memory is called the physical address, such as the followingtask_struct

So how are the process address space and physical memory connected? In the operating system, in order for the process to find its own data in the physical memory, a mapping table will be maintained for each process, which must be the process address. A mapping relationship is constructed between space and physical memory, and this table is called page table. The left side of it is the virtual address, and the right side is the address of the data in physical memory. There is a mapping relationship between them. It can be understood with the following picture:

The above is the simple structure of the page table. There are other fields in the page table that we will introduce later; among them, the process can search the page table to find the physical address corresponding to the physical memory through the address in the process address space.

We know from the above that after the process runs, there will be a process address space, and at the system level they all have their own page table mapping structure; because our parent process above will also have child processes , so the operating system will copy the pcb of the parent process to the child process, so the child process will also have its own process address space and page table, They are all obtained from the parent process. They are independent of each other and do not affect each other, such as the following figure:

When our child process modifies the data, it finds the corresponding data through the page table, but the operating system finds that the parent process is using this data, so the child process cannot directly modify the data because they are independent, so the operating system The values ​​in the physical memory will be copied on write to generate a new physical address, and then the data that the child process wants to modify will overwrite the data in between; at this time, the physical address in the child process will change, combined with the following Figure understanding:

So we understand why they have the same address but different values, because the two processes have independent process address spaces and page tables. The copy-on-write occurs in physical memory, and the physical address is also changed. , the virtual address has not changed, so the same virtual addresses do not affect each other.

3. Address space and area division

(1) Address space

We know from the above that physical memory is directly managed by the operating system. When our process needs to use space, the operating system will give the process a virtual address space, and this virtual address space is the address space. When we have more processes, since the operating system needs to manage the processes, the operating system must also manage the address space, because if it is not managed, everything will be messed up. So how to manage it? We’ll talk about it below.

(2) Regional division

The so-called regional division is to use start and end Such int or long long variables to distinguish a linear space Start and end; the space size can be adjusted. When adjusting, you only need to start and end Just expand or reduce the value; don't just see the range of the space, we can use all addresses within the space range.

(3)Manage address space

The address space must be managed by the operating system, because each process has an address space, and the address space must be managed in the system. How to manage it? We have learned before, first describe, then organize, so the address space must ultimately be a kernel data structure object, which is a kernel structure!

We found that the contents of the process address space are all divided into regions! So we can divide the areas for management!

InLinux, thisprocess/virtual address spaceIt’s calledstruct mm_struct, and it probably looks like this:

		struct mm_struct
		{
			long code_start;
			long code_end;
			long data_start;
			long data_end;
			long heap_start;
			long heap_end;
			long stack_start;
			long stack_end;
			......
		}

wherestruct mm_struct this structure is pointed to by a pointer called struct mm_struct* mm, and this pointer exists in task_struct , so when each process is created, it will create a struct mm_struct* mm, and there will be a process address space; so why is our process address space not memory? The reason is because it is just a kernel data structure.

4. Why do we need an address space?

1. Let the process view the memory from a unified perspective, so any process can turn the disordered memory data into order through the address space + page table, and plan it in categories!

Why do you say that? Let's think about it, when our program is loaded into memory, is it loaded in order? No, the program is loaded into the memory out of order, but through the address space and page table, the out-of-order memory data can be changed into order, making the process think that the data is classified according to its way!

2. There is a virtual address space, which can effectively perform security checks on process access to memory!

First of all, we need to understand that in the page table, there is another column called Access Permission Field. Its structure is as follows :

What is the use of the access rights field? It will check whether the corresponding permissions are met when accessing physical memory, or whether the address is legal. If the permissions are not met or the address is illegal, it will be directly intercepted in the page table and access to the address in physical memory will not be allowed.

For example the following code:

		int main()
		{
			char* s = "hello";
			*s = 'h';
			return 0;
		}

The above code cannot run normally, because we know that"hello" is a constant and cannot be modified, so why is this Woolen cloth? At this time we know that the address of s is in the character constant area in the process address space, and in the page table mapped through the character constant area The access permission field is read-only, that is, r, so when we need to write modifications, it is directly intercepted in the page table.

In addition, the process performs various conversions and various accesses, and this process must be running! So there is a register in cpu called CR3, which stores the page table The address of this page table is the physical address. Because this process is running in cpu, the content in CR3 It is in the hardware context of the process! So when the process switches out, essentially the contents of the CR3 register will be saved to the context of the current process. So each process has its own page table.

3. Decouple process management and memory management!

At the operating system level, when we have a program on the disk that needs to be loaded into the memory, we first need to apply for memory in the memory, then fill in the content and page table, and then establish a mapping relationship; there is also a column in the page table Content is specifically used to determine whether an address has allocated memory and content in the memory. It may contain two bits, such as 1/0 meansyes/no allocated memory, 1/0 means< /span>; for example, the following picture:Whether there is content in the memory

When our process takes a virtual address to find a physical address, assuming that the memory has not allocated a physical address to it at this time, the operating system will pause the process and load the corresponding address from the disk. The corresponding program is stored in the memory, and then the page table is filled in to establish the mapping relationship; this process is called page fault interrupt! We will introduce this concept later, but let’s understand it first.

When we apply for memory in the memory above, then fill in the content and page table, and then establish the mapping relationship, this set of processes is calledMemory management; The process does not need to pay attention to this process, and the process does not know this process; all the process needs to do is to schedule what should be scheduled, and access what should be accessed. This set is called Process management; Process management does not care about memory management,so process management and memory management are realized at the operating system level because of the existence of address space and page table. Decoupling of modules!

Finally, through page tables, processes can also be mapped to different physical memories, thereby achieving process independence!

5. Space allocation expansion

The address space we can currently use is used in user space. In the address space, there is still a part of the space reserved for the operating system itself; the space used by our users is called user space, which is 3GB; The space used by the operating system itself is called kernel space, which is 1GB. As shown in the figure below:

In the struct mm_struct structure, there is actually a pointer, which points to a vm_area_struct structure. What is its use? Our address space is divided into many areas, but there are always some areas that have not been used yet, so when we want to divide our own area, we can apply for one vm_area_struct Object, this object has two values: start and end, respectively are the beginning and end of the space, and there is also a next pointer that points to the object of the next structure. They form a linked list structure, so there is With such a structure, we can divide our own space according to our own needs! It can be understood according to the following figure:

Among them, our mm_struct structure is actually called a memory descriptor; and vm_area_struct is called a linear space; these two concepts combined are called address space ! But due to convenience, we think mm_struct is the address space!