Linux CPU utilization and load relationship under
1, CPU utilization and load
CPU utilization shows the percentage of CPU during program run real-time occupied; cpu utilization rate reflects the current cpu how busy because the peaks and valleys that process takes cpu processing time may be in a waiting state but has not yet io released into the wait.
CPU load refers to the process of occupation of the cpu time and the number of processes waiting for a certain period of time cpu, cpu time waiting process here refers to the process of waiting to be awakened, not including the wait state in the process.
High CPU utilization, does not mean that the CPU load is large. There is no necessary relationship between the two. Whether the CPU utilization is high or low, with the back of how much is not necessarily related tasks in the queue.
In Linux / Unix, CPU utilization is divided into user mode, the system state and the idle state, respectively, the time the CPU is in user mode execution time, the execution time of the system kernel, executing system processes and idle.
Usually said CPU utilization refers to: CPU execute total execution time non-system idle process time / CPU.
2, CPU load rate using a high
The causes sentence summary is this: wait too much disk I / O completion of the process leading to the process queue length is too large, but the process cpu to run a little, so that the load is too large to reflect, cpu utilization is low.
The following are specific content Principle Analysis: Why before analysis under high load introduction to what is the load, multi-tasking operating system, process scheduling and other related concepts.
2.1 What is the load
What is the load: the load is being processed and waiting cpu statistics and the number of processes cpu process over a period of time, that is cpu usage statistics for the queue length, the smaller the number, the better (if more than the CPU core is 0.7 * not normal)
load is divided into two parts: CPU load, IO load
For example, suppose you have a program of large-scale scientific computation performed, although the program does not run frequently from the disk input and output, but the process will take a long time to complete. Since this program is primarily used for calculation, the logic judgment processing, the processing speed of the procedure depends on the computation speed of cpu. Such cpu load a program called "compute-intensive programs."
Another type of program, mainly from the large amount of data saved to disk search to find any file. The processing speed of the search process does not depend on cpu, but depends on the reading speed of the disk, i.e. the input and output (input / output, I / O ). The faster the disk, the less time it takes to retrieve. Such I / O load program, called "I / O-intensive programs."
2.2 What is a multitasking operating system
Linux operating system capable of handling several tasks with different names. But the process of running multiple tasks, cpu and disk hardware resources on these limited tasks that need to be shared programs. Even a very short time interval, while switching between processing tasks to the side, which is multi-tasking.
Fewer cases running task, the system waits for such switching operation does not occur. But when increasing tasks, such as task A is performed on CPU computing, the next task if B and C also want to calculate, then we need to wait for the CPU is idle. In other words, even if the process is to run a task, you have to wait until his turn to run, such a state of waiting for the delay on the performance of the program is running.
2.3 Process Scheduling
What is the process of scheduling:
Process Scheduling is also called by some cpu context switch means is: CPU switches to another process that needs to save the current state of the process and restore the state of another process: the currently running task into the ready (or suspend, interrupt) state, another selected ready task becomes the current task. The process of scheduling the operating environment, including preservation of the current task, the task of recovery will be running the operating environment.
In the linux kernel, each process management table there is a "process descriptor". The process descriptor will be adjusted in accordance with priorities in descending order, has a reasonable order of running processes (tasks). This adjustment is the work process scheduler.
The scheduler is divided and state management process, such as:
- Cpu wait state allocation of resources.
- Waiting for disk input and output finished state.
In the following talk about the difference between the state of the process:
| status | Explanation |
| Operating mode (running) | As long as cpu idle, can be run at any time |
| Interruptible sleep (interruptible) | Recovery time is unpredictable long wait state. E.g., from a keyboard input device. |
| Uninterruptible sleep: (uninterruptible) | The main waiting state for a short time. Such as disk input and output wait. Blocked IO process |
| Ready state (Runnable) | Pause signal in response to operation of the interrupt status. |
| Dead state (Zombie) | Processes are created by the parent process, and destroy; it does not destroy the child process the parent process, when they were destroyed, the child process because there is no parent is destroyed, it will be transformed into rigid state. |
The following are examples to illustrate the state transition process:
There are three processes A, B, C run simultaneously. First, each process is run after generating state, that is, start running the state, rather than the current operating state, unable to distinguish the state of the running and wait for the state to run in the linux kernel will be running status and running state is called running state.
- Process A: running
- Process B: running
- Process C: running
Three processes running immediately become scheduling objects. In this case, assuming that the scheduler assigned to process A permission to run the CPU.
- Process A: running (running)
- Process B: running
- Process C: running
A process of allocating the CPU, so the process A start processing. B and C are in the process of this waiting process A move CPU. A process carried out after a number of assumptions to calculate, need to read data from the disk. After then issues a read request to the disk data A, data prior to the request arrives, without any work. This condition is called "waiting for I / O operations end is blocked." In the front I / O completion processing, process A has been in waiting, will turn uninterruptible sleep state (uninterruptible), do not use the CPU. So the priority calculation process scheduler View B and C of the process, the CPU permission to run higher priority to the party. It is assumed that the process B takes precedence over the process C.
- Process A: uninterruptible (waiting for disk I / uninterruptible state)
- Process B: running (running)
- Process C: running
B process has just started running, we need to wait for the user's keyboard input. B then waits for the user to enter the key input, likewise blocked. The result becomes the processes A and B are waiting for output, running processes C. Then processes A and B are waiting for the state, but waiting for disk input and output and waits for keyboard input to a different state. Wait for keyboard input events wait indefinitely, while reading the disk is to be completed within a short time waiting for the event, which are two different wait states. Each process state as follows:
- Process A: uninterruptible (waiting for disk I / uninterruptible state)
- Process B: interruptible (waiting for keyboard input and output / interruptible state)
- Process C: running (running)
The process C is assumed during operation, the process A requests data from the disk reaches the buffer means. Then the hard kernel initiated the interrupt signal, the kernel knows disk read is complete, the recovery process A runnable state.
- Process A: running (running)
- Process B: interruptible (waiting for keyboard input and output / interruptible state)
- Process C: running (running)
After this process C will become some kind of wait states. Such as CPU occupancy exceeds the time limit, the end of the task, enter the I / O wait. Once these conditions are met, the dispatcher can complete the handover process from the state C to the process the process A.
Load meaning:
Load represents the "average wait for the process." In the above process of state transformation process, in addition to running the state, others are waiting for the state, then other states will join the waiting process to load it?
Facts have proved that only the process in the running state (running) and can not be interrupted state (interruptible) will be added to the load waiting process, that is, the following process in both cases will show the value of the load.
- Even if the need for immediate use CPU, also need to wait for other processes run out of CPU
- Even the need to continue treatment, but also must wait for the disk input and output can be completed
The following describes a scenario intuitive feel why only operating mode (running) and interrupt status (the Interruptible) will be added to the load.
Such as: the process is very CPU resources, for example, the process of making animated encoding, although would be the same as other types of treatment, reflecting the results of the system has become very slow, there are large amounts of data is read from the disk, the system reflect also become very slow. But on the other hand, no matter how many processes are waiting for keyboard input and output operations there, will not let the system slow response.
What a scene will cause CPU load is low and it is very high indeed?
By analyzing the specific meaning of the above load is quite clear, the load is summed up in one sentence: need to run the process but the process must queue waiting to be processed before the number to complete the process. Specifically, that is, the following two situations:
- Wait for the process to be licensed to run CPU authority
- Waiting for disk I / O completion process
low and high cpu load that is waiting for disk I / O completion of the process too much, it will lead to queue length is too large, so it reflects the load is too large, but actually this time cpu is assigned to perform other tasks or idle, there are several specific scenario.
Scene One: A disk read and write requests too much will lead to a large number of I / O wait
Mentioned above, the efficiency is higher than cpu disk and cpu running the above process requires access to disk files, this time initiated calls to the kernel cpu will file a request to let the kernel to get the disk file, this time will switch to other processes or idle, this task will be converted to non-interrupted sleep. When too much of this will lead to read and write requests uninterruptible sleep process, resulting in high load, cpu low.
Scene Two: MySQL does not exist in the index statement or deadlocks, etc.
We all know that MySQL's data is stored in the hard disk, if needed sql query, you need to first load the data from disk into memory. When the data is particularly large, if there is no index sql statement is executed, it will cause the number of lines scanned the table is too large I / O blocking or deadlock statement, will result in I / O blocked, resulting in uninterruptible sleep too much process, leading to heavy load.
Specific solutions can run the show full processlist command to view the thread waits for the situation, to which the statement out to be optimized in MySQL.
Scene Three: External hard drive failure, common to hang up NFS, but the NFS server failure
For example, our system mount the NFS shared storage such as an external hard disk, there is often a large number of read and write requests to access files stored NFS, NFS Server fault if this time, it will cause the process to obtain read and write requests have been less resources, so that the process has been uninterruptible state, resulting in a high load.
1, CPU utilization and load
CPU utilization shows the percentage of CPU during program run real-time occupied; cpu utilization rate reflects the current cpu how busy because the peaks and valleys that process takes cpu processing time may be in a waiting state but has not yet io released into the wait.
CPU load refers to the process of occupation of the cpu time and the number of processes waiting for a certain period of time cpu, cpu time waiting process here refers to the process of waiting to be awakened, not including the wait state in the process.
High CPU utilization, does not mean that the CPU load is large. There is no necessary relationship between the two. Whether the CPU utilization is high or low, with the back of how much is not necessarily related tasks in the queue.
In Linux / Unix, CPU utilization is divided into user mode, the system state and the idle state, respectively, the time the CPU is in user mode execution time, the execution time of the system kernel, executing system processes and idle.
Usually said CPU utilization refers to: CPU execute total execution time non-system idle process time / CPU.
2, CPU load rate using a high
The causes sentence summary is this: wait too much disk I / O completion of the process leading to the process queue length is too large, but the process cpu to run a little, so that the load is too large to reflect, cpu utilization is low.
The following are specific content Principle Analysis: Why before analysis under high load introduction to what is the load, multi-tasking operating system, process scheduling and other related concepts.
2.1 What is the load
What is the load: the load is being processed and waiting cpu statistics and the number of processes cpu process over a period of time, that is cpu usage statistics for the queue length, the smaller the number, the better (if more than the CPU core is 0.7 * not normal)
load is divided into two parts: CPU load, IO load
For example, suppose you have a program of large-scale scientific computation performed, although the program does not run frequently from the disk input and output, but the process will take a long time to complete. Since this program is primarily used for calculation, the logic judgment processing, the processing speed of the procedure depends on the computation speed of cpu. Such cpu load a program called "compute-intensive programs."
Another type of program, mainly from the large amount of data saved to disk search to find any file. The processing speed of the search process does not depend on cpu, but depends on the reading speed of the disk, i.e. the input and output (input / output, I / O ). The faster the disk, the less time it takes to retrieve. Such I / O load program, called "I / O-intensive programs."
2.2 What is a multitasking operating system
Linux operating system capable of handling several tasks with different names. But the process of running multiple tasks, cpu and disk hardware resources on these limited tasks that need to be shared programs. Even a very short time interval, while switching between processing tasks to the side, which is multi-tasking.
Fewer cases running task, the system waits for such switching operation does not occur. But when increasing tasks, such as task A is performed on CPU computing, the next task if B and C also want to calculate, then we need to wait for the CPU is idle. In other words, even if the process is to run a task, you have to wait until his turn to run, such a state of waiting for the delay on the performance of the program is running.
2.3 Process Scheduling
What is the process of scheduling:
Process Scheduling is also called by some cpu context switch means is: CPU switches to another process that needs to save the current state of the process and restore the state of another process: the currently running task into the ready (or suspend, interrupt) state, another selected ready task becomes the current task. The process of scheduling the operating environment, including preservation of the current task, the task of recovery will be running the operating environment.
In the linux kernel, each process management table there is a "process descriptor". The process descriptor will be adjusted in accordance with priorities in descending order, has a reasonable order of running processes (tasks). This adjustment is the work process scheduler.
The scheduler is divided and state management process, such as:
- Cpu wait state allocation of resources.
- Waiting for disk input and output finished state.
In the following talk about the difference between the state of the process:
| status | Explanation |
| Operating mode (running) | As long as cpu idle, can be run at any time |
| Interruptible sleep (interruptible) | Recovery time is unpredictable long wait state. E.g., from a keyboard input device. |
| Uninterruptible sleep: (uninterruptible) | The main waiting state for a short time. Such as disk input and output wait. Blocked IO process |
| Ready state (Runnable) | Pause signal in response to operation of the interrupt status. |
| Dead state (Zombie) | Processes are created by the parent process, and destroy; it does not destroy the child process the parent process, when they were destroyed, the child process because there is no parent is destroyed, it will be transformed into rigid state. |
The following are examples to illustrate the state transition process:
There are three processes A, B, C run simultaneously. First, each process is run after generating state, that is, start running the state, rather than the current operating state, unable to distinguish the state of the running and wait for the state to run in the linux kernel will be running status and running state is called running state.
- Process A: running
- Process B: running
- Process C: running
Three processes running immediately become scheduling objects. In this case, assuming that the scheduler assigned to process A permission to run the CPU.
- Process A: running (running)
- Process B: running
- Process C: running
A process of allocating the CPU, so the process A start processing. B and C are in the process of this waiting process A move CPU. A process carried out after a number of assumptions to calculate, need to read data from the disk. After then issues a read request to the disk data A, data prior to the request arrives, without any work. This condition is called "waiting for I / O operations end is blocked." In the front I / O completion processing, process A has been in waiting, will turn uninterruptible sleep state (uninterruptible), do not use the CPU. So the priority calculation process scheduler View B and C of the process, the CPU permission to run higher priority to the party. It is assumed that the process B takes precedence over the process C.
- Process A: uninterruptible (waiting for disk I / uninterruptible state)
- Process B: running (running)
- Process C: running
B process has just started running, we need to wait for the user's keyboard input. B then waits for the user to enter the key input, likewise blocked. The result becomes the processes A and B are waiting for output, running processes C. Then processes A and B are waiting for the state, but waiting for disk input and output and waits for keyboard input to a different state. Wait for keyboard input events wait indefinitely, while reading the disk is to be completed within a short time waiting for the event, which are two different wait states. Each process state as follows:
- Process A: uninterruptible (waiting for disk I / uninterruptible state)
- Process B: interruptible (waiting for keyboard input and output / interruptible state)
- Process C: running (running)
The process C is assumed during operation, the process A requests data from the disk reaches the buffer means. Then the hard kernel initiated the interrupt signal, the kernel knows disk read is complete, the recovery process A runnable state.
- Process A: running (running)
- Process B: interruptible (waiting for keyboard input and output / interruptible state)
- Process C: running (running)
After this process C will become some kind of wait states. Such as CPU occupancy exceeds the time limit, the end of the task, enter the I / O wait. Once these conditions are met, the dispatcher can complete the handover process from the state C to the process the process A.
Load meaning:
Load represents the "average wait for the process." In the above process of state transformation process, in addition to running the state, others are waiting for the state, then other states will join the waiting process to load it?
Facts have proved that only the process in the running state (running) and can not be interrupted state (interruptible) will be added to the load waiting process, that is, the following process in both cases will show the value of the load.
- Even if the need for immediate use CPU, also need to wait for other processes run out of CPU
- Even the need to continue treatment, but also must wait for the disk input and output can be completed
The following describes a scenario intuitive feel why only operating mode (running) and interrupt status (the Interruptible) will be added to the load.
Such as: the process is very CPU resources, for example, the process of making animated encoding, although would be the same as other types of treatment, reflecting the results of the system has become very slow, there are large amounts of data is read from the disk, the system reflect also become very slow. But on the other hand, no matter how many processes are waiting for keyboard input and output operations there, will not let the system slow response.
What a scene will cause CPU load is low and it is very high indeed?
By analyzing the specific meaning of the above load is quite clear, the load is summed up in one sentence: need to run the process but the process must queue waiting to be processed before the number to complete the process. Specifically, that is, the following two situations:
- Wait for the process to be licensed to run CPU authority
- Waiting for disk I / O completion process
low and high cpu load that is waiting for disk I / O completion of the process too much, it will lead to queue length is too large, so it reflects the load is too large, but actually this time cpu is assigned to perform other tasks or idle, there are several specific scenario.
Scene One: A disk read and write requests too much will lead to a large number of I / O wait
Mentioned above, the efficiency is higher than cpu disk and cpu running the above process requires access to disk files, this time initiated calls to the kernel cpu will file a request to let the kernel to get the disk file, this time will switch to other processes or idle, this task will be converted to non-interrupted sleep. When too much of this will lead to read and write requests uninterruptible sleep process, resulting in high load, cpu low.
Scene Two: MySQL does not exist in the index statement or deadlocks, etc.
We all know that MySQL's data is stored in the hard disk, if needed sql query, you need to first load the data from disk into memory. When the data is particularly large, if there is no index sql statement is executed, it will cause the number of lines scanned the table is too large I / O blocking or deadlock statement, will result in I / O blocked, resulting in uninterruptible sleep too much process, leading to heavy load.
Specific solutions can run the show full processlist command to view the thread waits for the situation, to which the statement out to be optimized in MySQL.
Scene Three: External hard drive failure, common to hang up NFS, but the NFS server failure
For example, our system mount the NFS shared storage such as an external hard disk, there is often a large number of read and write requests to access files stored NFS, NFS Server fault if this time, it will cause the process to obtain read and write requests have been less resources, so that the process has been uninterruptible state, resulting in a high load.