1. Concept
CPU binding refers to binding a process or thread to a specified CPU core for execution in a multi-CPU system. In Linux, we can use the CPU affinity attribute to bind a process to one or more CPU cores.
CPU Affinity is an attribute of a process, which indicates which CPUs the process scheduler can schedule the process to. This property requires a process to run on a specified CPU for as long as possible without being migrated to another processor.
CPU Affinity is divided into two types: soft affinity and hard affinity. Soft affinity is just a suggestion. If it is unavoidable, the scheduler will still schedule the process to other CPUs for execution; hard affinity is a rule that the scheduler must abide by. Linux kernels of version 2.6 and above allow developers to program to achieve hard affinity .
2. The meaning of using hard affinity
- Improve CPU cache hit ratio
The cache is not shared between CPUs. If the process frequently switches between multiple CPUs, the cache of the old CPU will be invalidated, and the advantage of using the CPU cache will be lost. If the process is only executed on a certain CPU, the performance cost caused by the cache invalidation that occurs when the process stops executing on one CPU and then re-executes on a different CPU can be avoided.
- Suitable for time-sensitive applications
In applications with high real-time requirements, we can bind important system processes to designated CPUs, and bind application processes to other CPUs. This approach ensures that time-sensitive applications can run while allowing other applications to use the remaining computing resources.
Kernel information through train: Linux kernel source code technology learning route + video tutorial code information
Learning through train: Linux kernel source code/memory tuning/file system/process management/device driver/network protocol stack
3. Process and CPU binding
In Linux, the CPU Affinity mask is represented by the structure cpu_set_t, and a series of macros are defined to operate the schedulable CPU set of the process:
#define _GNU_SOURCE
#include <sched.h>
void CPU_ZERO(cpu_set_t *set);
void CPU_SET(int cpu, cpu_set_t *set);
void CPU_CLR(int cpu, cpu_set_t *set);
int CPU_ISSET(int cpu, cpu_set_t *set);
int CPU_COUNT(cpu_set_t *set);
………………The specific functions are as follows:
PU_ZERO(): Clears the contents of the collection so that it does not contain any CPU.
CPU_SET(): Add cpu to the set.
CPU_CLR(): remove cpu from the set
CPU_ISSET() : Tests if the cpu is in the set.
CPU_COUNT(): Returns the number of CPUs contained in the collection.
In Linux, you can use the following two functions to set and get the CPU Affinity property of a process:
#define _GNU_SOURCE
#include <sched.h>
int sched_setaffinity(pid_t pid, size_t cpusetsize,const cpu_set_t *mask);
int sched_getaffinity(pid_t pid, size_t cpusetsize,cpu_set_t *mask);In addition, the following function can be used to know which CPU the current process is running on:
int sched_getcpu(void);
If the call is successful, the function returns a non-negative CPU number value.
Routine:
#define _GNU_SOURCE
#include <sched.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char *argv[])
{
cpu_set_t set;
int parentCPU, childCPU;
int j;
int cpu_num = -1;
if (argc != 3) {
fprintf(stderr, "Usage: %s parent-cpu child-cpu\n", argv[0]);
exit(EXIT_FAILURE);
}
parentCPU = atoi(argv[1]);
childCPU = atoi(argv[2]);
CPU_ZERO(&set);
switch (fork()) {
case -1: { /* Error */
fprintf(stderr, "fork error\n");
exit(EXIT_FAILURE);
}
case 0: { /* Child */
CPU_SET(childCPU, &set);
if (sched_setaffinity(getpid(), sizeof(set), &set) == -1) {
fprintf(stderr, "child sched_setaffinity error\n");
exit(EXIT_FAILURE);
}
sleep(1);
if (-1 != (cpu_num = sched_getcpu())) {
fprintf(stdout, "The child process is running on cpu %d\n", cpu_num);
}
exit(EXIT_SUCCESS);
}
default: { /* Parent */
CPU_SET(parentCPU, &set);
if (sched_setaffinity(getpid(), sizeof(set), &set) == -1) {
fprintf(stderr, "parent sched_setaffinity error\n");
exit(EXIT_FAILURE);
}
if (-1 != (cpu_num = sched_getcpu())) {
fprintf(stdout, "The parent process is running on cpu %d\n", cpu_num);
}
wait(NULL); /* Wait for child to terminate */
exit(EXIT_SUCCESS);
}
}
}The program first clears the CPU collection with CPU_ZERO, then calls the fork() function to create a child process, and calls the sched_setaffinity() function to set the CPU Affinity for the parent process and the child process respectively, and the input parameters parentCPU and childCPU specify the CPU numbers of the parent process and the child process respectively . Specify the CPUs of the parent process and the child process as 1 and 0, and the program output is as follows:
# ./affinity_test 1 0
The parent process is running on cpu 1
The child process is running on cpu 04. Thread and CPU binding
The binding between processes and CPUs was introduced earlier, so can threads be bound to CPUs? Of course it is possible. In Linux, you can use the following two functions to set and get the CPU Affinity property of a thread:
#define _GNU_SOURCE
#include <pthread.h>
int pthread_setaffinity_np(pthread_t thread, size_t cpusetsize, const cpu_set_t *cpuset);
int pthread_getaffinity_np(pthread_t thread, size_t cpusetsize, cpu_set_t *cpuset);Routine:
#define _GNU_SOURCE
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
static void *thread_start(void *arg)
{
......
struct thread_info *tinfo = arg;
thread = tinfo->thread_id;
CPU_ZERO(&cpuset);
CPU_SET(tinfo->thread_num, &cpuset);
s = pthread_setaffinity_np(thread, sizeof(cpu_set_t), &cpuset);
if (s != 0) {
handle_error_en(s, "pthread_setaffinity_np");
}
CPU_ZERO(&cpuset);
s = pthread_getaffinity_np(thread, sizeof(cpu_set_t), &cpuset);
if (s != 0) {
handle_error_en(s, "pthread_getaffinity_np");
}
for (j = 0; j < cpu_num; j++) {
if (CPU_ISSET(j, &cpuset)) { //如果当前线程运行在CPU j上，则输出信息
printf(" thread %d is running on cpu %d\n", tinfo->thread_num, j);
}
}
pthread_exit(NULL);
}
int main(int argc, char *argv[])
{
......
cpu_num = sysconf(_SC_NPROCESSORS_CONF); //获取系统的CPU数量
tinfo = calloc(cpu_num, sizeof(struct thread_info));
if (tinfo == NULL) {
handle_error_en(0, "calloc");
}
for (j = 0; j < cpu_num; j++) { //有多少个CPU就创建多少个线程
tinfo[j].thread_num = j;
s = pthread_create(&tinfo[j].thread_id, NULL, thread_start, &tinfo[j]);
if (s != 0) {
handle_error_en(s, "pthread_create");
}
}
for (j = 0; j < cpu_num; j++) {
s = pthread_join(tinfo[j].thread_id, NULL);
if (s != 0) {
handle_error_en(s, "pthread_join");
}
}
......
}The program first obtains the number of CPUs in the current system, cpu_num, and then creates threads according to the number of CPUs. There are as many threads as there are CPUs, and each thread runs on a different CPU. The results of running in a virtual machine (4 cores) are as follows:
$ ./thread_affinity
thread 1 is running on cpu 1
thread 0 is running on cpu 0
thread 3 is running on cpu 3
thread 2 is running on cpu 2
5. Use the taskset command to achieve CPU binding
The Linux taskset command is used to set or retrieve the CPU Affinity of a running process specified by pid, or to start a new process with a given CPU Affinity attribute. The CPU Affinity attribute is represented by a bit mask, wherein the lowest bit corresponds to the first logical CPU, and the last bit corresponds to the last logical CPU. The retrieved mask reflects only the bits corresponding to the CPUs on the physical system. If an invalid mask is given (i.e. there is no corresponding valid CPU mask on the current system), an error is returned. Masks are usually given in hexadecimal form. For example
0x00000001 means CPU #0,
0x00000003 means CPU #0 and #1,
0x0000000f means CPU #0 ~ #3
The options for the taskset command are as follows:
-a, --all-tasks
Set or retrieve the CPU Affinity property for all processes specified by pid.
-c, --cpu-list numbers
Specify a numeric list of processors, not a bitmask. Numbers are separated by commas and can include ranges. For example: 0,5,8-11.
-p, --pid
Operate the process specified by pid, do not start a new process.
The following uses the taskset command in Ubuntu 16.04 to illustrate how to use this command:
- Display the CPU command run by the process
: taskset -p 1
Result: pid 1's current affinity mask: f
Description: f indicates that process 1 is running on CPU#0~CPU#3
- Specify the process to run on a specific CPU
Command: taskset -cp 1,2 7737
Result: pid 7737's current affinity list: 0-3
pid 7737's new affinity list: 1,2
Description: This operation limits the process 7737 to run on CPU#1~CPU#2.
- Specify the CPU
command when the process starts: taskset -c 1-2 ./get_affinity
结果：This process is running on cpu 1
This process is running on cpu 2
Description: The get_affinity program obtains the CPU Affinity attribute of the current process through the sched_getaffinity() function and outputs prompt information.
6. Summary
This article introduces in detail the method of binding processes or threads to the CPU in the Linux environment through several routines, which has great reference value. If you find this article useful, please like and bookmark it.
Original Author: Proficient in Linux Kernel
Original address: Linux Kernel: Process Management: CPU Binding Technology - Zhihu (Copyright belongs to the original author, contact to delete infringement message)