Talk about Linux processes and threads from the perspective of the kernel

foreword

During the interview yesterday, the interviewer asked me a plain question –> "Talk about processes and threads in Linux";

Compared with everyone who has encountered this problem more or less in exams or interviews, as the saying goes:

Process is the basic unit of resource allocation, and thread is the basic unit of resource scheduling!

You and I both know this, but it feels a bit far-fetched and perfunctory to explain it this way. Remember that stereotypes are not memorization, but understanding, so that the interview can have depth, and with clear logical expressions, everything can be victorious!

In the discussion of processes and threads on the Internet, many of them focus on the difference between these two. But in fact, on Linux, the similarities between processes and threads are far greater than the differences . Threads under Linux are even called lightweight processes . (As for where the lightness is, our final conclusion will emerge later)

Today I will give you an in-depth comparison of processes and threads from the perspective of Linux kernel implementation.

The kernel's representation of processes and threads

Both are represented by task_struct

For threads, all task_struct fields are the same as processes (it is originally represented by a structure). Including status, pid, task tree relationship, address space, file system information, open file information and other fields, threads also have them.

---

For a process , this pid is what we usually call the process pid.

For threads , we assume that multiple threads are created under a process. Then the pid of each thread is different. But we generally need to record which process the thread belongs to. At this time, tgid comes in handy, and the process ID to which you belong is represented by the tgid field

In this way, the kernel can know which process the thread belongs to through tgid.

This is what I said earlier. The similarities between processes and threads are far greater than the differences . They are essentially the same thing, a task_struct ! Because process threads are so similar, threads under Linux have another name called lightweight process . (As for where the lightness is, our final conclusion will emerge later)

---

The process of creating a process

Each process under Linux is essentially a task_struct (PCB), which can refer to my article Linux virtual address space

In fact, when the process thread is created, the function used looks different. But in fact, in the underlying implementation, it is finally implemented using the same function:

By analyzing the kernel source code, the fork call is mainly to execute the do_fork function when creating a process .

Note: The parameters passed by the fork function call do_fork are SIGCHLD, 0, 0, NULL, NULL

//file:kernel/fork.c
SYSCALL_DEFINE0(fork)
{
    
    
 return do_fork(SIGCHLD, 0, 0, NULL, NULL);

The do_fork() function calls copy_process to complete the process creation.

//file:kernel/fork.c
long do_fork(...)
{
    
    
 //复制一个 task_struct 出来
 struct task_struct *p;
 p = copy_process(clone_flags, ...);
 ...
}

The source code of the copy_process() system call puts down the final introduction of the thread below, which is convenient for distinguishing the essential difference between creating a process and creating a thread

The process of creating a thread

Same as above:

By analyzing the source code, the pthread_create creation thread in the lib library will call the **clone()** system call, passing in a set of flags for it.

//file:nptl/sysdeps/pthread/createthread.c
static int
create_thread (struct pthread *pd, ...)
{
    
    
 int clone_flags = (CLONE_VM | CLONE_FS | CLONE_FILES | CLONE_SIGNAL
    | CLONE_SETTLS | CLONE_PARENT_SETTID
    | CLONE_CHILD_CLEARTID | CLONE_SYSVSEM
    | 0); //这个flag传入了下面的do_clone()
 
 int res = do_clone (pd, attr, clone_flags, ...);
 ...
}

do_clone() will execute a piece of assembly code and enter the clone system call:

//file:sysdeps/unix/sysv/linux/i386/clone.S
ENTRY (BP_SYM (__clone))
	...
	movl	$SYS_ify(clone),%eax
	...

Then look at the implementation of the system call clone() and find that do_fork() is called:

//file:kernel/fork.c
SYSCALL_DEFINE5(clone, ......)
{
    
    
 return do_fork(clone_flags, newsp, 0, parent_tidptr, child_tidptr);
}

After verifying the above figure, the do_fork() function in the kernel is still executed . The following is the source code of do_fork():

//file:kernel/fork.c
long do_fork(unsigned long clone_flags, ...)
{
    
    
 //复制一个 task_struct 出来
 struct task_struct *p;
 p = copy_process(clone_flags, stack_start, stack_size,
    child_tidptr, NULL, trace);
 
 //子任务加入到就绪队列中去，等待调度器调度
 wake_up_new_task(p);
 ...
}

After verifying the above figure, the do_fork() function further executes the copy_process() system call, and the source code is as follows:

//file:kernel/fork.c
static struct task_struct *copy_process(...)
{
    
    
 //4.1 复制进程 task_struct 结构体
 struct task_struct *p;
 p = dup_task_struct(current);
 ...
 
 //4.2 拷贝 files_struct
 retval = copy_files(clone_flags, p);
 
 //4.3 拷贝 fs_struct
 retval = copy_fs(clone_flags, p);
 
 //4.4 拷贝 mm_struct
 retval = copy_mm(clone_flags, p);
 
 //4.5 拷贝进程的命名空间 nsproxy
 retval = copy_namespaces(clone_flags, p);
 
 //4.6 申请 pid && 设置进程号
 pid = alloc_pid(p->nsproxy->pid_ns);
 p->pid = pid_nr(pid);
 p->tgid = p->pid;
 if (clone_flags & CLONE_THREAD)
  p->tgid = current->tgid;
 
 ......
}

It can be seen that copy_process() first copies task_struct, and then there are many copy functions for some fields in task_struct. The processing rules of these copy functions depend on the parameter clone_flags passed in by clone. This is also the most essential difference between process threads. Let's summarize it below!

---

Creating Processes and Threads Similarities and Differences

It can be seen that compared with the fork system call used when creating a process, the clone system call for creating a thread is almost the same as fork. It also uses the do_fork function in the kernel, and finally goes to copy_process to complete the creation.

However, the difference in the creation process is that the flags in the clone_flags passed in when calling do_fork are different! .

• Flag when creating a process: only one SIGCHLD
• Flag when creating a thread: including CLONE_VM, CLONE_FS, CLONE_FILES, CLONE_SIGNAL, CLONE_SETTLS, CLONE_PARENT_SETTID, CLONE_CHILD_CLEARTID, CLONE_SYSVSEM.

Regarding the meaning of these flags passed in by multiple threads , we choose a few key ones to give a brief introduction:

• CLONE_VM: The new task shares the address space with the parent process
• CLONE_FS: The new task shares file system information with the parent process
• CLONE_FILES: The new task shares the file descriptor table with the parent process

What impact will these flags have on task_struct, let's look at the next content:

Demystifying the do_fork system call

We saw earlier that both process and thread creation are performed by calling the do_fork function in the kernel. In the implementation of do_fork , the core is a copy_process function, which generates a new task_struct by copying the parent process (thread) .

//file:kernel/fork.c
long do_fork(unsigned long clone_flags, ...)
{
    
    
 //复制一个 task_struct 出来
 struct task_struct *p;
 p = copy_process(clone_flags, stack_start, stack_size,
    child_tidptr, NULL, trace);
 
 //子任务加入到就绪队列中去，等待调度器调度
 wake_up_new_task(p);
 ...
}

After creation , call wake_up_new_task to add the newly created task to the ready queue, waiting for the scheduler to schedule execution . This code is very long, I have simplified it to a certain extent:

//file:kernel/fork.c
static struct task_struct *copy_process(...)
{
    
    
 //4.1 复制进程 task_struct 结构体
 struct task_struct *p;
 p = dup_task_struct(current);
 ...
 
 //4.2 拷贝 files_struct
 retval = copy_files(clone_flags, p);
 
 //4.3 拷贝 fs_struct
 retval = copy_fs(clone_flags, p);
 
 //4.4 拷贝 mm_struct
 retval = copy_mm(clone_flags, p);
 
 //4.5 拷贝进程的命名空间 nsproxy
 retval = copy_namespaces(clone_flags, p);
 
 //4.6 申请 pid && 设置进程号
 pid = alloc_pid(p->nsproxy->pid_ns);
 p->pid = pid_nr(pid);
 p->tgid = p->pid;
 if (clone_flags & CLONE_THREAD)
  p->tgid = current->tgid;
 
 ......
}

It can be seen that copy_process first copies a new task_struct , and then calls the copy_xxx series of functions to copy various core fields in the task_struct , and also applies for a new pid.

---

The processing method of various fields in copy task_struct is the key point of distinguishing process and thread:

• During the creation process, because the shared fields in task_struct are not passed in, they all have an extra copy and have their own memory space (note that there is copy-on-write)

The process of accessing physical memory through the mm_struct field:

• When creating a thread, because the shared flag bits of various fields in task_struct are passed in, each field corresponding to task_struct is shared

The process of accessing physical memory through the mm_struct field:

in conclusion

1. For the process , the core fields in various task_structs such as the address space mm_struct, the mount point fs_struct, and the open file list files_struct must be independently owned , and new memory needs to be applied for and initialized. (This also reflects that fields are not shared between processes, the basic unit of resource allocation)

2. For a thread , its address space mm_struct, directory information fs_struct, open file list files_struct and other core fields in various task_structs are shared with the task that created it ; (this also reflects the thread and the process that created it, fields, etc. Resource sharing, but the thread has a unique pid, which is the basic unit of scheduling)

In short, there is no special treatment for threads in the Linux kernel , and they are still managed by task_struct . From the perspective of the kernel , a thread in user mode is essentially a process . It’s just that it’s a little “ lightweight ” compared to ordinary processes (lightweight: shared fields)