Kernel soft interrupt tasklet mechanism

1. Introduction to soft interrupt IRQ

Soft interrupt (SoftIRQ) is an interrupt-based delay mechanism provided by the kernel. The Linux kernel defines the following soft interrupts:

enum
{
	HI_SOFTIRQ=0,			/*高优先级的tasklet*/
	TIMER_SOFTIRQ,
	NET_TX_SOFTIRQ,
	NET_RX_SOFTIRQ,
	BLOCK_SOFTIRQ,
	TASKLET_SOFTIRQ,  /*普通tasklet*/
	SCHED_SOFTIRQ,
#ifdef CONFIG_HIGH_RES_TIMERS
	HRTIMER_SOFTIRQ,
#endif
};

In addition, the kernel maintains a global IRQ array to record detailed information of different soft interrupts:

static struct softirq_action softirq_vec[32]  __cacheline_aligned_in_smp;

The type of the global variable is struct softirq_action , which is the function prototype of all IRQ interrupt processing operations. That is to say , the global variables maintained by the system record the information of each IRQ interrupt processing function , but its structure is very simple:

struct softirq_action
{
	void	(*action)(struct softirq_action *); /*中断处理函数指针*/
	void	*data; /*传递的参数*/
};

Since kernel developers do not recommend that we add custom soft interrupts, we provide a tasklet mechanism for this purpose. We can use tasklets to implement our own interrupt handling tasks. Tasklet is a task delay mechanism based on soft interrupt provided by the kernel to developers. The tasklet is just one of several softirqs defined by the kernel. Device drivers often implement certain delay operations through tasklets. In the soft interrupt tasklet is divided into two categories: one is high-priority HI_SOFTIRQ ; the other is TASKLET_SOFTIRQ . The detailed information about tasklet is introduced below:

2. tasklet_struct of tasklet structure

struct tasklet_struct
{
	struct tasklet_struct *next;
	unsigned long state;
	atomic_t count;
	void (*func)(unsigned long);
	unsigned long data;
};

Serial number	variable name	effect
1	next	Used to connect tasklet objects in the system to form a linked list
2	state	Record the status of the tasklet: TASKLET_STATE_SCHED indicates that the tasklet has been submitted. TASKLET_STATE_RUN indicates that the tasklet is executing (only for SMP)
3	count	0: enable, can be scheduled for execution. 1: disable, not executable;
4	func	Tasklet executive body, implemented by the developer
5	data	Parameters passed to func

3. Tasklet mechanism initialization

During the initialization of the Linux system, the softirq_init function is called to install the function execution bodies for HI_SOFTIRQ and TASKLET_SOFTIRQ , respectively tasklet_hi_action and tasklet_action :

void __init softirq_init(void)
{
	open_softirq(TASKLET_SOFTIRQ, tasklet_action, NULL);
	open_softirq(HI_SOFTIRQ, tasklet_hi_action, NULL);
}

The so-called installation execution function is to initialize the member parameters (a callback function pointer and a passed parameter) corresponding to these two interrupts on the IRQ global variable, and this part is realized by open_softirq (). The open_softirq () function is very simple to implement:

static struct softirq_action softirq_vec[32] __cacheline_aligned_in_smp;
void open_softirq(int nr, void (*action)(struct softirq_action*), void *data)
{
	softirq_vec[nr].data = data;
	softirq_vec[nr].action = action;
}

Through the execution of the above two functions, the initialization task of the tasklet mechanism is completed, namely:

softirq_vec[TASKLET_SOFTIRQ].action = tasklet_hi_action;
softirq_vec[HI_SOFTIRQ].action = tasklet_action;

4. Tasklet initialization

There are two initialization methods for tasklet objects, one is static mode and the other is dynamic mode :

4.1 Tasklet object static initialization

The declaration of the tasklet object consists of two functions. When the object is declared, the object name of the tasklet , the delay operation function func, and the parameters to be passed need to be implemented.

#define DECLARE_TASKLET(name, func, data) \     /*可以被调度执行*/
struct tasklet_struct name = { NULL, 0, ATOMIC_INIT(0), func, data }

#define DECLARE_TASKLET_DISABLED(name, func, data) \ /*不可以被调度执行*/
struct tasklet_struct name = { NULL, 0, ATOMIC_INIT(1), func, data }

Objects declared using DECLARE_TASKLET are in the enabled state, objects declared using DECLARE_TASKLET_DISABLED are in the disabled state.

4.2 Dynamic initialization of tasklet objects

Dynamic initialization is to directly define a tasklet_struct variable, and then use the tasklet_init () function to complete the initialization operation.

void tasklet_init(struct tasklet_struct *t,
		  void (*func)(unsigned long), unsigned long data)
{
	t->next = NULL;
	t->state = 0;
	atomic_set(&t->count, 0);
	t->func = func;
	t->data = data;
}

5. Tasklet object submission

The so-called tasklet objects submitted, tasket actually added to the declaration of tasklet_vec list managed, tasklet_vec is a per-CPU variable of type used in the system all tasklet objects submitted by tasklet_schedule function constitute the list. (linux-2.6.22)

void fastcall __tasklet_schedule(struct tasklet_struct *t)
{
	unsigned long flags;

	local_irq_save(flags);
	t->next = __get_cpu_var(tasklet_vec).list; /*使用头插法插入一个新节点 */
	__get_cpu_var(tasklet_vec).list = t;
	raise_softirq_irqoff(TASKLET_SOFTIRQ);
	local_irq_restore(flags);
}
static inline void tasklet_schedule(struct tasklet_struct *t)
{
	if (!test_and_set_bit(TASKLET_STATE_SCHED, &t->state))/*调度位为0，再提交；如果为1，则证明已经提交，无需再提交*/
		__tasklet_schedule(t);
}

• test_and_set_bit (nr, vaddr) function: modify the value of vaddr to nr, and return the value when vaddr is not modified.
• local_irq_save : Turn off local interrupt response.
• local_irq_restore : Turn on local interrupt.
• Note: The condition when scheduling is that the state is not set bit TASKLET_STATE_SCHED; and the tasklet's execution authority is judged by count, and the tasklet can only be executed when count = 0 .

The analysis is as follows:

• When scheduling a tasklet object, if the state bit is not TASKLET_STATE_SCHED, it means that the tasklet object has not been scheduled, then set the state value to TASKLET_STATE_SCHED (if the state value is already TASKLET_STATE_SCHED, indicating that it has been scheduled, there is no need to reschedule);
• Close the local interrupt, and then insert the tasklet object into the tasklet_vec list. The Linux2.6.22 kernel uses the head interpolation method , and some Linux versions use the tail interpolation method , and the members of struct tasklet_head are different.
• Send a software interrupt request, raise_softirq_irqoff (TASKLET_SOFTIRQ) to notify the kernel that there is a == “TASKLET_SOFTIRQ” == waiting for processing on the current processor, it uses an integer variable bit to indicate that the bit is pending IRQ operation, 0 means no, 1 means yes.
• Re-enable the local interrupt.

5. Tasklet internal execution body

After the tasklet task is submitted to the corresponding linked list of the system by tasklet_schedule , the system will traverse the linked list and then execute the tasklet task in sequence. Since the two interrupts HI_SOFTIRQ and TASKLET_SOFTIRQ are only different in priority, the other is basically the same in processing. Here only introduces the tasklet_action of TASKLET_SOFTIRQ .

static void tasklet_action(struct softirq_action *a)
{
	struct tasklet_struct *list;
	local_irq_disable();
	list = __get_cpu_var(tasklet_vec).list;     /*从tasklet_vec上取下链表，实际是链表的头指针*/
	__get_cpu_var(tasklet_vec).list = NULL; /*清空tasklet_vec链表，实际只是清list指针*/
	local_irq_enable();
	while (list) {
		struct tasklet_struct *t = list;
		list = list->next;
		if (tasklet_trylock(t)) {
			if (!atomic_read(&t->count)) {/*count=0  使能*/
				if (!test_and_clear_bit(TASKLET_STATE_SCHED, &t->state))/*清空调度位，返回原来值*/
					BUG();/*如果原来为0，属异常情况，应结束退出*/
				t->func(t->data);
				tasklet_unlock(t);
				continue;
			}
			tasklet_unlock(t);
		}

		local_irq_disable();
		t->next = __get_cpu_var(tasklet_vec).list;/*重新插入到tasklet_var中*/
		__get_cpu_var(tasklet_vec).list = t;
		__raise_softirq_irqoff(TASKLET_SOFTIRQ);
		local_irq_enable();
	}
}

• On the premise of turning off the local interrupt, move the pending tasklet_vec list head pointer on the current CPU to the temporary variable list and record it, then clear the tasklet list, and then re-enable the local interrupt. The purpose of re-local interruption is to enable the system to reschedule new tasklet tasks to the tasklet_vec list;
• Traverse the linked list
  • Use tasklet_trylock to determine whether the current tasklet task is processed on other CPUs;
    • Get the count value of the current tasklet task. If count = 0, it means that the current task (enable) can be scheduled for execution. If the value is 1, it means that the current tasklet task (disable) cannot be scheduled;
    • Check the TASKLET_STATE_SCHED bit of the state, if it is 1, it will be submitted normally; but if it is 0, it means that the current processing function is scheduling a tasklet that has not been submitted. This is an abnormal situation, and it will directly exit and return;
    • Execute the execution body registered by tasklet
    • Unlock and re-execute the next tasklet task.
  • If the return value of tasklet_trylock is 0, it means that the task may be processed on other CPUs, then you need to resubmit the tasklet task to the tasklet_vec list, and then trigger a soft interrupt to wait for the next soft interrupt to be re-executed;
• As you can see from the above code,Before a tasklet task is executed, the TASKLET_STATE_SCHED on its state is cleared, which means that unless the tasklet is resubmitted, the next soft interrupt will not execute the tasklet task again. This is a one-shot feature: submit once, schedule once, and delete it from the CPU's tasklet_var list after running, unless the tasklet task is submitted again .
• When the tasklet is executed, the interrupt is re-enabled, which is the original intention of the soft interrupt design. If during the execution, other soft mid-segments arrive, and the new tasklet is also on the CPU, then the new tasklet will be scheduled to tasklet_vec , at this time will not affect the list of executing lists, the tasket task has been executed TASKLET_STATE_SCHED Is cleared, and TASKLET_STATE_SCHED of the new tasklet task is enabled, the new tasklet will be executed in the next IRQ schedule

6. Tasklet other operations

6.1 tasklet_disable

tasklet_disable and tasklet_disable_nosync can make the tasklet task cannot be scheduled to run, in fact the most critical operation is atomic_inc (& t-> count); because when count = 0 ( enable ), the tasklet can be called and run.

static inline void tasklet_disable_nosync(struct tasklet_struct *t)
{
	atomic_inc(&t->count);
	smp_mb__after_atomic_inc();
}
static inline void tasklet_disable(struct tasklet_struct *t)
{
	tasklet_disable_nosync(t);
	tasklet_unlock_wait(t);
	smp_mb();
}

6.2 tasklet_enable

tasklet_enable is to achieve the tasklet can be scheduled to run by the count-1 in the tasklet object.
Note: A tasklet in the disabled state can be submitted to tasklet_vec, but it will not be scheduled for execution

static inline void tasklet_enable(struct tasklet_struct *t)
{
	smp_mb__before_atomic_dec();
	atomic_dec(&t->count);
}

6.3 tasklet_kill

This function is used to clear the status bit of TASKLET_STATE_SCHED on a tasklet object , so that IRQ can no longer schedule and run this task. If the current tasklet task is running, then tasklet_kill will be busy until the task is completed; if an object has been submitted to the system but has not been scheduled to run, then tasklet_kill will sleep until the tasklet is deleted from the tasklet_vec list. Therefore, the function is a function that may be blocked.

• It can be seen that after a tasklet is scheduled, it will be executed once.

void tasklet_kill(struct tasklet_struct *t)
{
	if (in_interrupt())
		printk("Attempt to kill tasklet from interrupt\n");

	while (test_and_set_bit(TASKLET_STATE_SCHED, &t->state)) {
		do
			yield();
		while (test_bit(TASKLET_STATE_SCHED, &t->state));
	}
	tasklet_unlock_wait(t);
	clear_bit(TASKLET_STATE_SCHED, &t->state);
}