How Linux workqueue works - a brief introduction

http://blog.csdn.net/myarrow/article/details/8090504

1. What is workqueue
       The Workqueue mechanism in Linux is to simplify the creation of kernel threads. Kernel threads can be created by calling the interface of workqueue. And the number of threads can be created according to the number of current system CPUs, so that the transactions processed by the threads can be parallelized. Workqueue is a simple and effective mechanism in the kernel, which obviously simplifies the creation of kernel daemons and facilitates user programming.

      Workqueue (workqueue) is another form of postponing the execution of work. The work queue can postpone the work to a kernel thread for execution, that is, the lower part can be executed in the context of the process. The most important thing is that the work queue can be rescheduled or even sleep.

2. Data structure
     We call the task executed later as work, and the data structure describing it is work_struct:

 

[cpp] view plain copy

1. struct work_struct {  
2.     atomic_long_t data;        /*Parameters of the work processing function func*/  
3. #define WORK_STRUCT_PENDING 0        /* T if work item pending execution */  
4. #define WORK_STRUCT_STATIC 1        /* static initializer (debugobjects) */  
5. #define WORK_STRUCT_FLAG_MASK (3UL)  
6. #define WORK_STRUCT_WQ_DATA_MASK (~WORK_STRUCT_FLAG_MASK)  
7.     struct  list_head entry;         /*Pointer for connection work*/  
8.     work_func_t func;               /*work processing function*/  
9. #ifdef CONFIG_LOCKDEP  
10.     struct lockdep_map lockdep_map;  
11. #endif  
12. };  

      These jobs are organized in a queue structure into a workqueue (workqueue) whose data structure is workqueue_struct:

[cpp] view plain copy

1. struct workqueue_struct {  
2.  struct cpu_workqueue_struct *cpu_wq;  
3.  struct list_head list;  
4.  constchar *name;   /*workqueue name*/   
5.  int  singlethread;    /*is it single thread - single thread we prefer the first CPU -0 means use the default worker thread event*/  
6.  int freezeable;  /* Freeze threads during suspend */  
7.  int rt;  
8. };   

     If it is multi-threaded, Linux creates cpu_workqueue_struct according to the number of current system CPUs, and its structure is:

[cpp] view plain copy

1. truct cpu_workqueue_struct {  
2.  spinlock_t lock; /* Because the worker thread needs to frequently process the work connected to it, it needs to be protected by a shackle */  
3.  struct list_head worklist;  
4.  wait_queue_head_t more_work;  
5.  struct  work_struct *current_work;  /*current work*/  
6.  struct  workqueue_struct *wq;    /*The workqueue it belongs to*/  
7.  struct  task_struct * thread ;  /*task context*/  
8. } ____cacheline_aligned;  

       In this structure, it mainly maintains a task queue and a waiting queue that the kernel thread needs to sleep, and also maintains a task context, that is, task_struct.
       The relationship between the three is as follows:

 

3. Create work
3.1 Create work queue
a. create_singlethread_workqueue(name)
        The implementation mechanism of this function is shown in the figure below. The function returns a pointer variable of type struct workqueue_struct, and the memory address pointed to by the pointer variable is dynamically generated by calling kzalloc inside the function . So the driver calls without using the work queue:

        void destroy_workqueue(struct workqueue_struct *wq) to release the memory address here.

 

        The cwq in the figure is a per-CPU type address space. For create_singlethread_workqueue, even for multi-CPU systems, the kernel is only responsible for creating a worker_thread kernel process. After the kernel process is created, it will first define a wait node in the graph, and then check the worklist in cwq in a loop body. If the queue is empty, then the wait node will be added to more_work in cwq, and then Sleep in this waiting queue.

        Driver calls queue_work (struct workqueue_struct *wq, struct work_struct *work) to add work nodes to wq. Work will be added to the linked list pointed to by cwq->worklist in turn. queue_work adds a work node to cwq->worklist, and calls wake_up to wake up the worker_thread process sleeping on cwq->more_work. wake_up will first call the autoremove_wake_function function on the wait node, and then remove the wait node from cwq->more_work.

        The worker_thread is scheduled again and starts to process all the work nodes in the cwq->worklist... When all the work nodes are processed, the worker_thread re-adds the wait node to the cwq->more_work, and then sleeps again in the waiting queue until the Driver calls queue_work ...

b. create_workqueue

 

 

 

       相对于create_singlethread_workqueue, create_workqueue同样会分配一个wq的工作队列，但是不同之处在于，对于多CPU系统而言，对每一个CPU，都会为之创建一个per-CPU的cwq结构，对应每一个cwq，都会生成一个新的worker_thread进程。但是当用queue_work向cwq上提交work节点时，是哪个CPU调用该函数，那么便向该CPU对应的cwq上的worklist上增加work节点。

c.小结
       当用户调用workqueue的初始化接口create_workqueue或者create_singlethread_workqueue对workqueue队列进行初始化时，内核就开始为用户分配一个workqueue对象，并且将其链到一个全局的workqueue队列中。然后Linux根据当前CPU的情况，为workqueue对象分配与CPU个数相同的cpu_workqueue_struct对象，每个cpu_workqueue_struct对象都会存在一条任务队列。紧接着，Linux为每个cpu_workqueue_struct对象分配一个内核thread，即内核daemon去处理每个队列中的任务。至此，用户调用初始化接口将workqueue初始化完毕，返回workqueue的指针。

        workqueue初始化完毕之后，将任务运行的上下文环境构建起来了，但是具体还没有可执行的任务，所以，需要定义具体的work_struct对象。然后将work_struct加入到任务队列中，Linux会唤醒daemon去处理任务。

       上述描述的workqueue内核实现原理可以描述如下：

 

 

3.2  创建工作
       要使用工作队列，首先要做的是创建一些需要推后完成的工作。可以通过DECLARE_WORK在编译时静态地建该结构：
       DECLARE_WORK(name,void (*func) (void *), void *data);
      这样就会静态地创建一个名为name，待执行函数为func，参数为data的work_struct结构。
      同样，也可以在运行时通过指针创建一个工作：
      INIT_WORK(structwork_struct *work, woid(*func) (void *), void *data);

4. 调度
a. schedule_work

       在大多数情况下, 并不需要自己建立工作队列，而是只定义工作, 将工作结构挂接到内核预定义的事件工作队列中调度, 在kernel/workqueue.c中定义了一个静态全局量的工作队列static struct workqueue_struct *keventd_wq;默认的工作者线程叫做events/n，这里n是处理器的编号，每个处理器对应一个线程。比如，单处理器的系统只有events/0这样一个线程。而双处理器的系统就会多一个events/1线程。
       调度工作结构, 将工作结构添加到全局的事件工作队列keventd_wq，调用了queue_work通用模块。对外屏蔽了keventd_wq的接口，用户无需知道此参数，相当于使用了默认参数。keventd_wq由内核自己维护，创建，销毁。这样work马上就会被调度，一旦其所在的处理器上的工作者线程被唤醒，它就会被执行。

b. schedule_delayed_work(&work,delay);
      有时候并不希望工作马上就被执行，而是希望它经过一段延迟以后再执行。在这种情况下，同时也可以利用timer来进行延时调度，到期后才由默认的定时器回调函数进行工作注册。延迟delay后，被定时器唤醒，将work添加到工作队列wq中。

      工作队列是没有优先级的，基本按照FIFO的方式进行处理。

 5. 示例

[cpp] view plain copy

1. #include <linux/module.h>  
2. #include <linux/init.h>  
3. #include <linux/workqueue.h>  
4.   
5. static struct workqueue_struct *queue=NULL;  
6. static struct work_struct   work;  
7.   
8. staticvoid work_handler(struct work_struct *data)  
9. {  
10.        printk(KERN_ALERT"work handler function.\n");  
11. }  
12.   
13. static int __init test_init(void)  
14. {  
15.       queue=create_singlethread_workqueue("hello world");/*创建一个单线程的工作队列*/  
16.       if (!queue)  
17.             goto err;  
18.   
19.        INIT_WORK(&work,work_handler);  
20.        schedule_work(&work);  
21.   
22.       return0;  
23. err:  
24.       return-1;  
25. }  
26.   
27. static   void __exit test_exit(void)  
28. {  
29.        destroy_workqueue(queue);  
30. }  
31. MODULE_LICENSE("GPL");  
32. module_init(test_init);  
33. module_exit(test_exit);  

序号	接口函数	说明
1	create_workqueue	用于创建一个workqueue队列，为系统中的每个CPU都创建一个内核线程。输入参数： @name：workqueue的名称
2	create_singlethread_workqueue	用于创建workqueue，只创建一个内核线程。输入参数： @name：workqueue名称
3	destroy_workqueue	释放workqueue队列。输入参数： @ workqueue_struct：需要释放的workqueue队列指针
4	schedule_work	Schedule the execution of a specific task, and the executed task will be hooked into the workqueue provided by the Linux system—keventd_wq input parameters: @work_struct: concrete task object pointer
5	schedule_delayed_work	Delay a certain time to perform a specific task. The function is similar to schedule_work, with an additional delay time. Input parameters: @work_struct: specific task object pointer @delay: delay time
6	queue_work	Schedules the execution of a task in a specified workqueue. Input parameters: @workqueue_struct: The specified workqueue pointer @work_struct: specific task object pointer
7	queue_delayed_work	Delay scheduling executes a task in a specified workqueue. The function is similar to queue_work, and the input parameter has an additional delay.

Reprinted from: http://bgutech.blog.163.com/blog/static/18261124320116181119889/