| The thread pool is also a multi-threaded processing method. It is to add the task proposed by the "producer" thread to the "task queue", and then some threads automatically complete the tasks on the "task queue". |
Multi-threaded programming, create a thread, specify to complete a certain task, and wait for the thread to exit. Although it can meet the programming needs, when we need to create a large number of threads, a large amount of CPU may be consumed during the creation process and the process of destroying the threads. It increases a lot of overhead. Such as: the copy of the folder, the response of the WEB server.
The thread pool is used to solve a problem similar to this, which can reduce the overhead caused by frequent creation and destruction of threads.
Thread pool technology idea: Generally, pre-created thread technology is used, that is, a certain number of threads need to be created in advance. After these threads are created in advance, if there are no tasks in the "task queue", let these threads sleep. Once there are tasks, wake up the threads to perform the tasks. After the tasks are executed, there is no need to destroy the threads until you want to When exiting or shutting down, at this time, you call the function of destroying the thread pool to destroy the thread.
After the thread completes the task, it will not be destroyed, but will automatically execute the next task. Moreover, when there are many tasks, you can have a functional interface to increase the number of threads, and when there are fewer tasks, you can have a functional interface to destroy some threads.
If the time to create and destroy threads is negligible compared to the time to execute tasks, then there is no need to use thread pools in this case.
"Task queue" is a shared resource "mutually exclusive access"
The thread pool is essentially a data structure, and a structure is needed to describe it:
struct pthread_pool //implementation of thread pool
{
//Generally there will be the following members
//Mutual exclusion lock, used to protect this "task queue"
pthread_mutex_t lock; //mutex lock
//Thread condition variable indicates whether there is a task in the "task queue"
pthread_cond_t cond; //Condition variable
bool shutdown; //Indicates whether to exit the program bool: type false / true
//Task queue (linked list), pointing to the first task that needs to be pointed to
//All threads get the task "shared resource" from the task list
struct task * task_list;
//There are multiple threads in the thread pool, and each thread has tid, and an array is needed to save tid
pthread_t * tids; //malloc()
//The number of threads in service in the thread pool, the current number of threads
unsigned int active_threads;
//The maximum number of tasks in the thread pool task queue
unsigned int max_waiting_tasks;
//How many tasks are currently on the thread pool task queue
unsigned int cur_waiting_tasks;
//......
};
//The task node on the task queue (linked list), as long as it can describe a task well,
//The thread will continue to take tasks from the task queue
struct task //task node
{
// 1. The task represented by the task node, the "function pointer" points to the function to be executed by the task (cp_file)
void*(* do_task)(void * arg);
//2. Pointer, point to the parameter of the task pointing to the function (file descriptor)
void * arg;
//3. Pointer of task node type, pointing to the next task
struct task * next;
};
The thread pool framework code is as follows, and the function is self-filled:
Function interface required for operating thread pool: pthread_pool.c, pthread_pool.h
Think of the "thread pool" as an outsourcing company. What you need to accomplish is to operate the function interface provided by the thread pool.
pthread_pool.c
#include "pthread_pool.h"
/*
init_pool: thread pool initialization function, initialize thread_num initial threads in the specified thread pool
@pool: pointer to the thread pool you want to initialize
@threa_num: The number of threads starting in the thread pool you want to initialize
return value:
Success 0
Fail-1
*/
int init_pool(pthread_pool * pool , unsigned int threa_num)
{
//Initialize the structure of the thread pool
//Initialize thread mutex
pthread_mutex_init(&pool->lock, NULL);
//Initialize thread condition variables
pthread_cond_init(&pool->cond, NULL);
pool->shutdown = false ;// Do not exit
pool->task_list = (struct task*)malloc(sizeof(struct task));
pool->tids = (pthread_t *)malloc(sizeof(pthread_t) * MAX_ACTIVE_THREADS);
if(pool->task_list == NULL || pool->tids == NULL)
{
perror("malloc memery error");
return -1;
}
pool->task_list->next = NULL;
//How many threads are initialized to serve in the thread pool at the beginning
pool->active_threads = threa_num;
//Indicates the maximum number of tasks in the thread pool
pool->max_waiting_tasks = MAX_WAITING_TASKS;
//The current number of tasks in the task queue in the thread pool
pool->cur_waiting_tasks = 0;
//Create thread_num threads, and let the threads perform task allocation functions,
//Record the tid of all threads
int i = 0;
for(i = 0; i < threa_num; i++)
{
int ret = pthread_create(&(pool->tids)[i], NULL, routine, (void*)pool);
if (ret! = 0)
{
perror("create thread error");
return -1;
}
printf("[%lu]:[%s] ===> tids[%d]:[%lu]",pthread_self(),
__FUNCTION__, i , pool->tids[i]);
}
return 0;
}
/*
routine: Task deployment function.
All threads execute this function at the beginning, this function will continue from the task queue of the thread pool
Take down the task node and execute it.
The task node contains "function pointer" h "function parameter"
*/
void * routine(void * arg)
{
//arg represents the pointer of your thread pool
while()
{
//Get thread mutex lock, lock
//When the thread pool is not over, continuously remove the node from the task queue of the thread pool
//Go to execute.
//Release thread mutex, unlock
//Release the task node
}
}
/*
destroy_pool: Destroy the thread pool, ensure that all tasks have been completed before destruction
*/
int destroy_pool(pthread_pool * pool)
{
//Release all space and wait for the task to complete (join).
//Wake up all threads
//Use the join function to reclaim each thread resource.
}
/*
add_task: add tasks to the task queue, the task pointed to by do_task (function pointer) and
The parameter pointed to by arg is saved to a task node and added to the pool task queue.
@pool: The thread pool where you want to add tasks
@do_task: The task you need to add (cp_file)
@arg: The parameter (file descriptor) of the task you want to perform
*/
int add_task(pthread_pool *pool,void*(* do_task)(void * arg), void*arg)
{
//Encapsulate the second parameter and the third parameter into struct task
//Add it to the pool->task task queue
//Note that the task queue is a shared resource
//If you want to wake up the waiting thread after the task.
}
//If there are many tasks, add threads pthread_create to the thread pool
int add_threads(pthread_pool * pool, unsigned int num);
{
//Create num threads newly, let each thread execute the thread allocation function
//Add each newly created thread tid to pool->tids
}
//If there are few tasks, reduce the number of threads in the thread pool pthread_cancel join
int remove_threads(pthread_pool * pool, unsigned int num)
{
//Use pthread_cancel to cancel num threads
//Use the pthread_join function to reclaim resources.
}
pthread_pool.h
#ifndef __PTHREAD_POOL_H__
#define __PTHREAD_POOL_H__
//Indicates the maximum number of threads in the thread pool
#define MAX_ACTIVE_THREADS 20
//Indicates the maximum number of tasks in the thread pool
#define MAX_WAITING_TASKS 1024
//The task node on the task queue (linked list), as long as it can describe a task well,
//The thread will continue to take tasks from the task queue
struct task //task node
{
// 1. The task represented by the task node, the "function pointer" points to the function to be executed by the task (cp_file)
void*(* do_task)(void * arg);
//2. Pointer, point to the parameter of the task pointing to the function (file descriptor)
void * arg;
//3. Pointer of task node type, pointing to the next task
struct task * next;
};
struct pthread_pool //implementation of thread pool
{
//Generally there will be the following members
//Mutual exclusion lock, used to protect this "task queue"
pthread_mutex_t lock; //mutex lock
//Thread condition variable indicates whether there is a task in the "task queue"
pthread_cond_t cond; //Condition variable
bool shutdown; //Indicates whether to exit the program bool: type false / true
//Task queue (linked list), pointing to the first task that needs to be pointed to
//All threads get the task "shared resource" from the task list
struct task * task_list;
//There are multiple threads in the thread pool, and each thread has tid, and an array is needed to save tid
pthread_t * tids; //malloc()
//The number of threads in service in the thread pool, the current number of threads
unsigned int active_threads;
//The maximum number of tasks in the thread pool task queue
unsigned int max_waiting_tasks;
//How many tasks are currently on the thread pool task queue
unsigned int cur_waiting_tasks;
//......
};
/*
init_pool: thread pool initialization function, initialize thread_num in the specified thread pool
Initial thread
@pool: pointer to the thread pool you want to initialize
@threa_num: The number of threads starting in the thread pool you want to initialize
return value:
Success 0
Fail-1
*/
int init_pool(pthread_pool * pool , unsigned int threa_num);
/*
routine: Task deployment function.
All threads execute this function at the beginning, this function will continue from the task queue of the thread pool
Take down the task node and execute it.
The task node contains "function pointer" h "function parameter"
*/
void * routine(void * arg);
/*
destroy_pool: Destroy the thread pool, ensure that all tasks have been completed before destruction
*/
int destroy_pool(pthread_pool * pool);
/*
add_task: add tasks to the task queue, the task pointed to by do_task (function pointer) and
The parameter pointed to by arg is saved to a task node and added to the pool task queue.
@pool: The thread pool where you want to add tasks
@do_task: The task you need to add (cp_file)
@arg: The parameter (file descriptor) of the task you want to perform
*/
int add_task(pthread_pool *pool,void*(* do_task)(void * arg), void*arg);
//If there are many tasks, add threads pthread_create to the thread pool
int add_threads(pthread_pool * pool, unsigned int num);
//If there are few tasks, reduce the number of threads in the thread pool pthread_cancel join
int remove_threads(pthread_pool * pool, unsigned int num);
#endif