Linux System Programming (7): Thread Synchronization

References

• Advanced Programming in UNIX Environment (3rd Edition)
• Dark Horse Programmer-Linux System Programming

1. Synchronization concept

• The so-called synchronization means starting at the same time and in coordination. Different objects have slightly different understandings of "synchronization"
  • Device synchronization refers to specifying a common time reference between two devices
  • Database synchronization refers to making the contents of two or more databases consistent, or partially consistent as needed.
  • File synchronization refers to keeping files in two or more folders consistent
• Synchronization in programming and communication: The word "same" should refer to collaboration, assistance, and mutual cooperation. The purpose is to coordinate and run in a predetermined order.

1.1 Thread synchronization

• Thread synchronization means that when a thread issues a certain function call, the call does not return until the result is obtained. At the same time, other threads cannot call this function to ensure data consistency.

• The resulting phenomenon is called "time related error" (time related). To avoid this data confusion, threads need to be synchronized
  • The purpose of "synchronization" is to avoid data confusion and solve time-related errors. In fact, not only synchronization is needed between threads, but also synchronization mechanisms are needed between processes , signals, etc. Therefore, all situations where "multiple control flows jointly operate a shared resource" require synchronization

1.2 Reasons for data confusion

• 1. Resource sharing (exclusive resources will not)
• 2. Scheduling is random (meaning there will be competition for data access)
• 3. Lack of necessary synchronization mechanism between threads

Among the above three points, the first two points cannot be changed. To improve efficiency, resources must be shared when transmitting data. As long as resources are shared, competition will inevitably occur. As long as there is competition, data will easily become chaotic. So we can only solve it from point 3, so that multiple threads are mutually exclusive when accessing shared resources.

2. Mutex mutex

• Linux provides a mutex lock (also called a mutex). Each thread attempts to lock before operating on the resource. It can operate only after successfully locking, and it will be unlocked after the operation . Resources are still shared, and threads still compete, but through "locks" resource access becomes a mutually exclusive operation.

• But it should be noted that only one thread can hold the lock at the same time . When thread A locks and accesses a global variable, B tries to lock it before accessing it. If it cannot get the lock, B blocks. The C thread does not lock, but directly accesses the global variable. It can still be accessed, but data confusion will occur.
  • Therefore, the mutex lock is essentially a "suggestion lock" (also known as "collaborative lock") provided by the operating system . It is recommended to use this mechanism when multiple threads in the program access shared resources, and there is no mandatory limit.
  • Therefore, even with mutex, if a thread accesses data without following the rules, data chaos will still occur.

2.1 Main application functions

#include <pthread.h>

// 返回值都是: 成功返回 0，失败返回错误号
// pthread_mutex_t 类型，其本质是一个结构体
// pthread_mutex_t mutex; 变量 mutex 只有两种取值 1、0

// 初始化一个互斥锁(互斥量) --> 初值可看作 1
// 参 1: 传出参数，调用时应传 &mutex
// restrict 用来限定指针变量。被该关键字限定的指针变量所指向的内存操作，必须由本指针完成
// 参 2: 互斥量属性。是一个传入参数，通常传 NULL，选用默认属性(线程间共享)
int pthread_mutex_init(pthread_mutex_t *restrict mutex, const pthread_mutexattr_t *restrict attr);

// 销毁一个互斥锁
int pthread_mutex_destroy(pthread_mutex_t *mutex);

// 加锁。可理解为将 mutex-- (或 -1)，操作后 mutex 的值为 0
int pthread_mutex_lock(pthread_mutex_t *mutex);

// 解锁。可理解为将 mutex++ (或 +1)，操作后 mutex 的值为 1
int pthread_mutex_unlock(pthread_mutex_t *mutex);

// 尝试加锁
int pthread_mutex_trylock(pthread_mutex_t *mutex);

2.2 Locking and unlocking

2.2.1 lock and unlock

• lock attempts to lock. If the lock is unsuccessful, the thread blocks until other threads holding the mutex unlock it.
• unlock actively unlocks the function and wakes up all threads blocked on the lock at the same time. Which thread is awakened first depends on priority and scheduling. Default: block first, wake up first
• For example: T1 T2 T3 T4 uses a mutex lock. T1 is locked successfully, and other threads are blocked until T1 is unlocked. After T1 is unlocked, T2, T3 and T4 are all awakened and automatically try to lock again. It can be assumed that the successful initial value of mutex lock init is 1, the lock function is to change mutex–, and the unlock function is to change mutex++

2.2.2 lock and trylock

• lock If the lock fails, it will block and wait for the lock to be released.
• If trylock fails to lock, it will directly return an error number (such as EBUSY) without blocking.
• Lock the shared resource before accessing it and unlock it immediately after the access is completed. The "granularity" of the lock should be as small as possible

2.3 Locking step test

• Create lock-->Initialize-->Lock-->Access shared data-->Unlock-->Destroy lock

  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>
  #include <unistd.h>
  #include <errno.h>
  #include <pthread.h>
  
  pthread_mutex_t mutex;   // 定义一把全局互斥锁
  
  void *tfn(void *arg) {
        
        
      srand(time(NULL));
  
      while(1) {
        
        
          pthread_mutex_lock(&mutex);    // 加锁，为 0
          printf("hello ");
          sleep(rand() % 3);             // 模拟长时间操作共享资源，导致 cpu 易主，产生与时间有关的错误
          printf("world\n");
          pthread_mutex_unlock(&mutex);  // 解锁，为 1
          sleep(rand() % 3); 
      }   
  
      return NULL;
  }
  
  int main(int argc, char *argv[]) {
        
        
      pthread_t tid;
      srand(time(NULL));
      int ret = pthread_mutex_init(&mutex, NULL);  // 初始化互斥锁，为 1
      if (ret != 0) {
        
        
          fprintf(stderr, "mutex init error: %s\n", strerror(ret));
          exit(1);
      }   
  
      pthread_create(&tid, NULL, tfn, NULL);
      while (1) {
        
        
          pthread_mutex_lock(&mutex);      // 加锁
          printf("HELLO ");
          sleep(rand() % 3); 
          printf("WORLD\n");
          pthread_mutex_unlock(&mutex);    // 解锁
          sleep(rand() % 3); 
      }   
  
      pthread_join(tid, NULL);
  
      pthread_mutex_destroy(&mutex);       // 销毁互斥锁
  
      return 0;
  }
  

  $ gcc pthread_shared.c -o pthread_shared -pthread
  $ ./pthread_shared
  HELLO WORLD
  hello world
  HELLO WORLD
  ...
  

3. Deadlock

• It is a phenomenon caused by improper use of locks.
  • The thread attempts to lock the same mutex A twice ( locking a lock repeatedly )
  • Thread 1 owns lock A and requests lock B; thread 2 owns lock B and requests lock A ( each of the two threads holds one lock and requests the other )

Case

#include <stdio.h>
#include <unistd.h>
#include <pthread.h>

#if 1
int var = 1, num = 5;
pthread_mutex_t m_var, m_num;

void *tfn(void *arg) {
    
    
    int i = (int)arg;

    if (i == 1) {
    
    
        pthread_mutex_lock(&m_var);
        var = 22;
        sleep(1);       //给另外一个线程加锁,创造机会.

        pthread_mutex_lock(&m_num);
        num = 66; 

        pthread_mutex_unlock(&m_var);
        pthread_mutex_unlock(&m_num);

        printf("----thread %d finish\n", i);
        pthread_exit(NULL);

    } else if (i == 2) {
    
    
        pthread_mutex_lock(&m_num);
        var = 33;
        sleep(1);

        pthread_mutex_lock(&m_var);
        num = 99; 

        pthread_mutex_unlock(&m_var);
        pthread_mutex_unlock(&m_num);

        printf("----thread %d finish\n", i);
        pthread_exit(NULL);
    }

    return NULL;
}

int main(void) {
    
    
    pthread_t tid1, tid2;
    int ret1, ret2;

    pthread_mutex_init(&m_var, NULL);
    pthread_mutex_init(&m_num, NULL);

    pthread_create(&tid1, NULL, tfn, (void *)1);
    pthread_create(&tid2, NULL, tfn, (void *)2);

    sleep(3);
    printf("var = %d, num = %d\n", var, num);

    ret1 = pthread_mutex_destroy(&m_var);      //释放琐
    ret2 = pthread_mutex_destroy(&m_num);
    if (ret1 == 0 && ret2 == 0) 
        printf("------------destroy mutex finish\n");

    pthread_join(tid1, NULL);
    pthread_join(tid2, NULL);
    printf("------------join thread finish\n");

    return 0;
}
#else 

int a = 100;

int main(void) {
    
    
    pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;

    pthread_mutex_lock(&mutex);
    a = 777;
    pthread_mutex_lock(&mutex);

    pthread_mutex_unlock(&mutex);

    printf("-----------a = %d\n", a);
    pthread_mutex_destroy(&mutex);

    return 0;
}

#endif

$ gcc deadlock.c -o deadlock -pthread
$ ./deadlock 
var = 33, num = 5
...

4. Read-write lock

• Similar to mutex, but read-write lock allows higher parallelism. Its characteristics are: write exclusive, read sharing, write lock priority is high

4.1 Read-write lock status

• There is only one read-write lock, but it has two states
  • Locked status in read mode (read lock)
  • Locked status in write mode (write lock)

4.2 Read-write lock characteristics

• When the read-write lock is "write mode lock" , all threads locking the lock will be blocked before unlocking.
• When the read-write lock is "locked in read mode" , if the thread locks it in read mode, it will succeed. If the thread locks it in write mode, it will block.
• When the read-write lock is "locked in read mode", there are both threads trying to lock in write mode and threads trying to lock in read mode.
  • Read-write locks block subsequent read-mode lock requests
  • Prioritize writing mode lock
  • Read locks and write locks block in parallel, and write locks have higher priority.

Read-write locks are very suitable for situations where the number of reads to the data structure is much greater than the number of writes.

4.3 Main application functions

#include <pthread.h>

// 返回值 成功返回 0，失败直接返回错误号

// 初始化一把读写锁
// 参 2: attr 表读写锁属性，通常使用默认属性，传 NULL 即可
int pthread_rwlock_init(pthread_rwlock_t *restrict rwlock, const pthread_rwlockattr_t *restrict attr);

// 销毁一把读写锁
int pthread_rwlock_destroy(pthread_rwlock_t *rwlock);

// 以读方式请求读写锁 (简称：请求读锁)
int pthread_rwlock_rdlock(pthread_rwlock_t *rwlock);
// 非阻塞以读方式请求读写锁 (非阻塞请求读锁)
int pthread_rwlock_tryrdlock(pthread_rwlock_t *rwlock);

// 以写方式请求读写锁 (简称：请求写锁)
int pthread_rwlock_wrlock(pthread_rwlock_t *rwlock);
// 非阻塞以写方式请求读写锁 (非阻塞请求写锁)
int pthread_rwlock_trywrlock(pthread_rwlock_t *rwlock);

// 解锁
int pthread_rwlock_unlock(pthread_rwlock_t *rwlock);

4.4 Read-write lock example

• 3 threads "write" global resources from time to time, and 5 threads "read" the same global resource from time to time.

#include <stdio.h>
#include <unistd.h>
#include <pthread.h>

int counter;                          // 全局资源
pthread_rwlock_t rwlock;

void *th_write(void *arg) {
    
    
    int t;
    int i = (int)arg;

    while (1) {
    
    
        t = counter;                  // 保存写之前的值
        usleep(1000);

        pthread_rwlock_wrlock(&rwlock);
        printf("=======write %d: %lu: counter=%d ++counter=%d\n", i, pthread_self(), t, ++counter);
        pthread_rwlock_unlock(&rwlock);

        usleep(9000);                 // 给 r 锁提供机会
    }
    return NULL;
}

void *th_read(void *arg) {
    
    
    int i = (int)arg;

    while (1) {
    
    
        pthread_rwlock_rdlock(&rwlock);
        printf("----------------------------read %d: %lu: %d\n", i, pthread_self(), counter);
        pthread_rwlock_unlock(&rwlock);

        usleep(2000);      // 给写锁提供机会
    }
    return NULL;
}

int main(void) {
    
    
    int i;
    pthread_t tid[8];

    pthread_rwlock_init(&rwlock, NULL);

    for (i = 0; i < 3; i++)
        pthread_create(&tid[i], NULL, th_write, (void *)i);

    for (i = 0; i < 5; i++)
        pthread_create(&tid[i+3], NULL, th_read, (void *)i);

    for (i = 0; i < 8; i++)
        pthread_join(tid[i], NULL);

    pthread_rwlock_destroy(&rwlock);   // 释放读写琐

    return 0;
}

$ gcc rwlock.c -o rwlock -pthread
$ ./rwlock 
----------------------------read 0: 140472231028480: 0
----------------------------read 3: 140472205850368: 0
----------------------------read 2: 140472214243072: 0
----------------------------read 4: 140472197457664: 0
----------------------------read 1: 140472222635776: 0
=======write 0: 140472256206592: counter=0 ++counter=1
=======write 1: 140472247813888: counter=0 ++counter=2
=======write 2: 140472239421184: counter=0 ++counter=3
----------------------------read 2: 140472214243072: 3
----------------------------read 3: 140472205850368: 3
...

5. Condition variables

• The condition variable itself is not a lock, but it can also cause thread blocking
• Usually used in conjunction with mutex locks to provide a place for multiple threads to meet

5.1 Main application functions

#include <pthread.h>

// 返回值 成功返回 0，失败直接返回错误号

// 初始化一个条件变量
// 参 2: attr 表条件变量属性，通常为默认值，传 NULL 即可
int pthread_cond_init(pthread_cond_t *restrict cond, const pthread_condattr_t *restrict attr);

// 销毁一个条件变量
int pthread_cond_destroy(pthread_cond_t *cond);

// 函数作用（1、2 两步为一个原子操作）
    // 1、阻塞等待条件变量 cond (参 1) 满足
    // 2、释放已掌握的互斥锁 (解锁互斥量)，相当于 pthread_mutex_unlock(&mutex);
    // 3、当被唤醒，pthread_cond_wait 函数返回时，解除阻塞并重新申请获取互斥锁 pthread_mutex_lock(&mutex);
int pthread_cond_wait(pthread_cond_t *restrict cond, pthread_mutex_t *restrict mutex);

// 限时等待一个条件变量
int pthread_cond_timedwait(pthread_cond_t *restrict cond, pthread_mutex_t *restrict mutex, const struct timespec *restrict abstime);

// 唤醒至少一个阻塞在条件变量上的线程
int pthread_cond_signal(pthread_cond_t *cond);

// 唤醒全部阻塞在条件变量上的线程
int pthread_cond_broadcast(pthread_cond_t *cond);

5.2 Producer-consumer condition variable model

• A typical case of thread synchronization is the producer-consumer model , and it is a common method to implement this model with the help of condition variables.
  • Suppose there are two threads, one simulating producer behavior and one simulating consumer behavior. Two threads operate a shared resource (generally called aggregation) at the same time. The producer adds products to it and the consumer consumes the products from it.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <errno.h>
#include <pthread.h>

void err_thread(int ret, char *str) {
    
    
    if (ret != 0) {
    
    
        fprintf(stderr, "%s:%s\n", str, strerror(ret));
        pthread_exit(NULL);
    }
}

struct msg {
    
    
    int num;
    struct msg *next;
};

struct msg *head;

pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;       // 定义/初始化一个互斥量
pthread_cond_t has_data = PTHREAD_COND_INITIALIZER;      // 定义/初始化一个条件变量

void *produser(void *arg) {
    
    
    while (1) {
    
    
        struct msg *mp = malloc(sizeof(struct msg));

        mp->num = rand() % 1000 + 1;                        // 模拟生产一个数据`
        printf("--produce %d\n", mp->num);

        pthread_mutex_lock(&mutex);                         // 加锁 互斥量
        mp->next = head;                                    // 写公共区域
        head = mp;
        pthread_mutex_unlock(&mutex);                       // 解锁 互斥量

        pthread_cond_signal(&has_data);                     // 唤醒阻塞在条件变量 has_data上的线程.

        sleep(rand() % 3);
    }

    return NULL;
}

void *consumer(void *arg) {
    
    
    while (1) {
    
    
        struct msg *mp;

        pthread_mutex_lock(&mutex);                         // 加锁 互斥量
        while (head == NULL) {
    
    
            pthread_cond_wait(&has_data, &mutex);           // 阻塞等待条件变量, 解锁
        }                                                   // pthread_cond_wait 返回时, 重新加锁 mutex

        mp = head;
        head = mp->next;

        pthread_mutex_unlock(&mutex);                       // 解锁 互斥量
        printf("---------consumer id: %lu :%d\n", pthread_self(), mp->num);

        free(mp);
        sleep(rand()%3);
    }

    return NULL;
}

int main(int argc, char *argv[]) {
    
    
    int ret;
    pthread_t pid, cid;

    srand(time(NULL));

    ret = pthread_create(&pid, NULL, produser, NULL);           // 生产者
    if (ret != 0) 
        err_thread(ret, "pthread_create produser error");

    ret = pthread_create(&cid, NULL, consumer, NULL);           // 消费者
    if (ret != 0) 
        err_thread(ret, "pthread_create consuer error");

    ret = pthread_create(&cid, NULL, consumer, NULL);           // 消费者
    if (ret != 0) 
        err_thread(ret, "pthread_create consuer error");

    ret = pthread_create(&cid, NULL, consumer, NULL);           // 消费者
    if (ret != 0) 
        err_thread(ret, "pthread_create consuer error");

    pthread_join(pid, NULL);
    pthread_join(cid, NULL);

    return 0;
}

$ gcc cond.c -o cond -pthread
$ ./cond 
--produce 208
---------consumer id: 140611653785344 :208
--produce 829
---------consumer id: 140611645392640 :829
--produce 191
--produce 625
---------consumer id: 140611662178048 :625
---------consumer id: 140611653785344 :191
--produce 926
---------consumer id: 140611645392640 :926
...

5.3 Advantages of condition variables

• Compared with mutex, condition variables can reduce competition
  • If mutex is used directly, in addition to the competition between producers and consumers for the mutex, consumers also need to compete for the mutex. However, if there is no data in the aggregation (linked list), the competition between consumers for the mutex is meaningless
  • With the condition variable mechanism, only when producers complete production will competition among consumers occur, which improves program efficiency.

6. Signal amount

• The semaphore is equivalent to a mutex with an initial value of N

  • Since the granularity of the mutex lock is relatively large, if you want to share part of the data of an object among multiple threads, there is no way to use the mutex lock. You can only lock the entire data object. Although this achieves the purpose of ensuring data correctness when multi-threaded operations share data, it virtually leads to a decrease in thread concurrency. Threads change from parallel execution to serial execution, which is no different from using a single process directly.

• Semaphore is a relatively compromise processing method, which can not only ensure synchronization and avoid data confusion, but also improve thread concurrency.

6.1 Main application functions

#include <semaphore.h>

// 返回值 成功返回 0，失败返回-1，同时设置 errno
// 规定信号量 sem 不能 < 0

// 初始化一个信号量
    // 参 1: sem 信号量
    // 参 2: pshared 取 0 用于线程间; 取非 (一般为 1) 用于进程间
    // 参 3: value 指定信号量初值
int sem_init(sem_t *sem, int pshared, unsigned int value);

// 销毁一个信号量
int sem_destroy(sem_t *sem);

// 给信号量加锁 --
int sem_wait(sem_t *sem);
// 尝试对信号量加锁 -- (与 sem_wait 的区别类比 lock 和 trylock)
int sem_trywait(sem_t *sem);
// 限时尝试对信号量加锁 --
int sem_timedwait(sem_t *sem, const struct timespec *abs_timeout);

// 给信号量解锁 ++
int sem_post(sem_t *sem);

• Basic semaphore operations

6.2 Producer-consumer semaphore model

/*信号量实现 生产者 消费者问题*/
#include <stdlib.h>
#include <unistd.h>
#include <pthread.h>
#include <stdio.h>
#include <semaphore.h>

#define NUM 5               

int queue[NUM];                                     // 全局数组实现环形队列
sem_t blank_number, product_number;                 // 空格子信号量, 产品信号量

void *producer(void *arg) {
    
    
    int i = 0;

    while (1) {
    
    
        sem_wait(&blank_number);                    // 生产者将空格子数--,为0则阻塞等待
        queue[i] = rand() % 1000 + 1;               // 生产一个产品
        printf("----Produce---%d\n", queue[i]);        
        sem_post(&product_number);                  // 将产品数++

        i = (i+1) % NUM;                            // 借助下标实现环形
        sleep(rand()%1);
    }
}

void *consumer(void *arg) {
    
    
    int i = 0;

    while (1) {
    
    
        sem_wait(&product_number);                  // 消费者将产品数--,为0则阻塞等待
        printf("-Consume---%d\n", queue[i]);
        queue[i] = 0;                               // 消费一个产品 
        sem_post(&blank_number);                    // 消费掉以后,将空格子数++

        i = (i+1) % NUM;
        sleep(rand()%3);
    }
}

int main(int argc, char *argv[]) {
    
    
    pthread_t pid, cid;

    sem_init(&blank_number, 0, NUM);                // 初始化空格子信号量为5, 线程间共享 -- 0
    sem_init(&product_number, 0, 0);                // 产品数为 0

    pthread_create(&pid, NULL, producer, NULL);
    pthread_create(&cid, NULL, consumer, NULL);

    pthread_join(pid, NULL);
    pthread_join(cid, NULL);

    sem_destroy(&blank_number);
    sem_destroy(&product_number);

    return 0;
}

$ gcc sem.c -o sem -pthread
$ ./sem
----Produce---384
-Consume---384
----Produce---916
-Consume---916
...