How to use semaphore

Introduction

Semaphore is a common synchronization mechanism in concurrent programming. Semaphore is used when you need to control the number of threads that access resources

scenes to be used

Semaphores are needed when you need to control the number of threads that access resources

Let me give you an example to help you understand. Suppose we have a set of pages to be crawled. Limited resources allow us to perform three crawling tasks at the same time. When three crawling tasks are executing at the same time, the next queued task can only be executed after one crawling task is executed. . Of course, this problem can be solved with Channel, but this time we use the semaphore primitive provided by Go to solve this problem. The code is as follows:

package main

import (
    "context"
    "fmt"
    "sync"
    "time"

    "golang.org/x/sync/semaphore"
)

func doSomething(u string) {
    
    // 模拟抓取任务的执行
    fmt.Println(u)
    time.Sleep(2 * time.Second)
}

const (
    Limit  = 3 // 同時并行运行的goroutine上限
    Weight = 1 // 每个goroutine获取信号量资源的权重
)

func main() {
    
    
    urls := []string{
    
    
        "http://www.example.com",
        "http://www.example.net",
        "http://www.example.net/foo",
        "http://www.example.net/bar",
        "http://www.example.net/baz",
    }
    s := semaphore.NewWeighted(Limit)
    var w sync.WaitGroup
    for _, u := range urls {
    
    
        w.Add(1)
        go func(u string) {
    
    
            s.Acquire(context.Background(), Weight)
            doSomething(u)
            s.Release(Weight)
            w.Done()
        }(u)
    }
    w.Wait()
    
    fmt.Println("All Done")
}

Pay attention to the problem

• 
  Both Acquire and TryAcquire methods can be used to acquire resources, the former will block the acquisition of the semaphore. The latter will obtain the semaphore non-blockingly, and return false if it cannot be obtained.
• After Release returns the semaphore, callers in the waiting queue will be awakened in a first-in first-out order. If the existing resources are not enough for the number of resources requested by the caller at the front of the waiting queue, all waiters will continue to wait.
• If a goroutine applies for more resources, it may wait for a relatively long time due to the strategy of waking up waiters after returning.

Reference:
Go concurrent programming-how to use semaphore and its realization principle