golang sync.singleflight solves the hotspot cache penetration problem

In the package goof sync, there is a singleflightpackage, which contains a singleflight.gofile, the code plus comments, a total of 200 lines. The content includes the following pieces:

1. GroupThe structure manages a group of related function call work, it contains a mutex and a map, mapis keythe name of the function, valueis the corresponding callstructure.
2. callThe structure represents a inflightor completed function call, including waiting components WaitGroup, call results valand err, call times dupsand notification channels chans.
3. DoThe method receives a keyand function fn, it will first check mapwhether there is already keya call of this in inflight, if so, wait and return the existing result, if not, create a new one calland execute the function call.
4. DoChanSimilar Dobut returns a channelto receive the result.
5. doCallThe method contains the logic of specific processing calls, which will add to deferand recover panicto distinguish between normal returnand before and after the function call runtime.Goexit.
6. If it happens panic, it will return panicwrapsan error to the waiting one channel, and if it is, goexitit will exit directly. Normal returnwill send results to all notifications channel.
7. ForgetThe method can forget a keycall, and Dothe function will be re-executed next time.

mapThis package implements keydeduplication of the same function calls through mutexes and to avoid double counting of existing calls, and at the same time channelnotifies the caller of the function execution result through the mechanism. In some scenarios that need to ensure a single execution, the methods in this package can be used.

The effect of caching and deduplication can be easily achieved by using singleflightto avoid repeated calculations. Next, let's simulate the cache penetration scenario that may be caused by concurrent requests, and how to use the package to solve this problem singleflight:

package main

import (
   "context"
   "fmt"
   "golang.org/x/sync/singleflight"
   "sync/atomic"
   "time"
   )

type Result string
// 模拟查询数据库
func find(ctx context.Context, query string) (Result, error) {
    
    
   return Result(fmt.Sprintf("result for %q", query)), nil
}

func main() {
    
    
   var g singleflight.Group
   const n = 200
   waited := int32(n)
   done := make(chan struct{
    
    })
   key := "this is key"
   for i := 0; i < n; i++ {
    
    
      go func(j int) {
    
    
         v, _, shared := g.Do(key, func() (interface{
    
    }, error) {
    
    
            ret, err := find(context.Background(), key)
            return ret, err
         })
         if atomic.AddInt32(&waited, -1) == 0 {
    
    
            close(done)
         }
         fmt.Printf("index: %d, val: %v, shared: %v\n", j, v, shared)
      }(i)
   }

   select {
    
    
   case <-done:
   case <-time.After(time.Second):
      fmt.Println("Do hangs")
   }

   time.Sleep(time.Second * 4)
}

In this program, if the query result is reused, sharedit will return true, and the penetration query will returnfalse

There is another problem in the above design, that is, when Do is blocked, all requests will be blocked, and memory problems may occur.

At this point, Doit can be replaced with DoChan, the implementation of the two is exactly the same, the difference is that the result is returned DoChan()by channel. Therefore, you can use selectthe statement to achieve timeout control

ch := g.DoChan(key, func() (interface{
    
    }, error) {
    
    
   ret, err := find(context.Background(), key)
   return ret, err
})
// Create our timeout
timeout := time.After(500 * time.Millisecond)

var ret singleflight.Result
select {
    
    
case <-timeout: // Timeout elapsed
   fmt.Println("Timeout")
   return
case ret = <-ch: // Received result from channel
   fmt.Printf("index: %d, val: %v, shared: %v\n", j, ret.Val, ret.Shared)
}

Actively return when timeout occurs, without blocking.

At this time, another problem is introduced. Each such request is not highly available, and the success rate cannot be guaranteed. At this time, a certain request saturation can be increased to ensure the final success rate of the business. At this time, one request or multiple requests does not make much difference for downstream services. At this time, it is only used to reduce the order of magnitude of the request, so you can   singleflight  use Forget()To improve the concurrency of downstream requests.

ch := g.DoChan(key, func() (interface{
    
    }, error) {
    
    
   go func() {
    
    
      time.Sleep(10 * time.Millisecond)
      fmt.Printf("Deleting key: %v\n", key)
      g.Forget(key)
   }()
   ret, err := find(context.Background(), key)
   return ret, err
})

Of course, this approach still cannot guarantee 100% success. If a single failure cannot be tolerated, a better processing solution needs to be used in a high-concurrency scenario, such as sacrificing part of real-time performance and fully using cache query + asynchronous update.