Reprinted: Analysis of the realization principle of Go concurrency control context
1 Introduction
Golang context is a commonly used concurrency control technology for Golang application development. The biggest difference between it and WaitGroup is that context has stronger control over derived goroutines, which can control multi-level goroutines.
Context translated into Chinese is "context", that is, it can control a group of goroutines in a tree structure, and each goroutine has the same context.
The typical usage scenario is shown in the figure below:
In the above figure, because goroutines derive child goroutines, and child goroutines continue to derive new goroutines, it is not easy to use WaitGroup in this case, because the number of child goroutines is not easy to determine. And using context can be easily achieved.
2. Context implementation principle
Context actually only defines the interface. Any class that implements this interface can be called a context. The official package implements several commonly used contexts, which can be used in different scenarios.
2.1 Interface definition
The src/context/context.go:Context in the source code package defines this interface:
type Context interface {
Deadline() (deadline time.Time, ok bool)
Done() <-chan struct{
}
Err() error
Value(key interface{
}) interface{
}
}
The basic context interface only defines 4 methods, which are briefly explained below:
2.1.1 Deadline()
This method returns a deadline and a bool value identifying whether the deadline has been set. If the deadline is not set, then ok == false, and the deadline is an initial value of time.Time
2.1.2 Done()
This method returns a channel, which needs to be used in a select-case statement, such as "case <-context.Done():".
When the context is closed, Done() returns a closed pipe, and the closed management is still readable, so the goroutine can receive a close request; when the context is not closed, Done() returns nil.
2.1.3 Err()
This method describes why the context is closed. The reason for closing is controlled by the context and does not require user settings. For example, Deadline context, the reason for closing may be due to the deadline, or it may be actively closed in advance, then the reasons for closing will be different:
- Due to deadline closed: "context deadline exceeded";
- Due to active closure: "context canceled".
When the context is closed, Err() returns the reason for closing the context; when the context is not closed yet, Err() returns nil;
2.1.3 Value()
There is a context, which is not used to control goroutines distributed in a tree, but to transmit information between goroutines distributed in a tree.
The Value() method is used for this type of context. This method queries the value in the map based on the key value. Use the following examples to illustrate.
2.2 Empty context
An empty context is defined in the context package, named emptyCtx, which is used as the root node of the context. The empty context simply implements the Context and does not contain any values. It is only used for the parent nodes of other contexts.
The emptyCtx type definition is shown in the following code:
type emptyCtx int
func (*emptyCtx) Deadline() (deadline time.Time, ok bool) {
return
}
func (*emptyCtx) Done() <-chan struct{
} {
return nil
}
func (*emptyCtx) Err() error {
return nil
}
func (*emptyCtx) Value(key interface{
}) interface{
} {
return nil
}
A public emptCtx global variable is defined in the context package, named background, which can be obtained using context.Background(). The implementation code is as follows:
var background = new(emptyCtx)
func Background() Context {
return background
}
The context package provides four methods to create different types of contexts. If there is no parent context when using these four methods, you need to pass in backgroud, that is, backgroud as its parent node:
- WithCancel()
- WithDeadline()
- WithTimeout()
- WithValue()
The context package implements the context interface struct, in addition to emptyCtx, there are three types of cancelCtx, timerCtx, and valueCtx. It is based on these three context instances that the above four types of context are implemented.
The relationship between the context types in the context package is shown in the following figure:
struct cancelCtx, valueCtx, and valueCtx are all inherited from Context. The three structs are introduced below.
2.3 cancelCtx
Src/context/context.go:cancelCtx in the source code package defines this type of context:
type cancelCtx struct {
Context
mu sync.Mutex // protects following fields
done chan struct{
} // created lazily, closed by first cancel call
children map[canceler]struct{
} // set to nil by the first cancel call
err error // set to non-nil by the first cancel call
}
All children derived from this context are recorded in children, and all children in this context will be cancleed when this context is cancle.
cancelCtx has nothing to do with deadline and value, so you only need to implement the Done() and Err() interfaces to expose the interface.
2.3.1 Done() interface implementation
According to the Context definition, the Done() interface only needs to return a channel, and for cancelCtx, it only needs to return the member variable done.
Here is a direct look at the source code, very simple:
func (c *cancelCtx) Done() <-chan struct{
} {
c.mu.Lock()
if c.done == nil {
c.done = make(chan struct{
})
}
d := c.done
c.mu.Unlock()
return d
}
Since cancelCtx does not specify an initialization function, cancelCtx.done may not have been allocated yet, so initialization needs to be considered.
cancelCtx.done will be closed when the context is canceled, so the value of cancelCtx.done generally goes through three stages: nil --> chan struct{} --> closed chan.
2.3.2 Err() interface implementation
According to the Context definition, Err() only needs to return an error to inform the reason why the context is closed. For cancelCtx, only the member variable err needs to be returned.
Or look at the source code directly:
func (c *cancelCtx) Err() error {
c.mu.Lock()
err := c.err
c.mu.Unlock()
return err
}
CancelCtx.err is nil by default. When the context is
canceled , specify an error variable: var Canceled = errors.New("context canceled").
2.3.3 Cancel() interface implementation
The internal method of cancel() is the most critical method to understand cancelCtx. Its function is to close itself and its descendants. The descendants are stored in the map of cancelCtx.children. The key value is the descendant object, and the value value is meaningless. Map is used here. Just to facilitate the query.
The pseudo code for the cancel method is as follows:
func (c *cancelCtx) cancel(removeFromParent bool, err error) {
c.mu.Lock()
c.err = err //设置一个error，说明关闭原因
close(c.done) //将channel关闭，以此通知派生的context
for child := range c.children {
//遍历所有children，逐个调用cancel方法
child.cancel(false, err)
}
c.children = nil
c.mu.Unlock()
if removeFromParent {
//正常情况下，需要将自己从parent删除
removeChild(c.Context, c)
}
}
In fact, the second method for cancel context returned by WithCancel() is exactly this cancel().
2.3.4 WithCancel() method implementation
The WithCancel() method does three things:
- Initialize a cancelCtx instance
- Add the cancelCtx instance to the children of its parent node (if the parent node can also be canceled)
- Return the cancelCtx instance and cancel() method
The implementation source code is as follows:
func WithCancel(parent Context) (ctx Context, cancel CancelFunc) {
c := newCancelCtx(parent)
propagateCancel(parent, &c) //将自身添加到父节点
return &c, func() {
c.cancel(true, Canceled) }
}
It is necessary to briefly explain the process of adding itself to the parent node here:
- If the parent node also supports cancel, that is to say, its parent node must have children members, then add the new context to children;
- If the parent node does not support cancel, continue to query upwards until a node that supports cancel is found, and the new context is added to children;
- If all parent nodes do not support cancel, start a coroutine to wait for the parent node to end, and then end the current context.
2.3.5 Typical use cases
A typical example of using cancel context is as follows:
package main
import (
"fmt"
"time"
"context"
)
func HandelRequest(ctx context.Context) {
go WriteRedis(ctx)
go WriteDatabase(ctx)
for {
select {
case <-ctx.Done():
fmt.Println("HandelRequest Done.")
return
default:
fmt.Println("HandelRequest running")
time.Sleep(2 * time.Second)
}
}
}
func WriteRedis(ctx context.Context) {
for {
select {
case <-ctx.Done():
fmt.Println("WriteRedis Done.")
return
default:
fmt.Println("WriteRedis running")
time.Sleep(2 * time.Second)
}
}
}
func WriteDatabase(ctx context.Context) {
for {
select {
case <-ctx.Done():
fmt.Println("WriteDatabase Done.")
return
default:
fmt.Println("WriteDatabase running")
time.Sleep(2 * time.Second)
}
}
}
func main() {
ctx, cancel := context.WithCancel(context.Background())
go HandelRequest(ctx)
time.Sleep(5 * time.Second)
fmt.Println("It's time to stop all sub goroutines!")
cancel()
//Just for test whether sub goroutines exit or not
time.Sleep(5 * time.Second)
}
In the above code, the coroutine HandelRequest() is used to process a request, and it will create two coroutines: WriteRedis() and WriteDatabase(). The main coroutine creates the context, and transfers the context between the sub-coroutines. The main coroutine can cancel all sub-coroutines at an appropriate time.
The program output is as follows:
HandelRequest running
WriteDatabase running
WriteRedis running
HandelRequest running
WriteDatabase running
WriteRedis running
HandelRequest running
WriteDatabase running
WriteRedis running
It's time to stop all sub goroutines!
WriteDatabase Done.
HandelRequest Done.
WriteRedis Done.
2.4 timerCtx
Src/context/context.go:timerCtx in the source code package defines this type of context:
type timerCtx struct {
cancelCtx
timer *time.Timer // Under cancelCtx.mu.
deadline time.Time
}
timerCtx adds a deadline to cancelCtx to mark the final time of automatic cancel, and timer is a timer that triggers automatic cancel.
As a result, WithDeadline() and WithTimeout() are derived. The implementation principles of these two types are the same, but the context is different:
- deadline: Specify a deadline, for example, the context will automatically end at 2018.10.20 00:00:00
- timeout: Specify the longest survival time, for example, the context will end after 30s.
For the interface, timerCtx also needs to implement Deadline() and cancel() methods on the basis of cancelCtx, and the cancel() method is rewritten.
2.4.1 Deadline() interface implementation
The Deadline() method just returns timerCtx.deadline. And timerCtx.deadline is set by WithDeadline() or WithTimeout() method.
2.4.2 Cancel() interface implementation
The cancel() method basically inherits cancelCtx, and only needs to close the timer additionally.
After timerCtx is closed, timerCtx.cancelCtx.err will store the reason for closing:
If it is manually closed before the deadline, the reason for the shutdown is the same as the cancelCtx display;
if it is automatically closed when the deadline comes, the reason is: "context deadline exceeded"
2.4.3 WithDeadline() method implementation
The implementation steps of WithDeadline() method are as follows:
- Initialize a timerCtx instance
- Add the timerCtx instance to the children of its parent node (if the parent node can also be cancelled)
- Start the timer, the context will be cancelled automatically after the timer expires
- Return the timerCtx instance and cancel() method
In other words, the timerCtx type context not only supports manual cancellation, but also automatically cancels after the timer arrives.
2.4.4 WithTimeout() method implementation
WithTimeout() actually calls WithDeadline, and the two implementation principles are the same.
Look at the code will be very clear:
func WithTimeout(parent Context, timeout time.Duration) (Context, CancelFunc) {
return WithDeadline(parent, time.Now().Add(timeout))
}
2.4.5 Typical use cases
The following example uses WithTimeout() to obtain a context and pass it in the coroutine:
package main
import (
"fmt"
"time"
"context"
)
func HandelRequest(ctx context.Context) {
go WriteRedis(ctx)
go WriteDatabase(ctx)
for {
select {
case <-ctx.Done():
fmt.Println("HandelRequest Done.")
return
default:
fmt.Println("HandelRequest running")
time.Sleep(2 * time.Second)
}
}
}
func WriteRedis(ctx context.Context) {
for {
select {
case <-ctx.Done():
fmt.Println("WriteRedis Done.")
return
default:
fmt.Println("WriteRedis running")
time.Sleep(2 * time.Second)
}
}
}
func WriteDatabase(ctx context.Context) {
for {
select {
case <-ctx.Done():
fmt.Println("WriteDatabase Done.")
return
default:
fmt.Println("WriteDatabase running")
time.Sleep(2 * time.Second)
}
}
}
func main() {
ctx, _ := context.WithTimeout(context.Background(), 5 * time.Second)
go HandelRequest(ctx)
time.Sleep(10 * time.Second)
}
Create a 10s timeout context in the main coroutine and pass it to the sub coroutine, and the context will be closed automatically in 10s. The program output is as follows:
HandelRequest running
WriteRedis running
WriteDatabase running
HandelRequest running
WriteRedis running
WriteDatabase running
HandelRequest running
WriteRedis running
WriteDatabase running
HandelRequest Done.
WriteDatabase Done.
WriteRedis Done.
2.5 valueCtx
Src/context/context.go:valueCtx in the source package defines this type of context:
type valueCtx struct {
Context
key, val interface{
}
}
valueCtx just adds a key-value pair on the basis of Context, which is used to transfer some data between all levels of coroutines.
Since valueCtx does not require cancel or deadline, it only needs to implement the Value() interface.
2.5.1 Value () interface implementation
It can be seen from the definition of valueCtx data structure that valueCtx.key and valueCtx.val represent its key and value values, respectively. The implementation is also very simple:
func (c *valueCtx) Value(key interface{
}) interface{
} {
if c.key == key {
return c.val
}
return c.Context.Value(key)
}
There is a detail to pay attention to here, that is, when the current context cannot find the key, it will look for the parent node, and if it cannot find it, it will eventually return interface{}. In other words, the value of the parent can be queried through the child context.
2.5.2 WithValue() method implementation
The implementation of WithValue() is also very simple, the pseudo code is as follows:
func WithValue(parent Context, key, val interface{
}) Context {
if key == nil {
panic("nil key")
}
return &valueCtx{
parent, key, val}
}
2.5.3 Typical Use Cases
The following sample program shows the usage of valueCtx:
package main
import (
"fmt"
"time"
"context"
)
func HandelRequest(ctx context.Context) {
for {
select {
case <-ctx.Done():
fmt.Println("HandelRequest Done.")
return
default:
fmt.Println("HandelRequest running, parameter: ", ctx.Value("parameter"))
time.Sleep(2 * time.Second)
}
}
}
func main() {
ctx := context.WithValue(context.Background(), "parameter", "1")
go HandelRequest(ctx)
time.Sleep(10 * time.Second)
}
In the above example main(), a context is obtained through the WithValue() method, and a parent context, key and value need to be specified. Then pass the context to the sub-coroutine HandelRequest, and the sub-coroutine can read the key-value of the context.
Note: In this example, the sub-coroutine cannot be automatically terminated because the context does not support cancle, which means that <-ctx.Done() can never return. If you need to return, you need to specify a cancelable context as the parent node when creating the context, and use the cancel() of the parent node to end the entire context at an appropriate time.
to sum up
- Context is just an interface definition, and different context types can be derived according to different implementations;
- cancelCtx implements the Context interface, and creates a cancelCtx instance through WithCancel();
- timerCtx implements the Context interface, and creates timerCtx instances through WithDeadline() and WithTimeout();
- valueCtx implements the Context interface and creates a valueCtx instance through WithValue();
The three context instances can be the parent nodes of each other, which can be combined into different application forms;