Go language study notes - Chapter 8 Goroutines and Channels (The Go Programming Language)

Chapter 8 Goroutines and Channels

• Concurrent programs in Go language can be implemented in two ways
  • Goroutines and channels, which support "communicating sequential processes" or CSP for short.
  • Multiple threads share memory.

8.1Goroutines

focus

• In the Go language, each concurrent execution unit is called a goroutine.
• In the main function goroutine, called main goroutine.
• To create a goroutine, use an ordinary function or method call with a keywordgo
• When the main function returns, all goroutines will be interrupted directly, and the program exits
• In addition to the main function exiting or directly terminating the program, the goroutine can also be terminated through goroutine communication

Common libraries and methods

• time.Millisecond time.Duration time.Sleep ``

8.2 Example: Concurrent Clock Service

focus

• none

Common libraries and methods

• net.Listen net.Listen.Accept net.Conn net.Dial
• time.Now().Format time.Sleep time.RFC1123 time.Parse

8.3 Example: Concurrent Echo Service

focus

• While using the go keyword, it is necessary to carefully consider whether the method passed between goroutines is safe when called concurrently. In fact, it is indeed unsafe for most types.

Common libraries and methods

• strings.ToUpper strings.ToLower
• bufio.NewScanner input.Scan input.Text

8.4Channels

focus

• channelsCommunication mechanism between Go language goroutines
• A channel is a communication mechanism through which one goroutine sends a value message to another goroutine.
• Each channel has a special type, and a channel that can send int type data is generally written as chan int.
• Using the built-in makefunction, we can create a channel:

ch := make(chan int) // ch has type 'chan int'

• Similar to map, channelit is also a makereference to the underlying data structure that is created, as is the zero value of channel nil.
• When we copy a channel or use for function parameter passing, we just copy a channel reference, so the caller and callee will refer to the same channel object
• Two channels of the same type can be compared using the == operator. If the two channels refer to the same object, then the result of the comparison is true. A channel can also be compared to nil.
• A channel has two main operations, send and receive, both of which use <-the operator .
• send: ch <- xreceive: x = <- ch; a receive operation that does not use the receive result is also legal

ch <- x // a send statement
x = <-ch // a receive expression in an assignment statement
<-ch // a receive statement; result is discarded

• Channel also supports closeoperations close(ch), which are used to close the channel, and any subsequent send operations based on the channel will cause a panic exception. For the channel that has been closed, it can also receive the data that has been successfully sent before, and if there is no data, a zero-value data will be generated.
• makechannelThe capacity represented by the second integer parameter created channel, if the capacity is greater than zero means a buffered channel.

ch = make(chan int) // unbuffered channel
ch = make(chan int, 0) // unbuffered channel
ch = make(chan int, 3) // buffered channel with capacity 3

8.4.1 Channels without caching

focus

• An unbuffered channel will cause the sender goroutine to block until it is received; conversely, if the receiver occurs first, the receiver's goroutine will be blocked until another goroutine sends content;
• Sending and receiving operations based on unbuffered Channels will cause two goroutines to do a synchronous operation. Unbuffered Channels are sometimes calledSync Channels。
• When sending data over an unbuffered Channel, the receiver receives the data before waking up the sender goroutine, called,happens before
• The go statement invokes a function literal, which is the usual form of starting a goroutine in the Go language.
• There are two important aspects to sending messages over channels:
  • Care about the value of the message itself;
  • Caring about the moment when the message is sent, when the moment when the communication occurs is emphasized, it is called a message event;
• For message events that do not carry additional information, you can use struct{}an empty structure as the type of the channels element, of course, you can also use bool or int types to achieve the same function

Common libraries and methods

• net.ResolveTCPAddr net.DialTCP

8.4.2 Channels in series (Pipeline)

focus

• Channels connected in series by multiple goroutines are called pipelines;
• Notify the receiver that there is no redundant value, which can be close(channelName)realized by means. When the channel is closed, sending data to the channel will cause panicthe receiver to no longer be blocked and return a zero value. If received cyclically, endless zero values ​​will be received.
• through x, ok := <-channelNamecanDetermine whether the channel is closed, returns successfully true, returns when the channel is closedfalse
• The range loop in Go language can iterate directly on channels, and jump out of the loop when the channel is closed and there is no value to receive
• Regardless of whether a channel is closed or not, it will be reclaimed by the automatic garbage collector of the Go language when it is not referenced
• Attempting to close a channel repeatedly will panic, and trying to close a channel with a nil value will also panic. Closing a channel also triggers a broadcast mechanism

8.4.3 Unidirectional Channel

focus

• One-way channel type, used for send-only or receive-only channels respectively.
• Type chan<- intrepresents a send-only intchannel, and type <-chan intrepresents a receive-only intchannel. The relative position of the arrow <-and the keyword chanindicates the direction of the channel
• It is a compiler error to call close on a receive-only channel
• Any assignment from a bidirectional channel to a unidirectional channel variable will result in an implicit
  conversion, but not vice versa.

8.4.4 Channels with caching

focus

• A buffered Channel internally holds a queue of elements. makeThe maximum capacity of the queue is specified by the second parameter when calling the function to create the channel.
• ch = make(chan string, 3)Indicates the creation of a buffered Channel that can hold three string elements. As shown in the picture:
  
• The send operation to the cache Channel is to insert elements at the end of the internal cache queue, and the receive operation is to delete elements from the head of the queue. That is, a cached channel can be understood as a queue
• If the buffer channel is full, it will block the sending goroutine, if it is empty, it will block the receiving goroutine
• The built-in capfunction can get the capacity of the channel, and the built-in lenfunction can get the number of effective elements of the channel;
• It is a mistake to use a channel with a buffer as a queue in the same goroutine. A simple queue can use slice
• The unbuffered channel causes no one to receive it and is stuck forever, calledgoroutine leak, is a BUG, ​​leaked goroutines are not automatically recycled
• Unbuffered channels have stronger guarantees that each send operation is synchronized with the corresponding receive operation; but with buffered channels, these operations are decoupled.
• Channel caching may also affect program performance

8.5 Concurrent loops

focus

• Pay attention to forthe goroutine in the loop, because one step will cause the variable value in the goroutine inot to be assigned. iWhat needs to be passed to the goroutine explicitly, rather than declared in the closure (loop variable snapshot problem):
  • The following call is correct;

  for _, f := range filenames {
  go func(f string) {
  thumbnail.ImageFile(f) // NOTE: ignoring errors
  ch <- struct{}{}
  }(f) // right
  }
  

  • The following call is wrong;

  for _, f := range filenames {
  go func() {
  thumbnail.ImageFile(f) // NOTE: incorrect!
  // ...
  }()
  }
  

• An example of a goroutine leak, this situation. The solution is to create a buffered channel of appropriate size, or create another goroutine that drains the other channels when the main goroutine returns the first error:

for _, f := range filenames {
  go func(f string) {
     _, err := thumbnail.ImageFile(f)
     errors <- err
     }(f)
}
for range filenames {
      if err := <-errors; err != nil {
      return err // NOTE: incorrect: goroutine leak!
      }
}

• The counter is incremented by one when each goroutine is created, and decremented by one when it exits, and the counter that waits until it is decremented to zero is calledsync.WaitGroup

Common libraries and methods

• image.Image image.Image.Bounds().Size().X image.Image.Bounds().Size().Y image.Rect image.NewRGBA image.Decode
• jpeg.Encode
• os.Open
• filepath.Ext
• strings.TrimSuffix
• sync.WaitGroup sync.WaitGroup.Add sync.WaitGroup.Done sync.WaitGroup.Wait
• runtime.GOMAXPROCS

8.6 Example: Concurrent Web crawler

focus

• Use goroutine to optimize the previous crawler program, 1. Unlimited goroutine; 2. Limit the number of goroutines; 3. Limit the depth of crawlers;
• Handling 1 Million Requests per Minute with Go

8.7 Select-based multiplexing

focus

• Multiplexing (multiplex):

select {
case <-ch1:
// ...
case x := <-ch2:
// ...use x...
case ch3 <- y:
// ...
default:
// ...
}

• select will wait for the case to be executed when there is a case that can be executed. When the condition is met, select will communicate and execute the statement after the case; other communications will not be executed at this time. A select statement without any case is written as select{}, which will wait forever.
• If multiple cases are ready at the same time, select will randomly select one to execute
• time.TickIt is a timer, unless it needs to be used throughout the life cycle of the program, to prevent goroutine leaks, the correct usage is as follows:

ticker := time.NewTicker(1 * time.Second)
<-ticker.C // receive from the ticker's channel
ticker.Stop() // cause the ticker's goroutine to terminate

• select will have a default to set which logic the program needs to execute when other operations cannot be processed immediately. It allows the program to receive the value of the channel without completely blocking.
• "Polling channel" can do non-blocking receive operations:

select {
case <-abort:
fmt.Printf("Launch aborted!\n")
return
default:
// do nothing
}

• The zero value of the channel is nil, and the sending and receiving operations of the nil channel will be blocked forever. In the select statement, the nil channel will never be selected, so nil can be used to activate or disable the case to achieve other input or Logic for timeout and cancellation when outputting events.

Common libraries and methods

• time.Tick time.After time.NewTicker time.NewTimer timer.Reset timer.C
• os.Stdin.Read

8.8 Example: Concurrent dictionary traversal

focus

• The label break break loopcan end both select and for loops at the same time

loop:
	for {
		select {
		case size, ok := <-fileSizes:
			if !ok {
				break loop // fileSizes was closed
			}
			nfiles++
			nbytes += size
		case <-tick:
			printDiskUsage(nfiles, nbytes)
		}
	}

Common libraries and methods

• ioutil.ReadDir
• os.FileInfo os.FileInfo.IsDir os.FileInfo.IsDir os.FileInfo.Size os.Stat
• os.Stderr
• filepath.Join
• flag.Parse flag.Args flag.Bool
• time.Tick
• sync.WaitGroup

8.9 Concurrent exits

focus

• The Go language does not provide a way to terminate another goroutine in one goroutine, because this will cause the shared variables between goroutines to fall into an undefined state.
• Under normal circumstances, it is difficult for us to know how many goroutines are running at a certain moment.
• Implementation of the broadcast mechanism: do not send values ​​to the channel, but broadcast by closing a channel. To notify all goroutines that need abort channel to exit.
• os.Stdin.Read(make([]byte, 1))A typical program connected to a terminal.
• Call one panic, and the runtime will dump the stack of each goroutine, which is useful for debugging whether all goroutines have exited before the main goroutine returns.

Common libraries and methods

• http.Request http.Get http.NewRequest http.DefaultClient.Do

8.10 Example: Chat service

focus

• none