Go recently used the data from Excel to guide internal server is http request
but found a problem after reading the file to open Goroutine can add unlimited
leads all card initialization request Ever since the card is dead
Simulation problem
- Simulation code
func main() {
pool := sync.WaitGroup{}
for i := 0; i < 500; i++ {
pool.Add(1)
go func(i int) {
resp, err := http.Get("http://ip.3322.org")
if err != nil {
fmt.Println(i, err)
} else {
defer resp.Body.Close()
result, _ := ioutil.ReadAll(resp.Body)
fmt.Println(i, string(result))
}
pool.Done()
}(i)
}
pool.Wait()
}
- No problem under the circumstances but a small number of relatively large number of cases you will find an error after the program directly stuck for some time and did not issue any request
problem solved
- In fact look out is to be launched at the same time too much for HTTP requests cause the system to send card data is not dead
- I think the number I would consider submitting a request can not limit Goroutine in Java with the Thread
- Use powerful Baidu really found a big brother already written coroutine pool
- I added the following code comments
package gopool
import (
"sync"
)
// Pool Goroutine Pool
type Pool struct {
queue chan int
wg *sync.WaitGroup
}
// New 新建一个协程池
func New(size int) *Pool {
if size <= 0 {
size = 1
}
return &Pool{
queue: make(chan int, size),
wg: &sync.WaitGroup{},
}
}
// Add 新增一个执行
func (p *Pool) Add(delta int) {
// delta为正数就添加
for i := 0; i < delta; i++ {
p.queue <- 1
}
// delta为负数就减少
for i := 0; i > delta; i-- {
<-p.queue
}
p.wg.Add(delta)
}
// Done 执行完成减一
func (p *Pool) Done() {
<-p.queue
p.wg.Done()
}
// Wait 等待Goroutine执行完毕
func (p *Pool) Wait() {
p.wg.Wait()
}
- Just then modify the test method
package main
import (
"io/ioutil"
"log"
"net/http"
"yumc.pw/cloud/lib/gopool"
)
func main() {
// 这里限制5个并发
pool := gopool.New(5)// sync.WaitGroup{}
for i := 0; i < 500; i++ {
pool.Add(1)
go func(i int) {
resp, err := http.Get("http://ip.3322.org")
if err != nil {
fmt.Println(i, err)
} else {
defer resp.Body.Close()
result, _ := ioutil.ReadAll(resp.Body)
fmt.Println(i, string(result))
}
pool.Done()
}(i)
}
pool.Wait()
}
- Perfect solution