360 cloud computing 360 cloud computing
Heroine declaration
With its high concurrency characteristics, Go language has been widely used in cloud computing development, and it is also very popular among developers. But do you really understand the go language? After many years of practical application of the go language, the author of this article now gives a detailed introduction on how to read data from io.Reader in the go language. I believe it will be of great help to go language lovers. Let's follow the author and study together.
PS: rich first-line technology, a wide range of forms, all in " 3 60 cloud computing " point of concern Oh!
1
Overview
During the development process, we often read data from io.Reader.type Reader interface {
Read(p []byte) (n int, err error)
}
- At most len(p) length data can be read at a time.
- When reading encounters error or EOF, it will return the number of bytes of data that have been read and error or EOF.
- The Read method does not modify the size of len(p).
- The use of io.EOF means it is over.
package main
import (
"fmt"
"io"
"net"
)
func main() {
// Establish a tcp connection
conn, err := net.Dial("tcp", "www.findme.wang:80")
if err != nil {
fmt.Println("dial error:", err)
return
}
defer conn.Close() // close the connection
// Construct http protocol content and initiate http request
httpReq := `GET / HTTP/1.0
Host: www.findme.wang
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36
Content-Type:application/x-www-form-urlencoded
Content-Length:0
`
_, err = fmt.Fprintf(conn, httpReq)
if err != nil {
fmt.Println("http request error:", err)
return
}
// read from conn
rsData := make([]byte, 0)
for {
// Read up to 512 bytes at a time
var tmp = make([]byte, 512)
n, err := conn.Read(tmp)
if n >= 0 {
rsData = append(rsData, tmp[:n]...)
}
if err == io.EOF {
fmt.Println("Data read completed")
break
} else if err != nil {
fmt.Println("Error when reading data:", err)
break
}
}
fmt.Println("The length of the data read:", len(rsData))
}
In the case, we use the for loop to read repeatedly. Is there a concise way?2
Use io.copy to read
io.copy is defined as follows:func Copy(dst Writer, src Reader) (written int64, err error) {
return copyBuffer(dst, src, nil)
}
Read the content of the reader into the data in dst, and read it into dst, so we need a writer. Come on, encapsulate one as follows:
package main
import (
"fmt"
"io"
"net"
)
type MyWriter struct {
data []byte
}
func (m *MyWriter) Write(p []byte) (n int, err error) {
if m.data == nil {
m.data = make([]byte, 0)
}
if p != nil && len(p) != 0 {
m.data = append(m.data, p...)
}
return len(p), nil
}
func main() {
// Establish a tcp connection
conn, err := net.Dial("tcp", "www.findme.wang:80")
if err != nil {
fmt.Println("dial error:", err)
return
}
defer conn.Close() // close the connection
// Construct http protocol content and initiate http request
httpReq := `GET / HTTP/1.0
Host: www.findme.wang
User- Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36
Content-Type:application/x-www-form-urlencoded
Content-Length:0
`
_, err = fmt.Fprintf(conn, httpReq)
if err != nil {
fmt.Println("http request error:", err)
return
}
w := new(MyWriter)
n, err := io.Copy(w, conn) // read the data in conn to the writer
if err != nil {
fmt.Println("Read err", err)
}
//fmt.Println(string(w.data))// print data
fmt.Println("The length of the data read:", n)
}
Reading data from io is simple, but a writer needs to be encapsulated. So, is there a similar writer in go? What about writers that can make it easy for us to get data?
So, we found strings.buffer, as follows:// A Builder is used to efficiently build a string using Write methods.
// It minimizes memory copying. The zero value is ready to use.
// Do not copy a non-zero Builder.
type Builder struct {
addr *Builder // of receiver, to detect copies by value
buf []byte
}
With strings.buffer, the code can be streamlined again.
package main
import (
"fmt"
"io"
"net"
"strings"
)
func main() {
// Establish a tcp connection
conn, err := net.Dial("tcp", "www.findme.wang:80")
if err != nil {
fmt.Println("dial error:", err)
return
}
defer conn.Close() // close the connection
// Construct http protocol content and initiate http request
httpReq := `GET / HTTP/1.0
Host: www.findme.wang
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36
Content-Type:application/x-www-form-urlencoded
Content-Length:0
`
_, err = fmt.Fprintf(conn, httpReq)
if err != nil {
fmt.Println("http request error:", err)
return
}
var sb strings.Builder
n, err := io.Copy(&sb, conn) // read the data in conn to the writer
if err != nil {
fmt.Println("Read err", err)
}
fmt.Println(sb.String()) // print res
fmt.Println("The length of the data read:", n)
}
In addition to using strings.buffer, we can also use bytes.Buffer.3
Use ioutil.ReadAll
ReadAll(r io.Reader) ([]byte, error) is to read all data from the input stream (reader) at one time until an error or EOF is sent. If the read fails, the read data and err are returned; if the read succeeds, the full data and nil are returned. That is to change the method, it will not return EOF, the case is as follows:package main
import (
"fmt"
"io/ioutil"
"net"
)
func main() {
// Establish a tcp connection
conn, err := net.Dial("tcp", "www.findme.wang:80")
if err != nil {
fmt.Println("dial error:", err)
return
}
defer conn.Close() // close the connection
// Construct http protocol content and initiate http request
httpReq := `GET / HTTP/1.0
Host: www.findme.wang
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36
Content-Type:application/x-www-form-urlencoded
Content-Length:0
`
_, err = fmt.Fprintf(conn, httpReq)
if err != nil {
fmt.Println("http request error:", err)
return
}
data, err := ioutil.ReadAll(conn)
if err != nil {
fmt.Println("Read err", err)
}
fmt.Println(string(data)) // print res
fmt.Println("The length of the data read:", len(data))
}
4
supplement
In addition, we can also use some methods provided by the io package, such as: io.ReadAtLeast, io.ReadFull, etc. 1. io.ReadAtLeast puts at least min bytes from the input stream into buf, and returns the read bytes Number and err, the structure is as follows:func ReadAtLeast(r Reader, buf []byte, min int) (n int, err error) {
if len(buf) < min {
return 0, ErrShortBuffer
}
for n < min && err == nil {
var nn int
nn, err = r.Read(buf[n:])
n += nn
}
if n >= min {//When reading bytes not less than min, set err to nil
err = nil
} else if n > 0 && err == EOF {
err = ErrUnexpectedEOF
}
return
}
If the length of buf is less than min, ErrShortBuffer will be triggered.
- If the number of bytes read is less than min, this will trigger an ErrUnexpectedEOF error.
- If the number of bytes read is not less than min, even if err is encountered, nil will be returned.
func ReadFull(r Reader, buf []byte) (n int, err error) {
return ReadAtLeast(r, buf, len(buf))
}
io.ReadFull essentially calls io.ReadAtLeast above, so I won't repeat it here.