What you are wasting today is tomorrow for those who died yesterday; what you hate now is the future you can not go back.
What you waste today is the tomorrow that those who died yesterday wished; what you hate now is the future you can't go back to.
How to send data from process a to process b over a simple tcp/ip connection?
In many cases, it's undoubtedly better to use a higher-level network protocol, hiding all the technical details under a fancy API. And there are already many to choose from, depending on the need: message queue protocols, grpc, protobuf, flatbuffers, restful web api, websockets, etc.
However, in some cases (especially in small projects), any method you choose may seem downright overkill.
1. connections is an io stream
net.Conn implements the io.Reader, io.Writer, io.Closer interfaces. So we are using the TCP link like using the io stream.
First, let's take a look at the definitions of these three types in the io package of the Golang source code:
type Reader interface {
Read(p []byte) (n int, err error)
}
type Writer interface {
Write(p []byte) (n int, err error)
}
type Closer interface {
Close() error
}Let's take a look at the definition of the net package Conn type in the Golang source code:
type Conn interface {
Read(b []byte) (n int, err error)
Write(b []byte) (n int, err error)
Close() error
LocalAddr() Addr
RemoteAddr() Addr
SetDeadline(t time.Time) error
SetReadDeadline(t time.Time) error
SetWriteDeadline(t time.Time) error
}
So we can send strings over a TCP connection, but how about sending complex types?
2. Go coding complex types
When it comes to sending structured data over the network, it's easy to think of json, but Go itself provides a gob package to operate directly on the io stream data, serialize and deserialize the data, without adding tags like json, and then laboriously json.Unmarshal() converts to binary data.
3. The basic elements of sending string data over TCP:
1. Sender
1. Open a link to a receiving process
2. Write a string
3. Close the link
Golang's net package already provides all the above methods.
ResolveTCPAddr() accepts a string representing a TCP address (localhost, 127.0.0.1:80, [::1]:80 all represent local port 80), returns a net.TCPAddr(), and returns an error if the address cannot be resolved .
DialTCP() accepts a net.Addr() and connects to this address, returning an open net.TCPConn connection object upon success.
If we don't need to be more detailed about dialing settings. We can use net.Dial directly instead.
If the link is successful, the link object can be encapsulated as a bufio.ReadWriter,
type ReadWriter struct {
*Reader
*Writer
}
We can use the ReadString() Writestring() ReadBytes() method to read the data
Note that buffered writes require flush() to be called after writing in order to forward all data to the underlying network connection
2. Sender
1. Start listening on the local port
2. When a request is received, a goroutine is initiated to process the request
3. Read data in this goroutine and optionally send a response.
4. Close the link
4. Handling of complex types
The server gives the object processing method according to the requested data type. Briefly how it works:
Step 1: When listen() receives a new link, a new goroutine is generated to execute the request method HandleMessage() of the corresponding data type. This function reads the command name from the connection and finds the appropriate handler function from the map. and call the function.
Part 2: The selected handler function reads and processes the request data.
A detailed description:
Initiate a request -> server monitoring -> determine the request data type and corresponding processing method -> new goroutine -> specific processing method processing.
Detailed code:
1. Project directory structure
2. Libraries
package lib
import (
"bufio"
"net"
"github.com/pkg/errors"
"fmt"
"sync"
"io"
"strings"
"encoding/gob"
)
// 混合类型的struct
type ComplexData struct{
N int
S string
M map[string]int
P []byte
C *ComplexData
}
const(
Port = ":61000" // 服务端接受的端口
)
/**
net.Conn 实现了io.Reader io.Writer io.Closer接口
Open 返回一个有超时的TCP链接缓冲readwrite
*/
func Open(addr string) (*bufio.ReadWriter, error) {
// Dial the remote process.
// Note that the local port is chosen on the fly. If the local port
// must be a specific one, use DialTCP() instead.
fmt.Println("Dial " + addr)
conn, err := net.Dial("tcp", addr)
if err != nil {
return nil, errors.Wrap(err, "Dialing "+addr+" failed")
}
return bufio.NewReadWriter(bufio.NewReader(conn), bufio.NewWriter(conn)), nil
}
type HandleFunc func(*bufio.ReadWriter)
type EndPoint struct{
listener net.Listener
// handlefunc是一个处理传入命令的函数类型。 它接收打包在一个读写器界面中的开放连接。
handler map[string]HandleFunc
// map不是线程安全的，所以需要读写锁控制
m sync.RWMutex
}
func NewEndPoint() *EndPoint{
return &EndPoint{
handler:map[string]HandleFunc{},
}
}
// 添加数据类型处理方法
func (e *EndPoint)AddHandleFunc(name string , f HandleFunc){
e.m.Lock()
e.handler[name] = f
e.m.Unlock()
}
// 验证请求数据类型，并发送到对应处理函数
func (e *EndPoint)handleMessage(conn net.Conn){
rw := bufio.NewReadWriter(bufio.NewReader(conn),
bufio.NewWriter(conn))
defer conn.Close()
for{
cmd, err := rw.ReadString('\n')
switch {
case err == io.EOF:
fmt.Println("读取完成.")
return
case err != nil:
fmt.Println("读取出错")
return
}
cmd = strings.Trim(cmd, "\n ")
e.m.RLock()
handleCmd , ok := e.handler[cmd]
if !ok{
fmt.Println("未注册的请求数据类型.")
return
}
//具体处理链接数据
handleCmd(rw)
}
}
func (e *EndPoint) Listen()error{
var err error
e.listener, err = net.Listen("tcp", Port)
if err != nil{
return errors.Wrap(err , "TCP服务无法监听在端口"+Port)
}
fmt.Println(" 服务监听成功：",e.listener.Addr().String())
for{
conn, err := e.listener.Accept()
if err != nil{
fmt.Println("心请求监听失败!")
continue
}
// 开始处理新链接数据
go e.handleMessage(conn)
}
}
func HandleStrings(rw *bufio.ReadWriter){
s, err := rw.ReadString('\n')
if err!= nil{
fmt.Println("链接无法读取.")
return
}
s = strings.Trim(s , "\n ")
// ....
_, err = rw.WriteString("处理完成......\n")
if err != nil{
fmt.Println("链接写入响应失败")
return
}
// 写入底层网络链接
err = rw.Flush()
if err != nil{
fmt.Println("Flush写入失败")
return
}
}
func HandleGob(rw *bufio.ReadWriter){
var data ComplexData
dec := gob.NewDecoder(rw)
err := dec.Decode(&data)
if err != nil{
fmt.Println("无法解析的二进制数据.")
return
}
fmt.Println("输出：", data, data.C)
}
3. Service Documentation
server.go
package main
import(
. "tcpNetWorking/lib"
"fmt"
"github.com/pkg/errors"
)
func server()error{
endpoint := NewEndPoint()
endpoint.AddHandleFunc("string", HandleStrings)
endpoint.AddHandleFunc("gob", HandleGob)
// 开始监听
return endpoint.Listen()
}
func main(){
err := server()
if err != nil {
fmt.Println("Error:", errors.WithStack(err))
}
}
client.go
package main
import (
"fmt"
. "tcpNetWorking/lib"
"github.com/pkg/errors"
"encoding/gob"
"strconv"
"log"
)
func client(ip string) error {
cpData := ComplexData{
N: 10,
S: "测试string 数据",
M: map[string]int{"A": 1, "B": 2},
P: []byte("测试[]byte数据"),
C: &ComplexData{
N: 256,
S: "Recursive structs? Piece of cake!",
M: map[string]int{"01": 1, "10": 2, "11": 3},
},
}
rw, err := Open(ip + Port)
if err != nil {
fmt.Println("客户端无法链接改地址：" + ip + Port)
return err
}
n, err := rw.WriteString("string\n")
if err != nil {
return errors.Wrap(err, "Could not send the STRING request ("+strconv.Itoa(n)+" bytes written)")
}
n, err = rw.WriteString("Additional data.\n")
if err != nil {
return errors.Wrap(err, "Could not send additional STRING data ("+strconv.Itoa(n)+" bytes written)")
}
err = rw.Flush()
if err != nil {
return errors.Wrap(err, "Flush failed.")
}
// Read the reply.
response, err := rw.ReadString('\n')
if err != nil {
return errors.Wrap(err, "Client: Failed to read the reply: '"+response+"'")
}
log.Println("STRING request: got a response:", response)
log.Println("Send a struct as GOB:")
log.Printf("Outer complexData struct: \n%#v\n", cpData)
log.Printf("Inner complexData struct: \n%#v\n", cpData.C)
enc := gob.NewEncoder(rw)
n, err = rw.WriteString("gob\n")
if err != nil {
return errors.Wrap(err, "Could not write GOB data ("+strconv.Itoa(n)+" bytes written)")
}
err = enc.Encode(cpData)
if err != nil {
return errors.Wrapf(err, "Encode failed for struct: %#v", cpData)
}
err = rw.Flush()
if err != nil {
return errors.Wrap(err, "Flush failed.")
}
return nil
}
func main(){
err := client("localhost")
if err != nil {
fmt.Println("Error:", errors.WithStack(err))
}
}
The logic is basically the same as the web routing service I wrote before, except that the data processing uses the binary form of the gob package. Take a look by the way
https://my.oschina.net/90design/blog/1604539