Accessing web services - Go's support for IPC

Article directory

Accessing web services - Go's support for IPC

One, socket and IPC

Socket (socket) is one of the core contents of network programming.

1.1 IPC (Inter-Process Communication)

IPC (Inter-Process Communication): Inter-process communication. Therefore, the concept of IPC mainly defines the method of communicating with each other between multiple processes.

The main IPC (inter-process communication) methods are: system signal (signal), pipe (pipe), socket (socket), file lock (file lock), message queue (message queue), semaphore (semaphone, also known as as a semaphore), etc.

Existing operating systems provide strong support for IPC (Inter-Process Communication), especially sockets.

1.2 socket (socket)

socket (socket), is an IPC (inter-process communication) method.

Among the many IPC methods, socket is the most versatile and flexible one:

The process of using socket for communication may not be limited to the same computer;
No matter where the two parties in the communication exist in any corner of the world, as long as they can be connected through the network card port of the computer and the network process, they can use the socket;

An operating system that supports sockets will provide a set of APIs to the outside world. Applications running on them will use this set of APIs to communicate with other programs on another computer on the Internet, or even other threads in the same program.

2. Go language supports IPC

The Go language provides certain support for IPC:

There are APIs for system signals in Go's os code package and os/signal representative package;
os.PipeFunctions can create named pipes, and os/execanother type of pipe (anonymous pipe) is supported in the code package;
Go language has a program entity supporting socket in the net code package;

2.1 socket system call

For example, in the Linux operating system, the API used to create a socket instance is represented by a system call named socket. This system call is part of the Linux kernel.

The so-called system call can be understood as a special C language function. They are the bridge between applications and the operating system kernel, and are the only way for applications to use operating system functions.

2.2 Standard library syscall of Go language

In the syscall code package of the Go language standard library, there is a function corresponding to this socket system call. The function signatures of the two are basically the same. They both accept three int type parameters and return a result that can represent a file descriptor.

func Socket(domain, typ, proto int) (fd int, err error) {
    
    
	if domain == AF_INET6 && SocketDisableIPv6 {
    
    
		return -1, EAFNOSUPPORT
	}
	fd, err = socket(domain, typ, proto)
	return
}

syscall.SocketThe function itself is out of the control of the platform.

At its bottom layer, the Go language is adapted for each operating system it supports, which is why this function is always valid no matter which platform it is on.

syscall.SoecktThe function accepts three parameters of int type, which represent the communication domain, type and protocol of the socket instance to be created respectively.

(1) socket communication domain

The socket communication domain mainly has several options: IPv4 domain, IPv6 domain, and Unix domain.

IPv4 domain (AF_INET): corresponding to the network based on the fourth version of IP protocol;

IPv4 is an address format in the Internet Protocol that is used to uniquely identify devices on the network. An IPv4 address is a 32-bit binary number, usually expressed as a dotted decimal number, where each value ranges from 0 to 255. An IPv4 address consists of a network number and a host number, where the network number is used to identify a network, and the host number is used to identify a device on the network. The range of IPv4 addresses is from 0.0.0.0 to 255.255.255.255, some of which are reserved for specific purposes, such as private networks and broadcast addresses. The shortage and bottleneck of IPv4 address space make people gradually turn to IPv6.
IPv6 domain (AF_INET6): corresponds to the network based on the sixth version of the IP protocol;

The IPv6 domain is an address family used to identify IPv6 addresses. Unlike IPv4, IPv6 addresses are represented by 128-bit binary numbers, usually expressed as 8 groups of 16-bit hexadecimal numbers, separated by colons, For example, 2001:0db8:85a3:0000:0000:8a2e:0370:7334. The IPv6 address space is extremely broad, far exceeding IPv4, and can support more devices to connect to the Internet. The address format of the IPv6 protocol is incompatible with IPv4, and a specific protocol stack is required to realize IPv6 communication.
UNIX domain socket (AF_UNIX): refers to a unique communication domain in a Unix-like operating system;

In the same computer where the roll has such an operating system, applications can establish socket connections based on this domain.
IPX/SPX (AF_IPX): For communications on Novell NetWare networks.
AppleTalk (AF_APPLETALK): used for communication between Apple machines.
Netlink (AF_NETLINK): Used for communication between the Linux kernel and user space.
Bluetooth (AF_BLUETOOTH): Used for communication between Bluetooth devices.

(2) The type of socket

There are four types of sockets, namely: SOCK_DGRAM, SOCK_STREAM, SOCK_SEQPACKET, SOCK_RAW.

SOCK_DGRAM (datagram socket): "DGRAM" stands for datagram, which is a datagram. It is an unreliable socket type with message boundaries but no logical connections .

Datagram sockets provide connectionless, unreliable data transfer, where each packet is independent. This type of socket is often used in network programming for the UDP protocol.
SOCK_STREAM (stream socket): Stream socket provides a connection-oriented, reliable, bidirectional byte stream transmission. This type of socket is often used in network programming of the TCP protocol, such as HTTP, FTP and other applications.

It is transmitted in the form of byte stream, without message boundary, but with logical connection, which can ensure the reliability of transmission and the order of data, and can also realize two-way transmission of data.

Such network communication transmits data in the form of byte streams rather than data packets. Byte streams are in bytes. The kernel cannot perceive how many messages are contained in a segment of bytes, and whether the messages are complete, and the application needs to control it by itself. One end of such network communications always receives and buffers data in the byte order in which it was sent by the other end. The application needs to find the message boundary in the data according to the agreement of both parties, and cut the data according to the boundary.
SOCK_SEQPACKET (serialized packet socket): Serialized packet socket provides connection-oriented, reliable, two-way data transmission, each packet has a fixed length. This type of socket is often used for sequential transfer applications, such as sequentially reading a stream of data.
SOCK_RAW (raw socket): The raw socket provides direct access to the underlying protocol, and the protocol header and data can be customized. This type of socket is often used in applications that require in-depth operations on network protocols, such as network diagnosis and attacks.

Message boundary: The program (kernel program) in the operating system kernel related to the socket is in units of messages when sending or receiving data.

A message can be thought of as a piece of data with fixed boundaries. The kernel program can automatically identify and maintain this boundary, and when necessary, cut the data into one message, or string multiple messages into continuous data. Therefore, the application program only needs to process for the message process.

Logical connection: Both communicating parties must establish a network connection before sending and receiving data. After the connection is established, the two parties can perform data transmission one-to-one.

The network communication based on the UDP protocol does not need to establish a logical connection. As long as the application program specifies the network address of the other party, the kernel program will immediately send the data message.

The advantage of not establishing a logical connection is that the sending speed is fast and the network resources are not occupied for a long time.

The disadvantages of not establishing a logical connection are: the sent data message is longer, the reliability of transmission cannot be guaranteed, the order of data cannot be realized, and data can only be transmitted in one direction.

(3) The protocol used by the socket instance

syscall.SocketThe third parameter of the function is used to indicate the protocol used by the socket instance.

Usually, as long as the values of the first two parameters are specified, there is no need to determine the value of the third parameter. Generally, it is sufficient to set it to 0. The kernel program will automatically choose the most suitable protocol.

For example:

When the first two parameters are: andsyscall.AF_INET , the kernel program will choose UDP as the protocol;syscall.SOCK_DGRAM
The first two parameters are: syscall.AF_INET6and syscall.SOCK_STREAM, the kernel program will choose TCP as the protocol;

2.3 net code package of Go language

Many program entities in the net code package of the Go0 language use syscall.Sorkcetfunctions directly or indirectly.

(1) `net.Dial`function

func Dial(network, address string) (Conn, error) {
    
    
	var d Dialer
	return d.Dial(network, address)
}

The first parameter network: (string type) determines what kind of socket instance the Go program will create at the bottom layer, and what protocol will be used to communicate with other programs.

Nine optional values commonly used by network:

tcp: Represents the TCP protocol, and the version of the IP protocol based on it is adaptive according to the value of the parameter address;
tcp4: represents the TCP protocol based on the fourth version of the IP protocol;
tcp6: represents the TCP protocol based on the sixth version of the IP protocol;
udp: Represents the UDP protocol, and the version of the IP protocol it is based on is adaptive according to the value of the parameter address;
udp4: represents the UDP protocol based on the fourth version of the IP protocol;
udp6: represents the UDP protocol based on the sixth version of the IP protocol;
unix: Represents an internal socket protocol under the Unix communication domain, with SOCK_STREAM as the socket type.
unixgram: represents an internal socket protocol under the Unix communication domain, with SOCK_DGRAM as the socket type;

"unixgram" means a Unix-domain datagram socket, used to transfer discrete packets of data between processes (similar to the UDP protocol).
unixpacket: represents an internal socket protocol under the Unix communication domain, with SOCK_SEQPACKET as the socket type;

"unixpacket" means a Unix domain packet socket, which is similar to a Unix domain stream socket, but provides atomic packet transmission, that is, it is guaranteed that the receiver will not see any data pack.

There are 3 options that are not commonly used in the network:

ip: IP protocol, which can be used for any network type.
ip4: IPv4-only IP protocol, which can be used for any network type.
ip6: IPv6-only IP protocol, which can be used for any network type.

It should be noted that the three optional values "unix", unixgram" and "unixpacket" can only be used on Unix systems. If these two values are used on a non-Unix system, the Dial function will return an error.

(2) `net.DialTimeout`function

net.DialTimeoutThe timeout given by the function represents the maximum time the function waits for the network connection to be established.

This is a relative time, represented by the value of the parameter timeout of this function.

The start time is almost net.DialTimeoutthe moment we call the function. After that, time will be spent on "parsing the values of the parameters network and address" and "creating a socket instance and establishing a network connection".

No matter which step is executed, as long as the network connection has not been established at the moment when the absolute timeout time is reached, the function will return an error value representing the timeout of the I/O operation.

net.DialTimeoutThe timeout function is net.Dialerrealized by using the structure.