Article directory
1. ARP protocol
1.1 Definition of ARP protocol
ARP protocol, also known as Address Resolution Protocol, is a protocol that converts IP addresses to MAC addresses in a local area network. ARP is a protocol that works between the data link layer and the network layer of the OSI model.
1.2 Working process of ARP protocol
The working process of ARP mainly includes four steps: ARP request, ARP reply, ARP cache, and ARP refresh.
- ARP request: A host broadcasts an ARP request to the LAN. The request contains the IP address and MAC address of the sending host, as well as the IP address of the target host, but does not include the MAC address of the target host.
- ARP reply: All hosts that receive an ARP request will check whether their own IP address is the same as the target IP address in the ARP request. If they are the same, the host will send an ARP reply containing its own IP address and MAC address.
- ARP cache: The host that sends the ARP request will save the IP address and MAC address mapping information in the received ARP reply. This saving process is called ARP cache. In this way, the next time you need to send data, you can directly look up the target MAC address from the ARP cache without issuing an ARP request again.
- ARP refresh: Since network conditions may change, the validity period of the ARP cache is usually not very long. After a period of time, the mapping relationship in the ARP cache needs to be refreshed.
1.3 Data structure of ARP protocol
The ARP message mainly consists of hardware type, protocol type, hardware address length, protocol address length, operation field, sender hardware address, sender protocol address, target hardware address, target protocol address, etc. Especially the operation field, which is used to identify the type of ARP message. If the value is 1, it means that this is an ARP request; if the value is 2, it means that this is an ARP reply.
1.4 ARP spoofing
ARP spoofing is a means of network attack. The attacker sends forged ARP responses, causing incorrect IP address and MAC address mapping information to be stored in the ARP cache of other hosts, thereby hijacking network data or tamper.
1.5 Limitations of ARP protocol
Although the ARP protocol is simple and easy to use, it also has some limitations. For example, the ARP protocol can only be used on IPv4 networks. For IPv6 networks, the ARP protocol has been replaced by NDP (Neighbor Discovery Protocol). In addition, since the ARP protocol has low security and is vulnerable to ARP spoofing attacks, other methods may be needed to obtain the MAC address in some networks that require high security.
1.6 The relationship between ARP protocol and Posix API
In the Posix API, there is no interface to directly operate the ARP protocol. Usually, the operation of the ARP protocol is automatically completed by the network stack of the operating system. However, by using raw socket, we can directly manipulate the data of the data link layer, so that we can send and receive ARP messages. At the same time, we can also obtain and set the contents of the ARP cache through the ioctl function.
2. ARP protocol example
Here is an example of a simple ARP request and response:
#include <stdio.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <string.h>
#include <arpa/inet.h>
#include <netinet/if_ether.h>
#include <net/if.h>
#include <netpacket/packet.h>
#define ARP_REQUEST 1
#define ARP_REPLY 2
typedef struct arp_header {
uint16_t hardware_type;
uint16_t protocol_type;
uint8_t hardware_len;
uint8_t protocol_len;
uint16_t opcode;
uint8_t sender_mac[6];
uint8_t sender_ip[4];
uint8_t target_mac[6];
uint8_t target_ip[4];
} arp_header_t;
void fill_arp_request(arp_header_t *arp) {
arp->hardware_type = htons(1);
arp->protocol_type = htons(ETH_P_IP);
arp->hardware_len = 6;
arp->protocol_len = 4;
arp->opcode = htons(ARP_REQUEST);
//填写你的MAC地址和IP地址
memcpy(arp->sender_mac, "\xaa\xbb\xcc\xdd\xee\xff", 6);
memcpy(arp->sender_ip, "\xc0\xa8\x01\x0a", 4); // 192.168.1.10
//填写目标MAC地址和IP地址
memcpy(arp->target_mac, "\x00\x00\x00\x00\x00\x00", 6);
memcpy(arp->target_ip, "\xc0\xa8\x01\x14", 4); // 192.168.1.20
}
int main() {
int sd;
char buffer[60];
struct sockaddr_ll socket_address;
//建立原始套接字
sd = socket(AF_PACKET, SOCK_RAW, htons(ETH_P_ARP));
if (sd == -1) {
perror("socket creation failed");
return -1;
}
//清空缓冲区和套接字地址
memset(buffer, 0x00, 60);
memset(&socket_address, 0x00, sizeof(socket_address));
//填充Ethernet帧头部
struct ethhdr *eth = (struct ethhdr *)buffer;
eth->h_proto = htons(ETH_P_ARP);
//填写你的MAC地址
memcpy(eth->h_source, "\xaa\xbb\xcc\xdd\xee\xff", 6);
//填写目标MAC地址
memcpy(eth->h_dest, "\xff\xff\xff\xff\xff\xff", 6);
//填充ARP请求
arp_header_t *arp = (arp_header_t *)(buffer + 14);
fill_arp_request(arp);
//设置目标地址
socket_address.sll_family = AF_PACKET;
socket_address.sll_protocol = htons(ETH_P_ARP);
socket_address.sll_ifindex = if_nametoindex("eth0");
//发送ARP请求
if (sendto(sd, buffer, 42, 0, (struct sockaddr*)&socket_address, sizeof(socket_address)) == -1) {
perror("send failed");
return -1;
}
return 0;
}
The above program first creates a raw socket, then fills it with an Ethernet frame and ARP request, and finally sends it to the network. This is a very simple ARP request to query the MAC address corresponding to a given IP address (192.168.1.20).
In the ARP header, hardware_typethe field refers to the hardware type, protocol_typethe field refers to the protocol type, hardware_lenand the field and protocol_lenfield represent the length of the hardware address and protocol address respectively. opcodeField indicates the type of ARP message, which can be ARP request or ARP reply.
3. ICMP protocol
3.1 Definition of ICMP protocol
Internet Control Message Protocol (ICMP) is part of the TCP/IP protocol suite. It is an important protocol at the network layer and is mainly used to send control messages between network devices. ICMP is often used to handle error conditions in network communications and perform related diagnosis.
3.2 Types of ICMP protocols
There are many types of ICMP messages, including but not limited to: destination unreachable (Type 3), source suppression (Type 4), redirection (Type 5), echo request and echo reply (Type 8 and Type 0), time Exceed (Type 11), parameter problem (Type 12), timestamp request and timestamp response (Type 13 and Type 14), etc.
3.3 How the ICMP protocol works
Take "echo request" and "echo response" (which is what we often call the ping command) as an example:
- The sender sends an "Echo Request" ICMP message with a specific identifier and sequence number to the target IP address.
- If the host at the destination IP address is reachable, it sends back an "echo reply" ICMP message with the same identifier and sequence number.
- After receiving the "echo response" message, the sender will calculate the time difference between sending the request and receiving the response. This time is what we often call network delay.
3.4 Application of ICMP protocol
The ICMP protocol is mainly used for network diagnosis. Common applications are as follows:
- Ping command: used to test whether the target host is reachable and the network delay.
- traceroute command: used to trace the routing path of data packets from the source host to the destination host.
- Routers and gateways: ICMP can be used to send error messages, such as "destination unreachable", "redirect", etc.
3.5 Limitations and security of the ICMP protocol
The ICMP protocol provides a way to handle and report network problems, but it also has some limitations and security issues.
- Limitations: ICMP cannot provide detailed information about network problems. For example, it cannot diagnose which application or service is causing the network problem.
- Security Issues: Because ICMP can be used to discover the structure and activity of a network, it may be used for network reconnaissance and denial of service (DoS) attacks.
3.6 Relationship between ICMP protocol and Posix API
In Posix API, we can send and receive ICMP messages by using raw socket and setsockopt functions. For example, we can create a raw socket, then use the setsockopt function to set the ICMP type and code, and finally use the sendto function to send ICMP messages. When receiving an ICMP message, we can use the recvfrom function to obtain the content and source address of the ICMP message.
4. ICMP protocol example
The following is an example of a simple ICMP echo request (ie ping). This program will send an ICMP echo request to a specified IP address and receive and print the ICMP echo response from that IP address:
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <netinet/ip.h>
#include <netinet/ip_icmp.h>
#include <arpa/inet.h>
#define PACKET_SIZE 4096
unsigned short checksum(void *b, int len) {
unsigned short *buf = b;
unsigned int sum = 0;
unsigned short result;
for (sum = 0; len > 1; len -= 2)
sum += *buf++;
if (len == 1)
sum += *(unsigned char*)buf;
sum = (sum >> 16) + (sum & 0xFFFF);
sum += (sum >> 16);
result = ~sum;
return result;
}
void ping(struct sockaddr_in *addr) {
const int val = 255;
int i, sockfd;
struct icmp sendicmp;
struct sockaddr_in from;
char recvbuf[PACKET_SIZE] = {
0};
sockfd = socket(AF_INET, SOCK_RAW, IPPROTO_ICMP);
if (sockfd < 0) {
printf("socket() failed\n");
return;
}
setsockopt(sockfd, SOL_SOCKET, SO_RCVBUF, &val, sizeof(val));
sendicmp.type = ICMP_ECHO;
sendicmp.code = 0;
sendicmp.checksum = 0;
sendicmp.un.echo.id = getpid();
sendicmp.un.echo.sequence = 1;
sendicmp.checksum = checksum(&sendicmp, sizeof(sendicmp));
if (sendto(sockfd, &sendicmp, sizeof(sendicmp), 0, (struct sockaddr *)addr, sizeof(struct sockaddr_in)) < 0) {
printf("sendto() failed\n");
return;
}
struct sockaddr_in from_addr;
socklen_t from_len = sizeof(from_addr);
if (recvfrom(sockfd, recvbuf, sizeof(recvbuf), 0, (struct sockaddr*)&from_addr, &from_len) < 0) {
printf("recvfrom() failed\n");
return;
}
struct ip *iph = (struct ip *)recvbuf;
struct icmp *icmph = (struct icmp *)(recvbuf + (iph->ip_hl<<2));
if (icmph->icmp_type == ICMP_ECHOREPLY) {
printf("Received ICMP echo reply from %s\n", inet_ntoa(from.sin_addr));
} else {
printf("Received non-echo reply packet\n");
}
}
int main() {
struct sockaddr_in addr_ping, *addr;
addr = &addr_ping;
memset(addr, 0, sizeof(struct sockaddr_in));
addr->sin_family = AF_INET;
addr->sin_addr.s_addr = inet_addr("8.8.8.8"); // Google DNS server
ping(addr);
return 0;
}
The main part of the above program is the ping function, which sends an ICMP echo request and receives the response. The ping function first creates a raw socket and sets the receive buffer size.
Then, create an ICMP echo request message, including type (ICMP_ECHO), code (0), checksum, ID (PID of the current process), and sequence number. Calculate the checksum and write it to the message.
Use the sendto function to send ICMP messages to the specified destination address. Then, use the recvfrom function to receive the response. If the type of the received ICMP message is ICMP_ECHOREPLY, it means that an ICMP echo response has been received. Print the source address and return it.
The target address is set to "8.8.8.8" in the main function, which is Google's public DNS server and can usually be used for ping tests.
5. User mode network protocol stack
5.1 Definition of user mode network protocol stack
User mode network protocol stack refers to the network protocol stack running in user space. Traditional network protocol stacks, such as the TCP/IP protocol stack, usually run in the kernel space, while the user-mode network protocol stack transfers the implementation of the network protocol stack from the kernel space to the user space.
5.2 Advantages of user-mode network protocol stack
- Easy to develop and debug: The user-mode network protocol stack can directly utilize various development tools and libraries in user space, making development and debugging more convenient.
- Scalability: The user-mode network protocol stack can be expanded and modified more flexibly, and can better adapt to various specific application requirements.
- High performance: The user-mode network protocol stack can bypass the kernel's system calls and context switching and directly access the network hardware, thereby improving the efficiency of network processing.
5.3 Disadvantages of user-mode network protocol stack
- Special driver support is required: Since the user-mode network protocol stack directly accesses network hardware, network device drivers need to provide corresponding support.
- Security issues: The user-mode network protocol stack directly accesses the network hardware. Without proper isolation and permission control, security issues may arise.
- Compatibility issues: The user-mode network protocol stack may need to modify the application code to adapt to its API, which may cause some compatibility issues.
5.4 Example of user mode network protocol stack
At present, there are already some implementations of user-mode network protocol stacks, such as DPDK (Data Plane Development Kit), Seastar, etc. These user-mode network protocol stacks usually provide a complete set of APIs that allow applications to directly perform network programming without relying on kernel network services.
5.5 Relationship between user mode network protocol stack and Posix API
User-space network protocol stacks typically require their own APIs for network programming, which may differ from the Posix API. However, some user-mode network protocol stacks, such as Seastar, provide interfaces compatible with Posix API, making it easier to migrate existing Posix API-based applications to user-mode network protocol stacks.
6. TCP protocol demonstration
6.1 Basic knowledge of TCP protocol
Transmission Control Protocol (TCP) is a connection-oriented, reliable, byte stream-based transport layer communication protocol. It provides a stable and reliable data transmission service on top of the Internet Protocol (IP).
The TCP protocol contains the following main features:
- Connection-oriented: Before data transmission, a connection must be established between the sender and the receiver.
- Reliable transmission: Ensure data is reliably transmitted from the sending end to the receiving end through mechanisms such as sequence numbers, confirmation responses, retransmissions, and flow control.
- Flow control and congestion control: Control the data transmission rate and prevent network congestion through mechanisms such as sliding window, slow start, congestion avoidance, fast retransmission and fast recovery.
6.2 TCP three-way handshake
The establishment process of a TCP connection is usually called the "three-way handshake":
- First handshake: The client sends a SYN packet (sequence number is X) to the server, requesting to establish a connection.
- Second handshake: After receiving the SYN packet, the server replies with a SYN-ACK packet (sequence number is Y, confirmation number is X+1) to the client to confirm receipt of the SYN packet.
- Third handshake: After receiving the SYN-ACK packet, the client replies with an ACK packet (confirmation number is Y+1) to the server to confirm receipt of the SYN-ACK packet.
6.3 TCP’s four waves
The closing process of a TCP connection is often called "four waves":
- The first wave: the party that actively closes (assumed to be the client) sends a FIN packet (sequence number is M) to the other party, indicating that it will no longer send data.
- The second wave: After receiving the FIN packet, the passively closed party (assumed to be the server) replies with an ACK packet (confirmation number is M+1) to the client to confirm receipt of the FIN packet.
- The third wave: After all data is sent, the passively closed party sends a FIN packet (serial number N) to the other party to indicate that it will no longer send data.
- The fourth wave: After receiving the FIN packet, the party that actively closes responds with an ACK packet (confirmation number is N+1) to the passively closed party to confirm receipt of the FIN packet.
6.4 Relationship between TCP protocol and Posix API
In the Posix API, we can use the following main functions to implement TCP communication:
- socket function: Create a new socket.
- bind function: Bind the socket to a specific address and port.
- listen function: listens for connection requests on the socket.
- connect function: initiates a connection request to a specific address and port.
- Accept function: Accepts a connection request and returns a new socket to handle the connection.
- send and recv functions: send and receive data on an established connection.
- close function: Close a socket and related connections.
The use of these functions and how they interact with the TCP protocol is the core content of Posix API network programming.
7. TCP protocol stack implementation
7.1 Hierarchical structure of TCP protocol stack
When implementing the TCP protocol stack, we usually organize the code according to the hierarchical structure of the protocol. Each layer of the protocol corresponds to one or more modules. for example:
- Application layer: This layer contains processing modules for application protocols such as HTTP, FTP, SMTP, etc.
- Transport layer: This layer mainly implements the TCP and UDP protocols.
- Network layer: This layer mainly implements the IP protocol, including IP address processing, routing, IP packet encapsulation and decapsulation and other functions.
- Data link layer and physical layer: These two layers are usually implemented by the operating system and network device drivers. Applications generally do not directly handle the protocols of these two layers.
7.2 Main modules of TCP protocol stack
In the TCP protocol stack, there are mainly the following modules:
- Socket management module: Responsible for managing socket creation, binding, monitoring, connection, closing and other operations.
- Connection management module: Responsible for the establishment, maintenance and closing of TCP connections, including three-way handshake, four-way wave, state transition and other functions.
- Data transmission module: Responsible for the sending and receiving of data, including data encapsulation and decapsulation, sequence number and confirmation response processing, retransmission mechanism, sliding window and other functions.
- Congestion control module: Responsible for controlling the data sending rate and preventing network congestion, including mechanisms such as slow start, congestion avoidance, fast retransmission and fast recovery.
7.3 Data flow of TCP protocol stack
In the TCP protocol stack, the flow path of data is as follows:
- The application sends data to the transport layer by calling the send function.
- After the transport layer receives the data, it encapsulates the data into one or more TCP packets based on the destination IP address and port number, and sends them to the network layer.
- After the network layer receives the TCP data packet, it encapsulates the data packet into one or more IP data packets according to the destination IP address and sends them to the data link layer.
- After the data link layer receives the IP packet, it sends it to the physical network.
The process of receiving data is opposite to that of sending data, starting from the physical network, passing through the data link layer, network layer, transport layer, and finally reaching the application.
7.4 Relationship between TCP protocol stack and Posix API
In the TCP protocol stack, the Posix API serves as the interface between the application and the transport layer. Applications send and receive data by calling Posix APIs, and the TCP protocol stack is responsible for implementing these APIs and processing data transmission according to the rules of the TCP protocol.