Students who need cloud servers and other cloud products to learn Linux can move to / --> Tencent Cloud <-- / --> Alibaba Cloud <-- / --> Huawei Cloud <-- / official website, lightweight cloud servers are low-cost to 112 yuan/year, and new users can enjoy ultra-low discounts on their first order.
Table of contents
1. The difference between IP address and MAC address
3. The impact of MTU on IP protocol
4. The impact of MTU on UDP protocol
5. The impact of MTU on the TCP protocol
1. How to unpack and separate MAC frames in Ethernet
3. Principle of forwarding in LAN
1. Introduction to ARP protocol
5. Some important technologies or protocols
1. DNS (application layer protocol, using UDP protocol)
2. ICMP (Internet Layer Protocol)
Routers forward IP packets to each other, while MAC frames run on the network cable. The essence of cross-network transmission is the result of forwarding by countless local area networks (subnets). Then to forward data in the same LAN, you need to rely on the Ethernet protocol of the data link layer.
1. MAC frame
1. The difference between IP address and MAC address
The IP address describes the overall starting point and end point of the road;
The MAC address describes the starting point and end point of each interval on the road;
2. MAC frame protocol
The MAC frame protocol of the data link layer stipulates that its payload cannot exceed 1500 bytes (MTU (maximum transmission unit, which can be modified)). This 1500-byte limit includes the upper layer header + payload.
The size of the data packet can only be controlled by the transport layer, but sometimes it still exceeds 1500 bytes, which can only be solved by fragmentation and assembly of the IP protocol. The sending end IP layer fragments, and each fragment will have an IP header. The end IP layer is assembled (byte exceeding the limit is not a mainstream situation). TCP and MAC frames do not care about IP fragmentation and assembly of data packets. This is just the behavior of the IP layer itself.
Routers are also hosts and have their own maximum MTU limits. If a router's MTU is 500 bytes, but it receives a 1500-byte message from a host in the LAN, the router will split the message again. Pieces assembled!
3. The impact of MTU on IP protocol
Due to the limitation of the data link layer MTU, larger IP data packets need to be subpackaged:
1. Divide the larger IP packet into multiple small packets and label each small packet;
2. The 16-bit identifier (id) of the IP protocol header of each small packet is the same;
3. In the 3-bit flag field of the IP protocol header of each small packet, the second bit is 0, indicating that fragmentation is allowed, and the third bit indicates the end mark (whether it is the last small packet, if so, it is set to 1, otherwise it is set to 1) is 0);
4. When arriving at the opposite end, these small packets will be reorganized in order, assembled together and returned to the transport layer;
5. Once any of these small packets is lost, the reassembly at the receiving end will fail, but the IP layer will not be responsible for retransmitting the data. If the transport layer is TCP, it will time out and retransmit, and UDP will show packet loss.
4. The impact of MTU on UDP protocol
1. Once the data carried by UDP exceeds 1472 (1500 - 20 (IP header) - 8 (UDP header)), it will be divided into multiple IP datagrams at the network layer.
2. If any one of these multiple IP datagrams is lost, it will cause the network layer reassembly at the receiving end to fail. This means that if the UDP datagram is fragmented at the network layer, the probability of the entire data being lost greatly increases.
5. The impact of MTU on the TCP protocol
1. A TCP datagram cannot be infinitely large, and is still subject to MTU. The maximum message length of a single datagram of TCP is called MSS (Max Segment Size);
2. During the process of establishing a TCP connection, the communicating parties will conduct MSS negotiation.
Ideally, the MSS value is exactly the maximum length at which IP will not be fragmented (this length is still subject to the MTU of the data link layer).
3. When sending SYN, both parties will write the MSS value they can support in the TCP header.
4. Then after both parties learn the other party's MSS value, they choose the smaller one as the final MSS.
5. The value of MSS is in the 40-byte variable length option of the TCP header (kind=2).
2. Ethernet protocol
1. How to unpack and separate MAC frames in Ethernet
1.1 Unpacking
Use a fixed-length strategy. As shown in the figure, the header is 6+6+2=14 bytes, and the CRC check bit is 4 bytes. When unpacking, just pinch the head and remove the tail to fix the bytes, and the rest is the payload.
1.2 points for
As shown in the figure, the header of the MAC frame of the Ethernet protocol has a type field. This field contains which protocol in the network layer this message should be delivered to. Among them, 0800 represents the upper layer is the IP protocol; 0806 represents the upper layer is the ARP protocol; 8035 represents the upper layer is the RARP protocol, etc.
2. MAC address
The MAC address has a total of 48 bits. Every network card in the world has a unique MAC address. (Actually, the MAC address only needs to be unique in the LAN)
Use the ifconfig command to view the MAC address of the machine:
3. Principle of forwarding in LAN
1. A host in the LAN sends a MAC frame. All hosts in the LAN can receive the MAC frame. Since the destination MAC address field exists in the MAC frame, the data link layer of the host other than the IP can receive the frame. After arriving, the MAC frame will be discarded and will not be delivered upward. Only the host with the destination MAC address will receive and deliver the MAC frame upward.
2. In the LAN, the network card has a "promiscuous mode" (not enabled by default). This mode does not give up any data frames. All data frames obtained from the LAN are delivered upward regardless of whether the target MAC address is mine. This is also the principle of LAN packet capture tool.
3. In a local area network, if multiple hosts transmit data at the same time, the analog signals on the bus will interfere with each other and cause collisions. Therefore, at the same time, only one host in the LAN can send messages. In order to control that only one host in the local area network can send messages at the same time, there are a variety of standards to control it. For example:
Token Ring: In the Token Ring network, a token is circulated on the ring. Only the node holding the token can send data, other nodes listen to the link to receive data. When a node needs to send data, it appends the data to the token and passes the token on. Each node checks whether there is additional data when receiving the token. If so, the node extracts the data and processes it.
Ethernet: Ethernet uses a protocol called CSMA/CD (Carrier Sense Multiple Access with Collision Detection) to manage and schedule data transmission. In CSMA/CD, each node listens to the channel on the network to detect whether other nodes are sending data. If the channel is detected to be idle, the node can start sending data; otherwise, the node waits for a random period of time before trying to send again.
Certain methods can be used to continuously send junk data in the LAN (the host sending the junk data does not participate in the collision detection and collision avoidance algorithm, so it keeps sending). At worst, the data of other hosts in the LAN will be stuck, or at worst, the LAN will be paralyzed.
4. The larger the LAN is, the more likely the hosts in the LAN will be to collide. A switch can be used in a LAN. First, the switch can segment the LAN into regions. For example, the data forwarded between hosts A and D in the figure will not be forwarded to the other side by the switch; secondly, the switch can identify local collisions. If a collision occurs on one side of the switch, then The collision data will not be forwarded, and the collision will not affect the data forwarding on the other side.
5. Regarding the data length of each MAC, the longer the data, the more likely a collision will occur. This is why the MAC frame protocol limits the payload to a maximum of 1500 bytes.
3. ARP protocol
The source host only knows the destination IP but not the MAC address of the destination host. When the message reaches the destination IP address, it needs to find the corresponding host to submit the message. In LAN communication, the MAC address of the target host must be known to encapsulate data into MAC frames.
At this time, the router with the destination IP only knows the destination IP of the message through the message. If you want to know the MAC address of the host corresponding to this IP, the router needs to use the ARP protocol to find the mapping relationship between the IP and MAC addresses.
1. Introduction to ARP protocol
The ARP protocol is a protocol between the data link layer and the network layer.
1. During network communication, the application program of the source host knows the IP address and port number of the destination host, but does not know the MAC address of the destination host.
2. The ARP protocol establishes a mapping relationship between the host's IP address and MAC address.
1. The source host sends an ARP request, asking "What is the hardware address of the host with the IP address 192.168.0.1", and broadcasts the request to the local network segment (the hardware address in the header of the Ethernet frame is filled in FF:FF:FF:FF :FF:FF means broadcast);
2. The destination host receives the broadcast ARP request and finds that the IP address matches the local host, then sends an ARP response packet to the source host and fills in its own hardware address in the response packet;
3. Each host maintains an ARP cache table, which can be viewed with the arp -a command . The entries in the cache table have an expiration time (usually 20 minutes). If an entry is not used again within 20 minutes, the entry becomes invalid and an ARP request must be sent next time to obtain the MAC address of the destination host.
4. When ARP receives a response, the latest response will prevail.
2. Format of ARP protocol
1. Notice that the source MAC address and the destination MAC address appear once each in the Ethernet header and the ARP request. This is redundant when the link layer is Ethernet, but it may be true if the link layer is other types of networks. necessary.
2. Hardware type refers to the link layer network type, 1 is Ethernet;
3. The protocol type refers to the address type to be converted, 0x0800 is the IP address;
4. The hardware address length is 6 bytes for Ethernet addresses;
5. The length of the protocol address is 4 bytes for the IP address;
6. An op field of 1 indicates an ARP request, and an op field of 2 indicates an ARP reply.
7. Hosts in the LAN can obtain the MAC addresses of all hosts in the current LAN through the ARP protocol.
3. ARP request and response
1. All F means that the field is not set. For example, the destination Ethernet address is not known for ARP requests. When the ARP request is broadcast (the Ethernet destination is all F), other hosts in the LAN can receive the MAC frame and deliver it up to the ARP protocol .
2. When these hosts receive MAC frames of the ARP type, they use the op field to distinguish whether the ARP message is a response or a request.
3. When the requested host sends an ARP reply, other hosts at the data link layer can use the Ethernet destination address field to distinguish whether the MAC frame is for me.
4. ARP spoofing
The bottom layer can steal the message through ARP spoofing, so after the message is encrypted through the HTTPS protocol, even if the middleman gets the message, it cannot decrypt the message without the secret key.
Stealing messages is only one aspect. ARP spoofing can also be used to continuously send ARP responses to a certain host (or to all hosts in the LAN). The content of the response is the router's IP + a scrawled MAC address. Then a certain host receives this response. It will mistakenly think that the MAC address of the router is the MAC address you scribbled. As long as your ARP responses continue, this host will not be able to access the Internet.
4. RARP protocol
Only the MAC address of a certain host is known, but its IP address is not known. Send a RARP request to all hosts in the LAN. The host with the specified MAC address will deliver the request data frame to the RARP protocol. If the host that does not specify the MAC gets the data frame and finds that the MAC address does not match, the MAC frame will be in the data. The link layer is discarded.
5. Some important technologies or protocols
1. DNS (application layer protocol, using UDP protocol)
The entire network protocol stack uses IP instead of domain name. The Baidu URL we visit is the domain name. This is because IP is not friendly to people's memory and use, while domain names and URLs are more intuitive. DNS is a system that maps domain names to IPs.
For example, Baidu’s domain name:
www.baidu.comUse .connection between domain names
com: First-level domain name. Indicates that this is an enterprise domain name. Also at the same level are "net" (network provider), "org" (non-profit organization), etc.
baidu: Second-level domain name, company name.
www: is just a common usage. In the past, when people used domain names, they often named them in a format similar to ftp.xxx.xxx/www.xxx.xxx to indicate the protocols supported by the host. You can omit it when entering the domain name in the browser.
At present, the hosts file is still retained on the computer, and the contents of the hosts file will still be searched first during the domain name resolution process.
Domain name resolution process: see "Illustrated TCP/IP" for details.
2. ICMP (Internet Layer Protocol)
A newly built network often needs to conduct a simple test first to verify whether the network is smooth; but the IP protocol does not provide reliable transmission. If a packet is lost, the IP protocol cannot notify the transport layer whether or not the packet was lost and the reason for the packet loss. reason.
1. Confirm whether the IP packet successfully reaches the target address;
2. Notify the reason why the IP packet was discarded during the sending process;
3. ICMP also works based on the IP protocol, but it is not a function of the transport layer, so people still attribute it to a network layer protocol;
4. ICMP can only be used with IPv4. If it is IPv6, ICMPv6 needs to be used;
The bottom layer of the ping command is implemented through ICMP:
1. What is pinged here is the domain name, not the URL! A domain name can be resolved into an IP address through DNS;
2. The ping command can not only verify the connectivity of the network, but also count the response time and TTL (Time To Live, life cycle in the IP packet);
3. The ping command will first send an ICMP Echo Request to the peer;
4. After the peer receives it, it will return an ICMP Echo Reply;
The ping command does not pay attention to the port, because the ping command is based on ICMP and is at the network layer; and the port number is the content of the transport layer. In ICMP, information such as port numbers is not paid attention to at all.
The traceroute command is also implemented based on the ICMP protocol and can print out the host of the executable program and how many routers it has passed before reaching the target host:
3. Proxy server
Forward proxy: The client wants to request the target server, and the proxy server proxies the client's request to the target server to request resources and sends them to the client, which is called forward proxy. (Collect client requests and manage requests)
Reverse proxy (as the entrance machine of the computer room): The client wants to request a certain resource from the server, and the proxy server will return the result to the client from the intranet server cluster. The client does not know who the target server of the request is.
Load balancing strategy: The client may send a large amount of data, and the reverse proxy server needs to reasonably distribute the data to the server cluster.
As a reverse proxy server, high configuration is required. The software service can be Nginx (Web server, which does not do business processing, only forwarding processing). When the server cluster in the reverse proxy server has processed the request, you can use The proxy server sends the results back to the client, or directly to the client.
The difference between NAT and proxy server:
1. From an application level, NAT equipment is one of the basic network equipment and solves the problem of insufficient IP. Proxy servers are closer to specific applications, such as bypassing the firewall through proxy servers. In addition, accelerators like Xunyou also use proxy servers;
2. From the perspective of underlying implementation, NAT works at the network layer and directly replaces IP addresses. Proxy servers often work at the application layer;
3. In terms of scope of use, NAT is generally deployed at the exit of the LAN. The proxy server can be done on the LAN, the WAN, or across networks.
4. From the perspective of deployment location, NAT is generally integrated on hardware devices such as firewalls and routers, while the proxy server is a software program that needs to be deployed on the server.