Network layer IP protocol and data link layer

IP protocol

Host: equipped with IP address; router: equipped with IP address, capable of routing control; node: collectively referred to as host and router;

protocol header format

• 4-digit version number (version): Specify the version of the IP protocol, for IPv4, it is 4.
• 4-bit header length (header length): How many 32bits is the length of the IP header, that is, the number of bytes of length * 4. 4bit means that the maximum number is 15, so the maximum length of the IP header is 60 bytes.
• 8-bit service type (Type Of Service): 3-bit priority field (deprecated), 4-bit TOS field, and 1-bit reserved field (must be set to 0). 4-bit TOS respectively represent: minimum delay, maximum throughput Quantity, highest reliability, and minimum cost. These four conflict with each other, and only one can be selected. For
• For applications like ssh/telnet, minimum latency is more important; for programs like ftp, maximum throughput is more important.
• 16-bit total length (total length): How many bytes the IP datagram occupies as a whole.
• 16-bit identifier (id): uniquely identifies the packet sent by the host. If the IP packet is fragmented at the data link layer, the id in each fragment is the same.
• 3-bit flag field: The first bit is reserved (reserved means it is not used now, but it may be used in the future if it has not been decided yet). The second bit is 1, which means that fragmentation is prohibited. At this time, if the packet length exceeds the MTU, the IP module The message will be discarded. The third bit means "more fragmentation", if it is fragmented,
• The last fragment is set to 1, and the others are 0. Similar to an end marker.
• 13-bit fragment offset (framegament offset): It is the offset of the fragment relative to the beginning of the original IP message. In fact, it indicates where the current fragment is in the original message. The actual number of bytes offset is This value * 8 is obtained. Therefore, except for the last message, the length of other messages must be an integer multiple of 8 (otherwise the messages are not continuous).
• 8-bit time to live (Time To Live, TTL): The maximum number of hops for a datagram to reach its destination. Generally, it is 64. Every time it passes through a route, TTL -= 1, until it reaches 0, it will be discarded Yes. This field is mainly used to prevent routing loops
• 8-bit protocol: Indicates the type of the upper layer protocol
• 16-bit header checksum: use CRC to verify whether the header is damaged.
• 32-bit source address and 32-bit destination address: Indicates the sender and receiver.
• Option field (variable length, up to 40 bytes):

Fragmentation

Network segment division

The IP address is divided into two parts, the network number and the host number

• Network number: ensure that the two network segments connected to each other have different identifiers;
• Host number: In the same network segment, hosts have the same network number, but must have different host numbers;

• Different subnets are actually hosts with the same network number put together.
• If a host is added to the subnet, the network number of the host is the same as that of the subnet, but the host number must not be the same as that of other hosts in the subnet

By setting the host number and network number reasonably, you can ensure that in the interconnected network, the IP address of each host is different.
Then the problem comes, manually managing the IP in the subnet is a very troublesome thing.

• There is a technology called DHCP, which can automatically assign IP addresses to new host nodes in the subnet, avoiding the inconvenience of manual IP management.
• General routers have a DHCP function. Therefore, a router can also be regarded as a DHCP server.

In the past, a scheme of dividing network numbers and host numbers was proposed, and all IP addresses were divided into five categories, as shown in the figure below (this figure is from [TCPIP]).

With the rapid development of the Internet, the limitations of this partitioning scheme quickly emerged. Most organizations applied for Class B network addresses, resulting in the allocation of Class B addresses quickly, while Class A wasted a lot of addresses;

• For example, if you apply for a class B address, in theory, more than 65,000 hosts can be allowed in a subnet. There are more hosts in the subnet with a class A address.
• However, in the actual network construction, there will not be so many cases in a subnet. Therefore, a large number of IP addresses are wasted.

In response to this situation, a new division scheme is proposed, called CIDR (Classless Interdomain Routing):

• Introduce an additional subnet mask (subnet mask) to distinguish the network number from the host number;

• The subnet mask is also a 32-bit positive integer. It usually ends with a string of "0";

• Perform a "bitwise AND" operation on the IP address and the subnet mask, and the result is the network number;

• The division of network number and host number has nothing to do with whether the IP address is class A, class B or class C;

• Destination IP & subnet mask of the current router = the destination network to which the packet is going

• Because different routers must be cascaded with at least two networks. The network number of each network may be different; each router must configure the corresponding subnet mask for the network directly connected to itself
  

special IP address

• Set all the host addresses in the IP address to 0, which becomes the network number, representing this LAN;
• Set all the host addresses in the IP address to 1, which becomes a broadcast address, which is used to send data packets to all hosts connected to each other in the same link;
• The IP address of 127.* is used for the local loopback (loop back) test, usually 127.0.0.1
  

Limitation on the number of IP addresses

We know that an IP address (IPv4) is a 4-byte 32-bit positive integer. Then there are only 2 to the 32nd power of IP addresses, which is about 4.3 billion. The TCP/IP protocol stipulates that each host needs to have an IP address.

This means that only 4.3 billion hosts can access the network?
In fact, due to the existence of some special IP addresses, the number is far less than 4.3 billion; in addition, IP addresses are not configured according to the number of hosts, but each Each network card needs to be configured with one or more IP addresses.

CIDR has alleviated the problem of insufficient IP addresses to a certain extent (increased utilization and reduced waste, but the absolute upper limit of IP addresses has not increased), but it is still not enough. There are three ways to solve it at this time:

• Dynamic allocation of IP addresses: only assign IP addresses to devices connected to the network. Therefore, devices with the same MAC address will not necessarily get the same IP address every time they connect to the Internet;
• NAT technology (mainstream solution) [The source IP address is forwarded and replaced in different intranets and network nodes of different levels]
• IPv6: IPv6 is not a simple upgraded version of IPv4. These are two irrelevant protocols and are not compatible with each other; IPv6 uses 16 bytes and 128 bits to represent an IP address; but IPv6 is not popular yet;

NAT technology

NAT technical background

We discussed before that in the IPv4 protocol, the number of IP addresses is insufficient.
NAT technology is currently the main means to solve the shortage of IP addresses, and it is an important function of the router;

• NAT can convert private IP into global IP when communicating externally. That is, it is a technical method to convert private IP and global IP into each other:
• Many schools, families, and companies use private IP for each terminal, and set global IP on routers or necessary servers;
• The global IP is required to be unique, but the private IP is not required; the same private IP in different LANs is completely unaffected;

NAT IP conversion process

• The NAT router replaces the source address from 10.0.0.10 with the global IP 202.244.174.37;
• When the NAT router receives external data, it will replace the target IP from 202.244.174.37 to 10.0.0.10;
• Inside the NAT router, there is an automatically generated table for address translation;
• When 10.0.0.10 sends data to 163.221.120.9 for the first time, the mapping relationship in the table will be generated;

NPT

Then the problem comes, if there are multiple hosts in the LAN accessing the same external network server, then the destination IP in the data returned by the server is the same. So how does the NAT router determine which
LAN to forward the data packet to? Host?
At this time, NAPT will solve this problem. Use IP+port to establish this relationship

Defects of NAT Technology

Since NAT relies on this translation table, there are many limitations:

• Unable to establish a connection to the internal server from outside the NAT;
• The generation and destruction of the replacement table requires additional overhead;
• Once the NAT device is abnormal during the communication process, even if there is hot standby, all TCP connections will be disconnected;

NAT and Proxy Servers

Routers often have the function of NAT devices, which are transferred through NAT devices to complete the communication process between subnet devices and other subnet devices. The
proxy server looks a bit like a NAT device. The client sends a request like a proxy server, and the proxy server will The request is forwarded to the server that actually wants to request
; after the server returns the result, the proxy server sends the result back to the client.
So what is the difference between NAT and proxy server?

• In terms of application, NAT device is one of the basic network devices, and it solves the problem of insufficient IP. Proxy server is closer to specific applications, such as bypassing the wall through proxy server, and accelerators like Xunyou also use proxy server.
• In terms of underlying implementation, NAT works at the network layer and directly replaces the IP address. Proxy servers often work at the application layer.
• In terms of the scope of use, NAT is generally deployed at the egress of the LAN, and the proxy server can be implemented in the LAN, or in the WAN, or across networks.
• From the perspective of deployment location, NAT is generally integrated on hardware devices such as firewalls and routers, while the proxy server is a software program that needs to be deployed
  on the server.

Proxy server is a widely used technology.
Over the wall: proxy in the wide area network.
Load balancing: proxy in the local area network.
Proxy server is divided into forward proxy and reverse proxy.

Private IP address and public IP address

If a local area network is established within an organization, the IP address is only used for communication within the local area network, and not directly connected to the Internet. In theory, any IP address can be used, but RFC 1918 stipulates the private IP address used to establish a local area network

• 10.*, the first 8 digits are the network number, a total of 16,777,216 addresses

• 172.16. to 172.31., the first 12 digits are the network number, a total of 1,048,576 addresses

• 192.168.*, the first 16 digits are the network number, a total of 65,536 addresses

• Included in this range, all become private IPs, and the rest are called global IPs (or public IPs);
  

• A router can be configured with two IP addresses, one is the WAN port IP, and the other is the LAN port IP (subnet IP).

• The hosts connected to the LAN port of the router are all subordinate to the subnet of the current router.

• Different routers have the same subnet IP (usually 192.168.1.1). The IP addresses of hosts in the subnet cannot be repeated. However, the IP addresses between subnets can be repeated.

• Each home router is actually a node in the subnet of the carrier router. Such carrier routers may have many levels, and the outermost carrier router, WAN port IP is a public network IP.

• When the host in the subnet needs to communicate with the external network, the router replaces the IP address in the IP header (with the WAN port IP), and replaces it step by step, and finally the IP address in the data packet becomes a public network IP. This technology is called NAT (Network Address Translation, Network Address Translation).

• If we want our own server program to be accessible on the public network, we need to deploy the program on a server with an external IP. Such a server can be purchased on Alibaba Cloud/Tencent Cloud

routing

In a complex network structure, find a route to the destination; the
routing process is the process of "asking the way" hop by hop (Hop by Hop).
The so-called "hop" is the data link layer. An interval. Specifically, in Ethernet, it refers to the frame transmission interval from the source MAC address to the destination MAC address. The

transmission process of IP data packets is also the same as asking for directions.

• When the IP data packet arrives at the router, the router will check the destination IP first;
• The router decides whether the data packet can be sent directly to the target host or needs to be sent to the next router;
• Repeat in turn until reaching the target IP address;
  then how to determine where the current data packet should be sent? This depends on maintaining a routing table inside each node;
  
• The routing table can be viewed using the route command
• If the destination IP hits the routing table, just forward it directly;
• The last line in the routing table is mainly composed of the next hop address and the sending interface. When the destination address does not match other lines in the routing table, it will be sent to the next hop address according to the interface specified by the default routing entry.

Suppose the network interface configuration and routing table on a host are as follows:

• This host has two network interfaces, one network interface is connected to the 192.168.10.0/24 network, and the other network interface is connected to the 192.168.56.0/24 network;
• Destination in the routing table is the destination network address, Genmask is the subnet mask, Gateway is the next hop address, Iface is the sending interface, the U flag in Flags indicates that this entry is valid (some entries can be disabled), and the G flag indicates that this entry The next hop address is the address of a certain router, and the entry without the G flag indicates that the destination network address is a network directly connected to the local interface, and does not need to be forwarded by the router;

Forwarding process example 1: If the destination address of the data packet to be sent is 192.168.56.3

• Perform an AND operation with the subnet mask in the first line to get 192.168.56.0, which does not match the destination network address in the first line
• Then do an AND operation with the subnet mask in the second line to get 192.168.56.0, which is the destination network address in the second line, so it is sent out from the eth1 interface;
• Since 192.168.56.0/24 is the network directly connected to the eth1 interface, it can be directly sent to the destination host without forwarding through the router;

Forwarding process example 2: If the destination address of the data packet to be sent is 202.10.1.2

• Compared with the first few items in the routing table in turn, it was found that they did not match;
• According to the default routing entry, it is sent from the eth0 interface to the 192.168.10.1 router;
• The next hop address is determined by the 192.168.10.1 router according to its routing table;

Routing Table Generation Algorithm

The routing table can be manually maintained by the network administrator (static routing), or it can be automatically generated by some algorithms (dynamic routing).

data link layer

Used for transfer between two devices (same data link node)

Get to know Ethernet

• "Ethernet" is not a specific network, but a technical standard; it includes not only the content of the data link layer, but also some aspects of the physical layer.
• Content. For example: specifies the network topology, access control method, transmission rate, etc.;
• For example, the network cable in Ethernet must use twisted pair; the transmission rate is 10M, 100M, 1000M, etc.;
• Ethernet is currently the most widely used LAN technology; alongside Ethernet are Token Ring, Wireless LAN, etc.

Ethernet frame format

• The source address and destination address refer to the hardware address of the network card (also called MAC address), the length is 48 bits, and it is solidified when the network card leaves the factory;
• The frame protocol type field has three values, corresponding to IP, ARP, and RARP;
• At the end of the frame is the CRC check code.

Know the MAC address

• The MAC address is used to identify the connected nodes in the data link layer;
• The length is 48 bits, and 6 bytes. It is generally represented by a hexadecimal number plus a colon (for example: 08:00:27:03:fb:19)
• It is determined when the network card leaves the factory and cannot be modified. The mac address is usually unique (the mac address in the virtual machine is not the real mac address, which may conflict; some network cards also support user configuration of the mac address).

Compare and understand MAC address and IP address

• The IP address describes the overall start and end of the journey;
• The MAC address describes the start and end of each interval on the road;

Know MTU

MTU is equivalent to the limit on the package size when sending express. This limit is the limit of the physical layer corresponding to different data links.

• The data length in the Ethernet frame stipulates a minimum of 46 bytes and a maximum of 1500 bytes. The length of the ARP data packet is not enough for 46 bytes, and padding bits must be added at the back;
• The maximum value of 1500 is called the maximum transmission unit (MTU) of Ethernet, and different network types have different MTUs;
• If a data packet is routed from the Ethernet to the dial-up link, and the length of the data packet is greater than the MTU of the dial-up link, the data packet needs to be fragmented (fragmentation);
• The MTU of different data link layer standards is different;

Effect of MTU on IP protocol

Due to the limitation of the MTU of the data link layer, subcontracting is required for larger IP data packets.

• Divide larger IP packets into multiple small packets, and label each small packet;
• The 16-bit identification (id) of each small packet IP protocol header is the same;
• In the 3-bit flag field of the IP protocol header of each small packet, the second bit is 0, indicating that fragmentation is allowed, and the third bit is used to indicate the end mark (whether the current is the last small packet, if yes, set it to 1, otherwise set it to 0 );
• When arriving at the opposite end, these small packets will be reassembled in order, assembled together and returned to the transport layer;
• Once any of these small packets are lost, the reassembly at the receiving end will fail. But the IP layer will not be responsible for retransmitting data;

The impact of MTU on the UDP protocol

• Once the data carried by UDP exceeds 1472 (1500 - 20(IP header) - 8(UDP header)), it will be divided into multiple IP datagrams at the network layer.
• If any one of these multiple IP datagrams is lost, it will cause the network layer reassembly failure at the receiving end. This means that if the UDP datagram is fragmented at the network layer, the probability of the entire data being lost is greatly increased.

The impact of MTU on the TCP protocol

• A datagram of TCP cannot be infinitely large, and is still subject to MTU. The maximum message length of a single datagram of TCP is called MSS (MaxSegment Size);
• In the process of establishing a TCP connection, the two communicating parties will conduct MSS negotiation.
• Ideally, the value of MSS is exactly the maximum length that IP will not be fragmented (this length is still subject to the MTU of the data link layer).
• When both parties send SYN, they will write the MSS value they can support in the TCP header.
• Then after both parties know the MSS value of the other party, they choose the smaller one as the final MSS.
• The value of MSS is in the 40-byte variable length option of the TCP header (kind=2);

Relationship between MSS and MTU

View hardware address and MTU

ARP protocol

RP is not a pure data link layer protocol, but a protocol between the data link layer and the network layer;

The role of the ARP protocol

The ARP protocol establishes the mapping relationship between the host IP address and the MAC address.

• During network communication, the application program of the source host knows the IP address and port number of the destination host, but does not know the hardware address of the destination host;
• The data packet is first received by the network card and then processed by the upper layer protocol. If the hardware address of the received data packet does not match the local machine, it will be discarded directly;
• Therefore, the hardware address of the destination host must be obtained before communication;

Workflow of the ARP protocol

• The source host sends an ARP request, asking "what is the hardware address of the host whose IP address is 192.168. :FF means broadcast);
• The destination host receives the broadcast ARP request and finds that the IP address matches the local host, then sends an ARP response packet to the source host, and fills in its own hardware address in the response packet;
• Each host maintains an ARP cache table, which can be viewed with the arp -a command. The entries in the cache table have an expiration time (usually 20 minutes). If an entry is not used again within 20 minutes, the entry will become invalid, and an ARP request will be sent next time to obtain the hardware address of the destination host.
  

Format of ARP datagram

• Note that the source MAC address and the destination MAC address appear once in the Ethernet header and in the ARP request, which is redundant when the link layer is Ethernet, but it may be necessary if the link layer is another type of network .
• The hardware type refers to the link layer network type, 1 is Ethernet;
• The protocol type refers to the address type to be converted, 0x0800 is the IP address;
• The hardware address length is 6 bytes for Ethernet addresses;
• The protocol address length is 4 bytes for and IP address;
• If the op field is 1, it means an ARP request, if the op field is 2, it means an ARP response.

Comprehension

Other important protocols or technologies

DNS

DNS is a set of systems that map domain names to IPs

DNS background

Introduction to Domain Names

domain name resolution process

You can refer to the relevant chapters of <<Illustrated TCP/IP>>

Use the dig tool to analyze the DNS process

For more usage of dig, see
https://www.imooc.com/article/26971?block_id=tuijian_wz

ICMP protocol

The ICMP protocol is a network layer protocol
. A newly built network often needs to conduct a simple test to verify whether the network is smooth; but the IP protocol does not provide reliable transmission. If
the packet is lost Packet loss and the reason for packet loss.

ICMP function

ICMP message format (understand)

ping command

Telnet is port 23, ssh is port 22, so what port is ping? The
ping command is based on ICMP, which is at the network layer. The port number is the content of the transport layer. Information such as port number is not concerned at all in ICMP.

traceroute command

What happens after the url is entered in the browser.

What happens after entering the url in the browser

Summarize