First understanding of network principles

Preface

In this era of information explosion, computer networks have penetrated into every aspect of our lives, from daily work to entertainment and leisure, all of which are inseparable from the support of the network. As a programmer, understanding network principles and using network programming is an indispensable ability for every programmer. Today I will share with you some knowledge about computer networks.

The development history of computer networks

If you want to understand the principles of the network, you need to first know how the current network developed. The history of network development can generally be divided into the following stages:

• Terminal-oriented computer networks . This stage was from the 1950s to the 1960s, when computers were still a new gadget. People used different physical media such as telephone lines, coaxial cables, etc. through various types of terminals, including remote terminals, dial-up terminals, etc. Open wire etc. for communication.
• A network based on shared hardware resources . This stage was from the 1960s to the 1970s. People began to realize that the sharing of hardware resources could greatly improve efficiency, so file servers, database servers, application servers, etc. appeared.
• A computer network standardized by network protocols . This stage was from the 1970s to the 1980s. With the continuous advancement of standardization work, many network protocols emerged, such as TCP/IP, HTTP, etc., which enabled different computer networks to communicate with each other.
• The birth and development of the Internet . This stage is from the 1990s to the present. The development of the Internet has greatly accelerated the spread of information, allowing people to easily obtain and share all kinds of information.

network interconnection

Network interconnection refers to connecting different networks to form a larger network system and realize data communication, resource sharing and collaborative work between networks. This process can include interconnection between LAN and LAN, interconnection between LAN and WAN, interconnection between WAN and WAN, etc. The purpose is to expand the scope of resource sharing so that more resources can be shared by more users.

local area network

Local Area Network (LAN) refers to a computer group composed of multiple computers interconnected within a certain area, usually within a radius of several kilometers. It can consist of two computers in an office or thousands of computers in a company. The LAN can realize functions such as file management, application software sharing, printer sharing, scheduling within work groups, email and fax communication services, etc. The LAN is closed and can communicate with remote LANs through a data communication network or a dedicated data circuit. , database or processing center are connected to form a larger-scale information processing system. A local area network is mainly composed of network hardware (including network servers, network workstations, network printers, network cards, network interconnection equipment, etc.), network transmission media, and network software.

Network communication can be easily carried out between hosts in a local area network, also called an intranet; communication between a local area network and a local area network is impossible without a connection .

How to build a local area network

There are three main ways to build a LAN network:

1. Establishing a local LAN through a router: The advantage of this method is that the host can dynamically obtain an IP address and does not need to manually set the IP address. However, the efficiency of intranet data transmission will be affected by the performance of the router.
2. Establishing a local LAN through switches: The characteristic is that the host needs to manually set the IP address, and the data transmission efficiency of the intranet is affected by the performance of the switch.
3. Establishing a local LAN through direct connection through network cables: This is also a relatively traditional method, and its characteristic is that the host needs to manually set the IP address. Its transmission efficiency is the highest among the three. The transmission efficiency of Category 5e network cable can reach 100M/s, and the transmission efficiency of Category 6ae network cable can even reach 1000M/s. This connection method is suitable for large file, multi-batch, and data transfer between hosts.

Wan

Wide Area Network (WAN), also known as external network and public network, is a remote network that connects computer communications in local area networks or metropolitan area networks in different regions. It usually spans a large physical range, ranging from tens to thousands of kilometers, and can connect multiple regions, cities or countries, or span several continents and provide long-distance communications, forming an international network. remote network.

The communication subnet of the WAN mainly uses packet switching technology. It can use public packet switching networks, satellite communication networks and wireless packet switching networks to interconnect local area networks or computer systems distributed in different areas to achieve the purpose of resource sharing.

The characteristics of WAN include adapting to the requirements of large-capacity and burst communication, adapting to the requirements of comprehensive business services, open device interfaces and standardized protocols, and perfect communication services for network management.

WAN is not the same as the Internet. If it is a global public WAN, it is called the Internet.

The so-called "local area network" and "wide area network" are just relative concepts. For example, our WAN with "Chinese characteristics" can also be regarded as a relatively large LAN.

Network communication basics

Network communication is to connect various isolated devices through the network, and realize communication between people, people and computers, and computers and computers through information exchange. But with so many computers, how do you know which computer to send to which computer? This requires the use of IP address for identification.

IP address

An IP address refers to an Internet Protocol address, which is the number of a device connected to the Internet. It assigns a logical address to every network and every host on the Internet to mask differences in physical addresses.

In computers, each computer has a corresponding IP address. The IP address is a list of numbers. In the current version of IPv4, the IP address is a 32-bit binary number divided into 4 bytes, 01100100.00000100.00000101.00000110. The most common representation method is to express these binary numbers in dotted decimal system, with a total of 4 segments, 100.4.5.6.

There are two types of IP addresses, one is a public address (global IP address), and the other is a private address (private IP address). Public addresses are directly managed by the Internet, and IP addresses are requested from the Internet for access. Private addresses are unregistered addresses and are used by specialized organizations.

Special IP

The IP address of 127.* is used for local loop back testing, usually 127.0.0.1.
The local loop back is mainly used for local-to-local network communication (for the sake of performance, the system will not use the network method internally. Transmission), for programs that develop network communication (that is, network programming), the common development method is local-to-local network communication.

The IP address solves the problem of locating the network host during network communication, but there is still a problem. After it is transmitted to the destination host, which process
will receive the data? This requires a port number to identify it.

The port number

The port number is an integer used to identify services in network communications, which can be compared to the "house number" of a house.

In network communications, hosts need a way to identify and connect to different applications and services. To achieve this, each application is assigned an independent port number.

There are two types of port numbers: logical port numbers and physical port numbers. The logical port number is used to distinguish different services. For example, the port number of the HTTP service is 80, the port number of the FTP service is 21, etc. The physical port number is used to connect the interfaces between physical devices, such as interfaces on ADSL Modems, hubs, switches, and routers used to connect other network devices.

The ports of an IP address are numbered using 16 bits. There can be up to 65536 ports, and the port numbers range from 0 to 65535. Different operating systems allocate temporary port numbers differently. Most TCP/IP implementations allocate port numbers between 1024-5000 to temporary port numbers. Port numbers in the range of 0~1023 are usually reserved by the system for special purposes, which are used by some common servers. For example:

• 22 -> ssh
• 80 -> http
• 443 -> https
• 23 -> telnet

Two different processes cannot bind the same port number, but one process can bind multiple port numbers.

Port allocation and management are controlled by the operating system or a specific network protocol (such as TCP/IP). By using specific port numbers, different applications and services can communicate correctly on the network.

After you know which program on which computer is transmitting data to the designated program on the designated computer, there is another problem. Because the transmitted data is all transmitted in binary form, how do you let the other party know that the data you are transmitting is In what format and in this format is it parsed into data we can understand? This requires the use of our protocol.

Network protocol

A network protocol is a set of rules, standards or conventions established for data exchange in a computer network. For example, a microcomputer user communicates with a mainframe operator on the network. Since the two data terminals use different character sets, the commands entered by the operators do not recognize each other. In order to enable communication, it is stipulated that each terminal must first convert the characters in its own character set into the characters in the standard character set before entering the network for transmission. After reaching the destination terminal, it is then converted into the characters in the terminal's character set. Of course, for incompatible terminals, in addition to changing the character set characters, other characteristics such as display format, line length, number of lines, screen scrolling method, etc. also need to be changed accordingly.

Network protocols are composed of three elements:

1. Semantics, which defines the meaning of information in the protocol.
2. Timing defines the sequence of events.
3. Syntax defines the format and transmission method of protocol information.

The role of network protocols

1. Data transmission: Network protocols specify the transmission method and transmission rate of data in the network, allowing data to be transmitted between different computers and devices. For example, the TCP/IP protocol specifies the process and format of data transmission, allowing data to be transmitted over the Internet.
2. Network management: Network protocols can also be used for network management. For example, the SNMP protocol can be used for the management and monitoring of network devices, allowing network administrators to remotely manage and monitor network devices.
3. Security: Network protocols can also be used for network security. For example, the SSL/TLS protocol can be used to encrypt network communications and protect data security. In addition, the IPSec protocol can be used to establish network security tunnels to protect the security of network communications.

In general, the role of network protocols in computer networks is to ensure the reliability, efficiency and security of data transmission, as well as to achieve remote management and monitoring of network equipment.

Quintuple

In the TCP/IP protocol, five-tuples are used to identify a network communication:

1. Source IP: identifies the source host
2. Source port number: identifies the process in the source host that sends data for this communication.
3. Destination IP: identifies the destination host
4. Destination port number: identifies the process in the destination host that receives data for this communication.
5. Protocol number: identifies the data format agreed upon by both the sending process and the receiving process.

We use an example to understand why these quintuples are needed for network communication.

Suppose I buy something from the Internet. After I complete the order, the merchant needs to send the sender's address, sender's name and phone number, and the recipient's address, recipient's name and phone number to the courier company. Then the recipient The address is equivalent to the destination IP , but it is not enough to just deliver the express to a certain community. You also need to know who to send it to, so the recipient's name and phone number here are equivalent to the destination port . Whether to use JD.com or ZTO Express? This one belongs to the protocol type , and the corresponding sender address, sender name and phone number are equivalent to the source IP and source port.

Protocol layering

Why is protocol layering needed? Because network communication scenarios are relatively complex, there are many problems that require the use of protocols. If a large protocol is used to handle all these scenarios, the size of the protocol will be very large, which is not conducive to our learning and understanding. , so in this case protocol layering occurs.

What is protocol layering?

Protocol layering is a way to implement network protocols. It divides the protocol architecture into several levels based on the functions and services required for network communication. Each level is responsible for completing certain tasks and providing specific services.

The purpose of protocol layering is to simplify the complexity of network protocols, improve the efficiency and reliability of protocol implementation, and reduce development costs and maintenance difficulties. Through layering, protocols between different layers can be designed and implemented independently, allowing different types of applications and devices to communicate in the same network and achieve interoperability.

Protocol layering has the characteristics of conceptualization and structure. Each layer has its specific functions and responsibilities. The upper layer protocol calls the lower layer protocol, and the lower layer protocol provides services for the upper layer protocol. Different levels of protocols communicate through interfaces, and each layer abstracts and encapsulates the services provided by its lower layer.

For example: Suppose I founded a company. When the company just started, there were only a dozen employees. I could manage these employees alone. However, as the company continued to expand, the number of employees expanded from a dozen to There are more than a hundred. At this time, it is unrealistic for me to manage all the employees of the company by myself, so at this time, I recruited a few middle-level leaders to be responsible for managing the lower-level employees, and I only need to manage these dozen middle-level leaders. That’s it, this can not only achieve the purpose of management but also reduce my burden.

Common protocol layered structures include TCP/IP protocol stack and OSI model. The TCP/IP protocol stack divides the protocol into application layer, transport layer, network layer and link layer, while the OSI model is divided into application layer, presentation layer, session layer, transport layer, network layer, link layer and physical layer. These layers have their specific functions and responsibilities. For example, the application layer is responsible for application communication and data transmission, and the transport layer is responsible for end-to-end data transmission.

• For the user, it does not care how the provider implements it, it only needs to use the interface
• For the provider, using the characteristics of encapsulation, the implementation details are hidden, and only the interface needs to be opened.
  

OSI seven-layer model

The OSI seven-layer model is an open system interconnection communication reference model proposed by the International Organization for Standardization. It aims to interconnect various computers around the world as a standard framework for networks. According to recommendation X.200, OSI divides the computer network architecture into the physical layer, data link layer, network layer, transport layer, session layer, presentation layer and application layer.

Specifically, the physical layer is responsible for converting data into electronic signals, the data link layer is responsible for encapsulating electronic signals into data frames, and sending the frame data to the network layer. The network layer is responsible for determining data routing, and the transport layer is responsible for providing reliable connections. The session layer is responsible for allowing users to establish connections, the presentation layer is responsible for negotiating data exchange formats, and the application layer is the interface between user applications and the network, responsible for sending and receiving data.

Each layer is responsible for a specific job, and then the data is transmitted from the higher layer to the lower layer for corresponding processing. The OSI seven-layer model has a wide range of functions and applications. It is one of the foundations of network communication and an important part of the computer network architecture.

The OSI model usually only appears in textbooks, but in reality we more commonly use the TCP/IP five-layer protocol.

TCP/IP five layer protocol

The TCP/IP communication protocol adopts a 5-layer hierarchical structure. Each layer calls the network provided by the next layer to complete its own needs.

• Application layer: Responsible for communication between applications, such as Simple Email Transfer (SMTP), File Transfer Protocol (FTP), Network Remote
  Access Protocol (Telnet), etc. Our network programming is mainly for the application layer.

• Transport layer: Responsible for data transmission between two hosts. Such as Transmission Control Protocol (TCP), which can ensure that data is sent reliably from the source host to
  the target host.

• Network layer: Responsible for address management and routing. For example, in the IP protocol, a host is identified by its IP address, and
  the data transmission line (routing) between the two hosts is planned through the routing table. Router works at the network layer.

• Data link layer: Responsible for the transmission and identification of data frames between devices. For example, the driver of the network card device, frame synchronization (that is, what signal is detected from the network cable
  is counted as the beginning of a new frame), conflict detection (automatic retransmission if a conflict is detected), data error checking,
  etc. There are standards such as Ethernet, Token Ring, and Wireless LAN. Switch works at the data link layer.

• Physical layer: Responsible for the transmission method of optical/electrical signals. For example, the common network cables (twisted pairs) used by Ethernet, the coaxial cables used by early Ethernet
  (now mainly used for cable TV), optical fibers, and the current wifi wireless network using electromagnetic waves all belong to the concept of the physical layer.
  The capabilities of the physical layer determine the maximum transmission rate, transmission distance, anti-interference, etc. Hub works at the physical layer.
  

Let's use an example to understand the role of each layer in the TCP/IP five-layer model.

When we buy a product online, we need to provide the sender address - source IP, sender name and phone number - source port number, recipient address - destination IP, recipient name and phone number ——Information such as the destination port number, the merchant uses this information to determine the shipping location and receiving location, and determines a certain express company. ——Protocol type, we belong to the application layer, and this merchant implements the transport layer The role of the transmission layer is to determine the starting position of data transmission . When the data is transmitted to the express company, the express company will first plan the route to see which route has the shortest distance and the fastest time. Planning the route is The role of the network layer is that after the express company determines the route, it will deliver the package to the courier. This journey must pass through many cities, and the transportation methods between different cities will also be different. Two nodes The data transmission route and data transmission method between them are what the data link layer needs to do , and the physical layer is hardware facilities such as network cables and signal towers . When the express delivery reaches us, how to use this product is what we Why we buy this product depends on our own decision, and the application layer is something that we programmers need to focus on.

The MAC address is used for data transmission between two adjacent nodes.

The layer where the network device is located

• For a host, its operating system kernel implements content from the transport layer to the physical layer, which is the lower
  four layers of the TCP/IP five-layer model;
• For a router, it implements the network layer to the physical layer, which is the lower three layers of the TCP/IP five-layer model;
• For a switch, it implements from the data link layer to the physical layer, which are the lower two layers of the TCP/IP five-layer model;
• For hubs, it only implements the physical layer;

Note that we are talking about switches and routers in the traditional sense, also known as Layer 2 switches (working on the lower
two layers ) and Layer 3 routers (working on the lower three layers of the TCP/IP five-layer model). layer).

When it comes to switches, you may ask what is a switch?

A switch is a network device used for forwarding electrical (optical) signals. It can provide an exclusive electrical signal path for any two network nodes connected to the switch. The most common switch is an Ethernet switch, and other common ones include telephone voice switches, fiber optic switches, etc.
According to the needs of information transmission at both ends of the communication, the switch can automatically send the information to be transmitted to the corresponding route that meets the requirements. This technology is called switching technology.
Switches can be divided into WAN switches and LAN switches based on different working locations. WAN switches are used in the data link layer and are devices that can complete information exchange functions in communication systems. LAN switches are used in LAN networking technology, and network switches are the most popular.

To put it more simply, everyone should have a router at home. There are usually 5 network ports on the router, and the functions of these five network ports are different:

• 1 WAN port, used to connect to the upper-level router
• 4 LAN ports for connecting to local computers

If there are many local computers, these four LAN ports are not enough, so you need to use a switch to connect the LAN ports, and then the local computer can connect to the switch. The switch can expand the network port.

Encapsulation and decentralization

Encapsulation and demultiplexing describe the basic data transmission process during network communication.

• Different protocol layers have different names for data packets. They are called segments at the transport layer, datagrams at the network layer
  , and frames at the link layer.
• When application layer data is sent to the network through the protocol stack, each layer of protocol must add a data header (header), called encapsulation
  (Encapsulation).
• The header information contains information such as how long the header is, how long the payload is, and what the upper layer protocol is.
• The data is encapsulated into frames and sent to the transmission medium. After reaching the destination host, each layer of protocol strips off the corresponding header, and
  hands the data to the corresponding upper layer protocol for processing according to the "upper layer protocol field" in the header.

Then we will explain to you the process of encapsulation and separation by simulating the process of me sending a WeChat message to others.

sender

1. Application layer

Suppose I send a "hello" to someone. When sent, the message will be packaged into an application layer packet according to the application layer protocol, which is the protocol specified by the programmer. Assume that the application layer protocol is like this.

Four fields, each separated by, constitute an application layer data packet (string concatenation). The separation of the above fields and the symbols used to separate them can be flexibly changed. When the data packet of the application layer is generated, the data will be transmitted to the transport layer through the API of the operating system.

2. Transport layer

When the transport layer receives the application layer data packet from the application layer, the transport layer will also package the data packet, which is essentially string concatenation, but the protocol used is a different protocol. Typical protocols in the transport layer are UDP and TCP protocols . I will explain the specific content of the protocols here in detail later. Here we take the UDP protocol as an example.

The data in the UDP header is binary data, and the UDP header mainly contains source port and destination port information . After the transport layer data packet is generated, this data will continue to be transmitted to the network layer.

3. Network layer

A typical protocol in the network layer is the IP protocol.

The IP header mainly contains important information such as source IP and destination IP . When the network layer data packet is generated, the data will continue to be handed over to the data link layer for transmission.

4. Data link layer

A typical protocol in the data link layer is the Ethernet protocol.

The Ethernet header contains important information such as the MAC address. The MAC address is also used to describe the address of a device on the network. After the data packet of the data link layer is generated, the data will continue to be passed to the physical layer for transmission.

5.Physical layer

When the physical layer gets the data packet from the data link layer, it converts the data into binary data and transmits it through optical/electrical signals.

When the data is sent, because A and B are generally not directly connected by network cables, they need to be forwarded through many routers and switches in the middle.

The above process is the process of encapsulating data. When B gets the data, it will "separate" the above data.

receiver

1.Physical layer

When the received data passes through the physical layer, the received photoelectric signal - "binary data -" Ethernet data will be passed, and then this data will be passed to the data link layer for processing.

2. Data link layer

Through the Ethernet protocol, the Ethernet data is parsed to parse out the Ethernet header, trailer and Ethernet payload. Based on the MAC address in the Ethernet header, it is decided to deliver the Ethernet payload to the destination address, which is the network layer.

3. Network layer

Through the IP protocol, this data packet is parsed, the IP header and IP payload are parsed, and the IP payload is passed to the transport layer according to the target IP in the IP header.

4. Transport layer

When passing through the transport layer, the data packet will be parsed through the UDP protocol, the UDP header and UDP payload will be parsed, and the UDP payload will be passed to the corresponding application layer according to the port number in the UDP header.

5. Application layer

When the data is transmitted to the application layer, the data will be transmitted to WeChat through the specified port number, and then the WeChat program will parse the data through the application layer protocol, and finally take out "hello" and display it on the interface.