Network knowledge small summary

OSI seven

Application layer (HTTP, DNS, FTP)
Presentation layer
Session Layer
Transport layer (TCP, UDP)
Network layer (IP)
Data link layer (MAC)
Physical Layer

UDP

Q: What is the difference between UDP and TCP?

A: First UDP protocol is connectionless-oriented, that is to say before the formal transfer of data does not need to connect from both sides. Then just UDP protocol data packets porter, does not ensure the orderly transfer without loss to the peer and UDP protocol and no flow control algorithm, compared to the general UDP TCP more portable.

TCP and UDP is basically the opposite of to establish a connection to disconnect all need to need to shake hands. In the process of transferring data, ensure data reliability through various algorithms, of course, the problem is caused UDP is not so efficient in comparison.

Feature

No connection, i.e. the connection before the data transfer is not required.
Packet-oriented, i.e., the transmitting side, the application layer passes data to the transport layer protocol is UDP, UDP only increasing the data to a UDP header identifying the protocol is UDP, and then passed to the network layer, is not split. At the receiving end, the data is passed to the network layer to the transport layer, UDP header removing only IP packets will be passed to the application layer, no spliced.
No congestion control, i.e., allowing loss of data when network congestion occurs.
Support many, one to one, many-to-many communication.
Header overhead small, only 8 bytes.

TCP

Feature

Connection-oriented, i.e., before the transfer to be connected.
One on one, full-duplex communication.
Byte stream oriented.
Provide reliable delivery, there is congestion control.

Three-way handshake

Q: Why establish a TCP connection requires three-way handshake, obviously twice to establish a connection?

A: Because this is to prevent the connection failure occurs in the case of a request packet segment is received the server, resulting in an error.

Imagine the following scenario. The client sends a connection request A, but because the network causes a timeout, then TCP retransmission timeout mechanism will start again send a connection request B. At this time, the request successfully reach the server, the server can be acknowledged on the establishment request, and after receiving the data connection is released.

Assuming this time after both ends of the connection request A closed finally arrived at the server, so this time the server assumes that the client and the need to establish a TCP connection, thus answering the request and enter the ESTABLISHED state. But the client is actually CLOSED state, it will cause the server waits a waste of resources.

The first handshake: a client sends a request message, a synchronization bit SYN is set, the initial sequence number seq counter x, and then enters SYN_SEND (synchronous sent) state, waiting for an acknowledgment to the server.
Second handshake: the server receiving the request packet, sends back an acknowledgment packet, the SYN and ACK (acknowledge bit) are set to 1, set ack acknowledgment number x + 1, the server's initial sequence number seq own set y, and then enters SYN_RECV (synchronization has been received) state.
Third handshake: kefuduan acknowledge packet is received, send an acknowledgment to the server, the ACK bit is set to 1, set ack acknowledgment number y + 1, seq set to x + 1, and then enters the ESTABLISHED (connection established) state. After the server receives, also enters ESTABLISHED state, the connection is completed.

Four waving

TCP is a full-duplex, and both ends need to send FIN ACK while disconnected.

The first handshake: a client stops sending data, transmits a release connection message to the server, FIN is set, then enter FIN_WAIT-1 (1 terminating wait) state.
The second handshake: server receives a connection release message, sends back an acknowledgment message, ACK set to 1, then enter CLOSE-WAIT (wait closed) after the state, clients receive a confirmation message, will enter FIN_WAIT-2 ( 2 termination waiting) state.

Then TCP connection is half-closed state, the client has stopped sending data, but the server is likely to continue to send data.
Third handshake: When the server finished sending data, the client sends a connection release message, FIN is set to 1, and then enters the LAST-ACK (acknowledgment last) state.
Fourth handshake: the client receives a connection release message, the acknowledgment message sent back, the ACK is set to 1, and then enter the TIME-WAIT (wait) state until 2MSL (2 times the maximum segment lifetime) after into the CLOSE state, the connection is closed. Server receives confirmation, entered CLOSE state, officially closed the connection.

Q: Why the client to enter the TIME-WAIT state, waiting time before entering the 2MSL CLOSED state?

A: In order to ensure the server can verify that the client received a response. If the client confirmed directly into finished CLOSED state response, if confirmation response because of network problems has not been reached, it will cause the server does not close properly.

ARQ protocol - timeout retransmission mechanism

Stop and wait protocol

Normal transfer process

A long period of sending messages to B, must stop transmitting and starts a timer and wait for the end of the response, received peer response to cancel the timer and sends the next packet within the period of time the timer.

Packet loss or error

Packet loss may occur in packet transmission process. This time over timer set time will again send the missing data until the end of the response, so it is necessary to send the data backed up each time.

Even if the normal transmission of messages to the peer, the problem may also occur during transmission packet error. At this time of the end will abandon the message and waits for the A-side retransmission.

PS: General timer set time will be greater than the average time of a RTT.

ACK timeout or lost

Response to the end of the transmission may also be the case of loss or timeout occurs. So over time A timer end will still be retransmitted packets. This time terminal B received the same number of packets and retransmitting the packet discards the response until the next sequence number A sends the packet.

Response may also occur in the case of a timeout of very late, then the terminal A determines whether the number has already received, if the received response can be had only dropped.

From the above description, I am sure you can see this is certainly not an efficient way. Assuming good network environment, each time you send data needs to wait for a while is certainly not acceptable. Well, since we can not accept this agreement is not so efficient, they continue to learn it relatively efficient protocol.

Continuous ARQ

In continuous ARQ, the transmit end has a transmission window , the data can be continuously transmitted within the window in the case where no response is received, so compared to the stop and wait ARQ protocol can reduce waiting time and improve the efficiency.

Cumulative confirmed

Continuous ARQ, the receiving end will continuously receive messages. If you stop and wait ARQ receiving a message is sent as a reply, it is a waste of resources. By cumulative confirmed that a reply can be unified response packet after receiving multiple messages. ACK flag in the packet can be used to tell the sender of the data before the serial number have all been received, the next send data after the serial number.

However, there are a total of confirmed drawbacks. In continuous receive messages, you may experience after receiving the packet sequence number 5, has not received the packet sequence number 6, but No. 7 after the message has been received. When this happens, only the ACK reply 6, this will result in the case of the transmitting side repeatedly transmits data.

Sliding window

In TCP, in fact maintains window ends: respectively sending side window and a receive window in bytes.

The sender window containing the data was sent but not received a response and can be sent but the data is not sent.

The transmitting end is determined by the window size of the reception window remaining. Receiver will receive window size of the current remaining write response packet sender receives a response after transmitting window disposed according to the value and size of the current network congestion, the size of transmitting window is constantly changing.

When the sender receives the reply message, it will follow the sliding window

Sliding window is a very important concept, it helps to realize the function of TCP flow control. The receiver via the message also inform the sender how much data can be sent, in order to ensure that the receiver can receive the data in time, prevent the full bandwidth of the receiver, but the sender has been sending data to appear.

Congestion Control

Congestion process includes four algorithms, namely: slow start, congestion avoidance, fast retransmit and fast recovery.

Slow start algorithm

Slow start algorithm, by definition, will be sent when the transfer window slowly began to expand exponentially, thus avoiding the outset transmit large amounts of data lead to network congestion. Surely we all downloaded resources, whenever we start the download time will find the download speed is slowly raise, rather than overnight direct pull full bandwidth.

Slow Start algorithm steps as follows

Initial connection setting a congestion window (Congestion Window) is 1 MSS (maximum amount of data in one segment)
Each had a window size multiplied by two RTT will
Exponential growth is certainly not without limit, so there is a threshold limit, when the window size is greater than the threshold value will start congestion avoidance algorithm.

Congestion Avoidance Algorithm

Congestion avoidance algorithm compared to a simple point, with every passing RTT plus a window size only, so to avoid the exponential growth of network congestion, slowly adjust the size to an optimum value.

During transmission case might timer expires, the TCP would think that this time the network congestion, will immediately carry out the following steps:

The threshold is set to half of the current congestion window
The congestion window to 1 MSS
Start Congestion Avoidance Algorithm

Fast retransmit

Fast retransmission and fast recovery usually appear together. Once the situation appears out of sequence packets received by the receiving end, the receiver will only respond to a final order of the correct message number. If the sender receives three duplicate ACK, without waiting for the timer expires but directly start the fast retransmit algorithm. The algorithm is divided into two types:

TCP Taho achieve the following

The threshold is set to half of the current congestion window
The congestion window to 1 MSS
Re-start slow start algorithm

TCP Reno achieve the following

Congestion window by half
The threshold is set to the current congestion window
Enter the fast recovery phase (end packet retransmission needed, upon receipt of a new ACK reply to withdraw from the stage), this way in case of loss of multiple packets is not so good
Use congestion avoidance algorithm

Fast recovery after the TCP New Ren improvements

TCP New Reno algorithm improves before TCP Reno defects algorithm. Before, fast recovery as long as a new ACK packet is received, it will exit quick recovery.

In TCP New Reno in, TCP sender write down the maximum number of three duplicate ACK segment.

If I have a segment data is 1 through 10 on the number of packets, which lost a number of 3 and 7 of the message, then the maximum number of segments is 10. Only the transmitting side receives the ACK number of the transponder 3. At this time the retransmission packets number 3, the reception Fangshun Li received ACK number will be sent if the transponder 7. TCP peer knows at this time a plurality of packets are not received, continues to send packets with sequence number 7, the receiver receives and sends Fangshun Li ACK response sequence number 11, then the transmitting side that the receiving segment has an end successful reception, the next will exit the fast recovery phase.

HTTP

Message composition

Request line
Neck
entity

Common status codes

2xx Success

200 OK server processes the request correctly
Range 206 requests, with the use of Content-Range, resumable

3xx Redirection

301 permanent redirect, the browser cache, the cache automatically jump to take the next
302 temporary redirect, the browser does not cache
304 not modified server resources unmodified, the cache has not expired, you can use the cache

4xx Client Error

400 client request syntax error
401 requires authentication information
403 Access Denied
404 requested resource was not found

5xx server error

500 server internal processing error
501 does not support server-side
503 server is down for maintenance or overloaded

HTTPS / TLS

HTTPS default port 443

Q: The difference between TLS and SSL?

A: SSL: (Secure Socket Layer, Secure Sockets Layer), a protocol layer between network layer protocols and application layer protocols positioned reliable connection-oriented. SSL mutual authentication through the use of digital signatures to ensure the integrity, the use of encryption to ensure privacy for secure communication between the client and the server. The protocol consists of two layers: SSL record protocol and the SSL handshake protocol.
TLS: (Transport LayerSecurity, Transport Layer Security), is used between two applications provide confidentiality and data integrity. The protocol consists of two layers: a recording TLS protocol and TLS handshake protocol.
SSL is designed to protect Web users developed by Netscape Communications, the current version is 3.0. The latest version of TLS 1.0 is a new protocol IETF (Engineering Task Force) to develop, it is built on SSL 3.0 protocol specification, is the follow-up version of SSL 3.0, the difference is minimal.

HTTPS, or to transmit information via HTTP, but the information is encrypted by TLS protocol.

TLS protocol located above the transport layer below the application layer. TLS protocol transmission for the first time need two RTT, followed by RTT can be reduced to a Session Resumption.

Two techniques used in TLS encryption, namely: symmetric encryption and asymmetric encryption.

Symmetric encryption :

Symmetric encryption is on both sides have the same secret key, both sides know how to decrypt the encrypted ciphertext.

This encryption is good, but the question is how to let both sides know the secret key. Because the data transmission network are taking, if the keys pass through the network, then, once the secret key is intercepted there is no encryption of sense.

Asymmetric encryption :

There are points of public and private key, a public key that everyone can know that the data can be encrypted with the public, but the data must be used to decrypt the private key to decrypt the private key of a public key distribution only just know.

This encryption can be the perfect way to solve the problems of symmetric encryption. Assume now that both ends of the need to use symmetric encryption, before this, you can use asymmetric encryption keys exchange.

Simple process is as follows: First, the server will publish the public key out, then the client will know the public key. Next, the client creates a secret key and public key encryption and sent by the server, service termination after receiving private key to decrypt ciphertext through the correct secret key, this time on both ends know what is the secret key.

TLS handshake

The client sends a random value and the encryption and protocol needs.
Server receives the client random value, they would generate a random value, and according to the protocol and encryption the client needs to use a corresponding manner, and sends its certificate (client certificate authentication if necessary Note)
The client receives the server certificate and verify valid, verified and then generates a random value by will, by the server's public key certificate to encrypt the random value sent to the server, if the server needs to verify that the client certificate will then shipped certificate
Server receives the encrypted random number value using the private key to decrypt and obtain the third random values, which have both ends of the three time random value may be generated according to the encryption key previously agreed upon by the three random values, the next cryptographic communication may be through the key
Through the above steps can be seen, in the TLS handshake phase, both ends of the asymmetric encryption manner of communication, but because asymmetric encryption performance loss is larger than symmetric encryption, so that when the formal data transmission, both ends communicate using symmetric encryption.

Request method

Q: The difference between the Post and Get?

A: First to introduce the concept of side effects and idempotent.

Side refers to resources on the server to make a change, the search is no side effects, side effects are registered.

Refers idempotent request transmission M and N times (two different and greater than 1), a consistent state of resources on a server, such as 10 and 11 registered accounts is not idempotent, changes to the article 10 and 11 It is idempotent. Because the former is more than an account (resources), the latter just update the same resources.

In the standard scenario, he said, Get more for no side effects, such as powers of scenes, such as search keywords. Scene Post used for side effects, no power, etc., such as registration.

Technically, he said:

Get request can be cached, Post can not

Post a relatively safe Get a little bit, because Get requests are contained in the URL (of course you want to be written bodyin is possible), and will be saved browser history. Post will not, but in case of capture they are the same.

URL have length limitations will affect the Get request, but the length limit is specified in the browser, not the provisions of RFC

Post support more encoding type does not limit the type of data

method

get access to resources
post transmission entity body
put to transfer files
head to get the message header
delete delete files
options inquiry request method support
trace the path traced
connect the connection requirements for tunneling protocol agent
patch supplement put method for local updates on a known resource

HTTP 1.0

defect

You can not reuse a link, the request is complete disconnects, need to slow start and TCP 3-way handshake.
Blocking thread (TCP connection can send a request, before the previous request has not completed, the subsequent requests are queued), influence each other leading to the request.

HTTP 1.1

Improve

A long connection (default keep-alive), Multiple Access
New
- HTTP (206, Range, Content-Range)
- Authentication (401)
- Cache
  - Expires (HTTP 1.0 existed)
  - Cache-Control
  - Last-Modified
  - ETag

HTTP 2.0

Q: HTTP 1.xx problems?

answer:

Head of line blocking

Multiple TCP connections, HTTP 1.xx version simultaneous requests depends on multiple TCP connections, the high cost of establishing a TCP connection, but also a problem of the slow start.

Redundant header, uncompressed header, and each will bring a cookie request header exactly like.

Text format, HTTP 1.xx version is transmitted in text format.

Only the client actively requests.

Multiplexing: HTTP 2 so that all communications are accomplished on a TCP connection, a connection can have any number above streams (Stream), the message is divided into one or more frames in a stream inside the transmission.
Binary framing layer: located between the application layer and the transport layer, using the binary protocol to transmit data in binary format
Header compression: the need to end the two sides in the browser and the server maintains a list of the same header, which includes a common header, and then encodes the header by Huffman coding, instead of just sending a Huffman coding when sending a request common header can be.
Server push: push server so that the server can predict resource needs of the client, the initiative pushed to the client's cache.

HTTP 3.0

While HTTP / 2 to solve a lot of problems before the old version, but there is still a huge problem, although this problem is not caused by itself, but the problem underlying support of the TCP protocol.

Since HTTP / 2 using multiplexing, the same general domain only need to use a TCP connection. When this connection appeared in the case of packet loss, it will result in performance situations HTTP / 2 is actually not as good as HTTP / 1 up.

Because in the event of a loss, the entire TCP retransmission must begin to wait, and it leads back all the data have been blocked. But for HTTP / 1, it can open a plurality of TCP connections, this situation will only affect one of the counter-connection may also be connected to the rest of the normal transmission of TCP data.

QUIC

Google engage QUIC protocol based on UDP, and use of the HTTP / 3, of course, before the HTTP / 3 called HTTP-over-QUIC, from the name, we can also be found, HTTP / 3 biggest transformation is the use of the QUIC.

While the efficiency of UDP is high, but not so reliable. QUIC Although based on UDP, but added a lot of features in the original basis, such as multiplexing, 0-RTT, use TLS1.3 encryption, flow control, and orderly delivery, retransmission and more. Here we have selected several important functions learning content under this agreement.

Multiplexer

While HTTP / 2 support multiplexing, TCP protocol but after all, is not this feature. QUIC original gifted achieve this functionality, a single data stream and can ensure an orderly transfer of delivery without affecting other data streams, such techniques to solve problems before TCP.

And QUIC performance in the mobile terminal will be better than TCP. Because TCP is based on IP and port to identify the connection, in this way changing the mobile client network environment is very fragile. But QUIC is to identify by way of a connection ID, regardless of your network environment changes, as long as the same ID, you can quickly re-connect.

0-RTT

By using technology similar to TCP Fast Open, cache context of the current session, the next time the recovery session, just prior to the service you want to cache by-side validation can be transmitted.

Error correction mechanism

If you say that this time I want to send three packages, then the agreement will be calculated XOR values of these three packages and issue a check packages individually, that is, issued a total of four packages.

Which occurs when the packet loss in the non-verification, the content can be calculated by the lost data packets other three packages.

Of course, this technique can only be used in case of loss of a package, if you lose you can not use multiple packets appear correction mechanism, and can only use a retransmission.

Cache

Strong Cache

Expires

Expires value corresponds to a GMT (Greenwich Mean Time), for example, Mon, 22 Jul 2002 11:12:01 GMTto tell the browser cache expiration time of the resource, if you have not had that point in time will not send the request, reads the resource content directly from the cache.

Cache-Control

Higher priority thanExpires

no-cache cache resources, but requires the use of a cache freshness initiation request for verification (304)
Do not cache no-store
max-age expiration time, in seconds (s)
public response can be a client, the proxy server cache
private response can only be client-side caching

Cache consultation

Last-Modified If-Modified-Since

When the server resources passed to the client, the last time the resource will change in Last-Modified: GMTthe form of added return together on a header to the client entity.

When you want to use the cache, the client will take If-Modified-Since: Last-Modified-valuethe request header to ask the server, the cache is fresh, if the server returns 304, directly fetch buffer used.

Defects: Last-Modified there is a problem, if a resource server is modified, but its actual contents did not change, because Last-Modified time match is not over and return the entire entity to the client (even if the client cache there are exactly the same resources).

ETag If-None-Match

Last-Modefied order to solve the problems, HTTP 1.1 also introduced ETag header field, the server generates a unique value by combining the contents of the documents and algorithm (usually hash) is sent to the client as the value of ETag.

When you want to use the cache, the client will take If-None-Match: ETag-valuethe request header to ask the server, the cache is fresh, if the server returns 304, directly fetch buffer used.

Note: the ETag higher priority than the Last-Modified

WebSocket

WebSocket aim is to solve the problem of two-way communication network transmission, that implement the service-side push.

Handshake

request

For compatibility, the current WebSocket use HTTP to complete the handshake, the need to carry the request header Upgradefield to indicate that the server requires WebSocket (WS, WSS) protocol to communicate.

ws protocol using the default 80port, wss protocol using the default 443port.

 GET /chat HTTP/1.1   //必需。

 Host: server.example.com  // 必需，WebSocket服务器主机名

 Upgrade: WebSocket // 必需，并且值为" WebSocket"。
 
 Connection: Upgrade // 必需，并且值为" Upgrade"。
 
 /* 必需，其值采用 base64 编码的随机16字节长的字符序列。 */
 Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ== 
   
   /* 浏览器必填，防止跨站攻击，浏览器一般会使用这个来标识原始域 */
 Origin: http://example.com 

 Sec-WebSocket-Protocol: chat, superchat //选填, 可用选项有子协议选择器。

 Sec-WebSocket-Version: 13 //必需, 版本。

response

Server receives requests handover protocol for communication, but still sends back an HTTP response packet, which carries Sec-WebSocket-Acceptheader, agreed connection, which generate the following values:

Globally unique (GUID, [RFC4122]) identification: 258EAFA5-E914-47DA-95CA-C5AB0DC85B11

Request header: Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ ==
Value, appends the string: "dGhlIHNhbXBsZSBub25jZQ == 258EAFA5-E914-47DA-95CA-C5AB0DC85B11";
SHA-1 obtained after: 0xb3 0x7a 0x4f 0x2c 0xc0 0x62 0x4f 0x16 0x90 0xf6 0x46 0x06 0xcf 0x38 0x59 0x45 0xb20xbe 0xc4 0xea
After Base64 obtained: s3pPLMBiTxaQ9kYGzzhZRbK + xOo =
The final result value as a first response value of Sec-WebSocket-Accept.

data transmission

Use WebSocket data frame (text, binary) communicates data piece after packaging framing transmission.

Sending end: a message into a plurality of frames, and sends the server;
The receiving side: receiving a message frame, and the frame associated reassembled into a complete message;

Stay connected

In order to maintain WebSocket client, real-time two-way communication server, you need to ensure that the client, TCP channel between the server keep the connection is not broken. However, for a long time there is no connection data exchanges, if maintained for a long time still, the connection may be a waste of resources involved.

But do not rule out some scenarios, the client, the server, although there is no data to and from a long time, but still need to stay connected. This time, you can use heartbeat to achieve.

Sender -> recipient: ping
Receiver -> Sender: pong