Computer network basics - application layer (a must-read for cloud migration)

basic concept

• Process: A program running on the host. A host can have multiple processes at the same time.
  • Communication via network message exchange
  • In P2P, a process can be either a client or a server.
  • Among a pair of processes, the one initiating the process is the client; the one waiting to be contacted is the server.
• Socket: The interface (API) that connects the application process and the transport layer
  • Developers can control everything about the API at the application layer
  • Very little choice and control over the transport layer
• process addressing
  • Host identification: IP address
  • Identifiers of different processes : port numbers
    • HTTP：80
    • HTTPs：443
    • Mail(SMTP)：25
    • POP3: 110
• The basis for process communication between different hosts: IP + port number
• Application layer protocol: public protocol (HTTP) + private (P2P)

Network requirements for transport services

• Requires 100% reliable data transmission: file transfer, telnet
• Allow certain data loss: VoIP
• Elastic bandwidth application: Email
• Applications with minimum bandwidth requirements: Live streaming

Internet transport layer services

TCP service

• Connection between Cilent/Server processes
• Reliable transmission - but no delay guarantee or bandwidth guarantee

UDP service

• No connection, best effort, no guarantee
• No handshake, easily blocked by fire protection configuration

The structure of web applications

• C/S (client/server) architecture: the server is online at all times, and the client can only communicate through the server
• P2P architecture: node IP is variable, intermittent network access, and nodes can communicate directly
  • In the P2P structure, nodes can be client processes and service processes at the same time.
• C/S+P2P hybrid structure: file retrieval-C/S, file transfer P2P

---

HTTP

basic concept

• HTTP (HyperText Transfer Protocol): Hypertext Transfer Protocol, application layer protocol
  • A Web page = an HTML file + multiple objects (pictures, text, music, etc.)
  • Based on TCP connection
  • Stateless protocol - the server does not maintain any information about past requests made by the client
• URL(Uniform Resoure Locator): Uniform Resource Locator
• URL format: Scheme://host:port/path, protocol;//hostname:port/file path

HTTP connection

• RTT (Round Trip Time) - the round trip time for sending an infinitesimal data packet between C/S
• Time required to request a web page file - 2RTT + file transfer time
  • The first RTT is the first two steps of the handshake phase
  • The second RTT is the third step of the handshake phase + request + response
• Non-persistent connection:
  • Establish TCP connection - Request 1 - Response 1 - Close TCP connection - Establish TCP connection - Request 2 - Response 2 - Close TCP connection
• Persistent connection:
  • Establish TCP connection-Request 1-Response 1-Request 2-Response 2-Close TCP connection

HTTP message

HTTP request message

• Composition: request line + header + message body

• HTTP1.0 request method
  • POST method: information is stored in the Body
  • GET method: Information is stored in the URL field
  • HEAD: Serve's response message only returns the header
• HTTP1.1 new methods:
  • PUT: Upload the file in the message body to the path specified by the URL
  • DELETE: Delete the file specified by the URL field

HTTP response message

• Composition: status line + header + message body

• Status code: Indicates the status of request processing
  • 200 0K: The request is successful and the information is in the returned response message.
  • 301 Moved Permanently: The requested object has been permanently moved
  • 400 Bad Request: The request cannot be understood by the server
  • 404 Not Found: The requested document is not on the server.

Cookies technology

• Core function: Solve the stateless characteristics of HTTP
• storage:
  • The browser has a cookies file (for local storage)
  • The database of the resource server has a cookies file (used to verify cookies)
  • There is a cookies field in the header of the HTTP request message (brought with it when requesting)
  • The HTTP response message has the word cookies in the header (bring it when responding)
• Features: Saved locally, present in the header field when requesting a response
• Cookies-like technologies, session and token mechanisms

Web caching/proxy server technology

basic concept

• Core function: Establish a Web cache to allow customers to quickly obtain resources without accessing the resource server (provided that there are corresponding resources in the proxy)
• When cache hits - proxy server = server + client
• Shorten response time, reduce server pressure, reduce traffic transmission, and improve efficiency

Conditional GET method

• Function: Ensure the timeliness of resources in the cache server
• Implementation: When the cache is hit, the proxy server sends a GET request to the resource server with the If-Modified-Since field in the header
  • When the resource is up-to-date: the resource server only returns the response headers
  • When the resource is not the latest: the resource server returns the response header + response body

---

Email application

Mail Transfer Protocol—SMTP

• SMTP (Simple Mail Transfer Protocol), based on TCP, port 25
• Features:
  • Communication between mail servers
  • Based on TCP + long connection
  • Server = client + server
  • Email messages must consist of 7-digit ASCII codes
  • Without using a relay server, even if two servers span the Pacific Ocean, the two servers transmit directly
• other
  • Three stages of transmission: handshake, transmission, shutdown
  • Command: ASCII text
  • Response: status code + statement
  • The message ends with CRLF.CRLF (CR=carriage return, LF=line feed)
• SMTP vs. HTTP:
  • HTTP: Pull protocol, each object is encapsulated in an independent response message.
    • Respond to a text + graphic file, encapsulated in two responses
  • SMTP: push protocol (push), multiple objects are put into the same message
    • Respond to a text + graphic file, encapsulated in a message
  • SMTP messages are forced to use 7-bit ASCII encoding, and HTTP has no restrictions.
  • Both use the command/response interaction mode, and the commands and status codes are ASCII codes.
• SMTP message header format: From+To+Subject (optional)

Mail Access Protocol

• The protocol for user agents to access server-side mail (differentiating SMTP transport protocols)

POP3

• POP3 (Post Office Protocol——Version 3): The third version of the Post Office Protocol, based on TCP, port 110
• Work stages: authorization, transaction processing, update
  • Authorization: The user agent (browser) sends the username + password to the server in clear text
  • Transaction processing: download emails to local
  • Update: Update the mail status of the server, such as automatically deleting read mails in the server after downloading
• Disadvantages: session information is not maintained, and it is inconvenient to use multiple devices

IMAP

• IMAP (Internet Mail Access Protocol): Internet Mail Access Protocol
• Maintain session information: Resolve multi-device issues

HTTP

• Use HTTP to interact with the browser and mail server to obtain mail, but still use SMTP to transmit mail!

---

DNS

basic concept

• DNS (Domain Name System): Domain name service system, based on UDP, port 53
• Function: Replaces IP address to facilitate memory (analogous to mobile phone address book service)
• HTTP, SMTP, and FTP are all based on DNS services

Working Mechanism

• Basic principle: Query layer by layer from bottom to top, and a caching mechanism can be set up (without accessing advanced DNS servers)
• Hierarchical, distributed

DNS message

• Message structure

---

P2P file distribution

• P2P (peer-to-peer): point-to-point transmission
• Web/Mail/DNS all adopt Cilent/Server structure, and P2P realizes equal protocol transmission.
• Architecture diagram: u represents the upload speed, d represents the download speed, and F represents the file size.

C/S architecture VS P2P architecture file transfer

• The file size is F, Us is the server upload speed, dmin is the minimum download speed, and N clients
• C/S architecture:
  • NF/Us indicates the time it takes for the server to upload a total of NF-sized data to N clients.
  • F/dmin represents the download time of the slowest client
  • Take the maximum of the two

• P2P architecture:
  • F/Us means that the server needs to upload at least one complete copy (which may be scattered among various clients)
  • F/dmin represents the download time of the slowest client
  • The last item indicates that the total file traffic is NF/total uplink speed

• As the number of users increases (under certain conditions), the advantages of P2P architecture become more and more obvious.

BitTorrent - P2P protocol

• Request - Scarcity Principle - Request the part with the fewest neighbors
• Response - Incentive Algorithm - Fast Upload and Fast Download

---

HTTP streaming and CDN

HTTP streaming

• Streaming video - the application plays the video as it is received and caches subsequent frames of the video.
• DASH (Dynamic Adaptive Streaming over HTTP): HTTP dynamic adaptation stream
  • In DASH, video is encoded into different versions at different bitrates. Customers dynamically play videos with appropriate bitrates based on bandwidth conditions.
  • The server sends notification files indicating that there are different versions

CDN

• CDN (Content Distribution Network): content distribution network
• Core: Redirecting using DNS
• Operation Mechanism
  • ①The user visits the webpage www.NetCinema.com
  • ②The user sends a DNS query request to the local DNS
  • ③The local DNS queries the authoritative server. The authoritative server detects that it is within the CDN range and does not return its own IP. Instead, it returns the DNS authoritative server for exclusive query within the CDN range.
  • ④Local DNS sends a request to the newly obtained DNS server
  • ⑤Local DNS returns the site server that has obtained the CDN content
  • ⑥The user and the exclusive content distribution server send a download request