[Jiwang] Detailed explanation of FTP protocol

1. Concept introduction

File Transfer Protocol, file transfer protocol , is an application layer protocol. Based on the client/server architecture, it implements file sharing services and uses TCP transmission.

2. Two working modes

FTP supports the following two working modes:

1. Active mode (POST) : The client sends port information to the FTP server, and the server actively connects to the port.
2. Passive mode (PASV) : The FTP server opens and sends port information to the client. The client connects to the port and the server passively accepts the port.
   Currently, most FTP servers use passive mode, but some scenarios require active mode.

(1) Active mode

Active mode workflow

1. The client randomly selects a port X and sends a request to the server port 21 (default port, which can be changed) through TCP to establish a command channel. Once the command channel is established, any subsequent operation instructions such as uploading and downloading will first be transmitted through the command channel. Of course, the process of establishing a command channel requires TCP's three-way handshake.

2. When operations such as uploading and downloading are involved, the client will randomly select port Y again, notify the server through the command channel, and request the establishment of a data channel. Different from the command channel, which only transmits instructions, the data channel is used for actual data transmission.

3. After receiving the notification, the server actively connects to the client's port Y through port 20 (default port). At this time, the data channel is officially established, and the client and server begin to transmit actual data.

   As you can see, the FTP protocol uses two links, the command channel for transmitting instructions and the data channel for transmitting data. It should be noted that the data channel will not be established at the beginning, and will only be temporarily established when data transmission is involved.

   Now the question is, it seems that the active mode can work very well, why is there a passive mode, and the passive mode has become mainstream?
   
   The answer is because in fact, there are often firewalls or routers between the client and the server. When the client requests to establish a data channel, the server will actively connect to port Y of the firewall instead of the client's port Y, and the data channel establishment fails. At this time, it appears that the client has connected to the server, but cannot transmit data.

   To avoid the embarrassment caused by firewalls, passive mode emerged.

(2) Passive mode

Passive mode workflow

1. Like the active mode, the client randomly selects a port X, sends a request to the server port 21 through TCP, and establishes a command channel.

2. When operations such as uploading and downloading are involved, the client notifies the server through the command channel and requests the establishment of a data channel. At this time, the client in passive mode does not randomly select a port to notify the server.

3. After the server receives the notification to establish the data channel, it will randomly select its own port Z and continue to inform the client through the command channel port Z.

4. The client actively connects to the server port Z through a random port Y of its own. At this time, the data channel is officially established.

   Passive mode solves the problem of the client hiding behind the firewall and being unable to establish a data channel, because the establishment of the data channel is initiated by the client and passively accepted by the server. However, when the server is hidden behind a firewall, the passive mode still cannot establish a data channel.
   
   Of course, there are ways to penetrate both firewalls and routers, so I won’t go into details here.

3. Reasons for the emergence of passive mode

In the history of FTP, there was an active mode, but why did the passive mode appear again?

A long time ago, there was no technology for sharing the Internet. Later, the following situation appeared:

Shared Internet access means that many computers share a public network IP to use the Internet. For example, a certain local area network shares the public network IP 210.33.25.1 for accessing the Internet. When an internal network user 192.168.0.1 accesses the FTP server on the external network, if active mode is used, 192.168.0.1 tells the FTP server that I need a certain file and that I have opened the x port. Because it is a shared Internet connection, when 192.168.0.1 exits the gateway, its own IP address is translated into the public IP address of 210.33.25.1. So the message received by the server is that 210.33.25.1 requires a certain file and has the x port opened. Then the FTP server will transfer data to the X port of 210.33.25.1, which of course will not succeed because the address that opens the x port is not 210.33.25.1.

In active mode, the two ports of FTP are relatively fixed. If the command port is x, the data port is x-1. By default, the command port is 21 and the data port is 20. Of course, both port numbers can be modified. It is very convenient to use the firewall in this way, as long as these two ports are opened. But if the client is sharing the Internet, normal FTP cannot be used.

Therefore, passive mode is definitely required.

Passive mode is more troublesome. By default, the command port is 21, but the data port is random.

4. Three authentication modes

1. Anonymous user mode: Anyone can log in to the FTP server directly without password verification. This mode is not safe and is generally only used to transfer unimportant public files. It is not recommended for use in production environments.
2. Local user mode: Verify login permissions through local users of the Linux system, which is more secure than anonymous user mode.
3. Virtual user mode: By verifying the login permissions of the virtual user, the virtual user can only access the FTP service provided by the Linux system, but cannot access other resources of the Linux system. This mode is more secure than the other two modes. If you have higher security requirements for server data, you can configure this mode yourself under the guidance of relevant professionals.

Extended knowledge point: port

A port can be simply understood as the outlet for communication between a computer and the outside world .

In network technology, port has two meanings: one is a physical port, also known as an interface, such as ADSL Modem, switches, and routers used to connect to other network devices, such as RJ-45 ports and SC ports. etc.; the second is the port on the software, which generally refers to the port in the TCP/IP protocol. The port range is from 0 to 65535, such as port 80 for browsing web services, port 21 for FTP services, etc.

The role of the software port: It is a symbol of unifying the application processes of the TCP/IP system and enables the application processes of computers running different operating systems to communicate with each other.

The connection service-oriented TCP protocol and the serviceless UDP protocol use 16bits port numbers (the port range is 0~65535) to represent and distinguish different applications in the network. The network layer protocol IP uses specific protocol numbers (TCP 6, UDP 17) To represent and distinguish transport layer protocols.

Three major categories of ports :

1. Well Known Ports: From 0 to 1023, they are tightly bound to some services. Usually communication on these ports indicates the protocol of a certain service. For example: Actual port 80 is always HTTP communication.
2. Registered Ports: from 1024 to 49151. They are loosely bound to some services. This means that many services are bound to these ports, and these ports are also used for many other purposes. For example: many systems handle dynamic ports starting around 1024.
3. Dynamic and/or Private Ports: from 49152 to 65535. In theory, these ports should not be assigned to these services. In practice, machines are usually assigned dynamic ports starting at 1024. But there are exceptions: SUN's RPC port starts at 32768.

5. The difference between FTPS and SFTP

When using FTP to transfer files, users need to obtain file transfer permission by providing credentials to the FTP server. Of course some public FTP servers may not require credentials to access their files, but the security of data transmission cannot be guaranteed. Any unencrypted data transmission on the public network is very dangerous, so in order to protect the security of transmitted data, the following two protocols are derived from FTP: FTPS and SFTP.

FTPS is divided into FTPS implicit SSL and FTPS explicit SSL

• FTPS Implicit SSL : This mode usually runs on port 990. In this mode all data exchange requires the establishment of an SSL session between the client and the server, and the server will reject any connection attempts that do not use SSL.

• FTPS Explicit SSL : The server under explicit SSL can support both FTP and FTPS sessions. Before starting a session, the client needs to establish an unencrypted connection with the FTP server and send the AUTH TLS or AUTH SSL command before sending the client credentials to request the server to switch the command channel to an SSL encrypted channel. After the channel is successfully established, the user The credentials are sent to the FTP server, thus ensuring that any commands during the session are automatically encrypted over the SSL channel.

  Quick summary : When implicit mode is enabled, the default port of FTP is changed to TCP/990, a secure connection is automatically established during the service period, and the client must also support the secure connection mode, that is, use SSL to connect. When explicit mode is enabled, the connection method and default port are the same as FTP, but the secure connection needs to be activated with an AUTH SSL/TLS type command before data can be transmitted normally.

SFTP

Secure File Transfer Protocol, secure file transfer protocol.
FTPS adds a layer of SSL to the FTP protocol. SFTP is a protocol based on the network protocol SSH (Secure Shell), which is completely different from FTPS. SFTP does not use separate command channels and data channels. Instead, both data and commands are transmitted in specially formatted data packets through a single connection.

SFTP provides two methods of verifying connections:

• As with FTP, only the user ID and password need to be verified when connecting. However, unlike FTP, these credentials are encrypted, which is SFTP's primary security advantage.
• In addition to passwords, it is possible to authenticate via SSH keys and connect via the SFTP protocol.

6. Frequently asked questions related to FTP

1. For servers, opening services and providing file uploads and downloads is inherently dangerous. If you really need to enable the FTP service, be sure to configure user permissions.
2. FTP itself is a clear text transmission, so the protocol itself is not secure. You can consider using SFTP that comes with SSH.
3. If you encounter the situation "You can connect to the FTP server, but cannot transfer files", you can troubleshoot the problem based on the FTP mode and the file permissions of the server.