Recently, the product has put forward a new requirement, which needs to detect port scanning behavior. It is mentioned that port scanning must not bypass the king of port scanning-Nmap. Therefore, in addition to the detection scheme that establishes a statistical model based on traffic quintuple statistics, identifying the traffic characteristics of Nmap becomes the key.
Nmap is an information collection tool commonly used by network security personnel. It is mainly used to detect hosts, scan ports and services, and has a variety of built-in scanning methods. Each method works differently, and its data packets and communication characteristics are also different.
Port scanning usually refers to the use of TCP, UDP, etc. in the information collection stage to detect the type of operating system and open services, and prepare for further attacks. Generally, common behaviors that affect network security, such as worms and network attacks, start with scanning. So we first analyze and extract features of the most common scanning methods through network traffic analysis.
Several common port scanning methods in N map
-sS (TCP SYN scan)
SYN scanning is the default and the most popular scanning option for good reasons. It executes very quickly, and can scan thousands of ports per second on a fast network without intrusive firewalls. SYN scanning is relatively unassuming and not easy to notice, because it does not complete a complete TCP three-way handshake connection, and can deal with any compatible TCP protocol stack, with fast scanning speed and high efficiency. It can also clearly and reliably distinguish the ports open (open), closed (closed), and filtered (filtered) status.
SYN scanning is often called semi-open scanning because it does not open a full TCP connection. It sends a SYN message, but it does not generate any conversation, and the target host hardly records the connection in the system log. SYN/ACK means the port is listening (open), and RST (reset) means there is no listener. If there is no response after several retransmissions, the port is marked as filtered.
As shown in the figure below, SYN scans to the TCP sequence diagram of the closed port, and tries to connect to the scanned IP with a fixed port, and most of the sessions have the same characteristics.
-sT (TCP connect() scan)
When SYN scan is not available, TCP Connect() scan is the default TCP scan. This is the case when the user does not have permission to send original messages or scan IPv6 networks. Nmap requires the operating system to establish a connection with the target machine and port by creating the connect() system call, instead of directly sending the original message like other scan types. This is the same high-level system call used by web browsers, P2P clients, and most other network applications to establish connections.
When Nmap connects and then closes the connection without sending data, many common UNIX systems will leave a record in syslog, sometimes an encrypted error message. Therefore, TCP connect() scanning will record a large number of connection requests and error messages in the log of the target host, which is very easy to be detected.
As shown in the figure below, packet capture analysis shows that TCP is fully connected and scanned, and it directly performs TCP three-way handshake with the scanned port. If the port is open, the three-way handshake is successful; if the port is not open, the host returns a RST ACK packet.
-sU (UDP scan)
Many popular services on the Internet run on the TCP protocol. Of course, there are some UDP services. DNS, SNMP, and DHCP (registered ports are 53, 161/162, and 67/68) are the most common three. UDP scanning, usually combined with ICMP. When sending a UDP packet without any data to the target host, if the return message is "ICMP port unreachable" (type 3, code 3), it means that the target port is closed, but the host is alive ; If a service responds to a UDP message, it indicates that the port is open.
Since UDP is connectionless, it will face response detection problems. The judgment of whether the port is open or filtered during detection will cause Nmap to time out and then detect again to prevent the detection frame or response from being lost, which will increase the detection time. Therefore, UDP scanning will find obvious slow speed problems. Actual applications usually require concurrent scanning or scanning the main port first.
The characteristic performance of UDP scanning in traffic analysis: There are a large number of UDP sessions that do not carry any data, and there will be a large number of ICMP port unreachable packets, as shown in the following figure.
-sN (Null scan)
Null scan is to send a data packet without any flag bit to the TCP port. At least one flag bit must be set in normal communication. According to RFC 793, the host sends a TCP packet without any flag bit. If the corresponding port of the target host is closed, it will return a RST packet. If there is no response, it means that the port is open.
So Null scan, if the TCP port is closed, it will respond with a RST packet, if it is open, there will be no response. As shown in the figure below, a large number of packets without any flags (tcp flags are all 0).
-sF (FIN scan)
Sometimes the firewall filters TCP SYN packets in order to prevent network scanning and detection. At this time, TCP FIN scanning can be used. This method is the same as the principle of Null scanning, sending TCP FIN packets to the port to be tested. If an RST packet is returned, it means that the port is closed; if there is no return, it means that the port is open.
Therefore, the performance of initiating a FIN scan on the flow is that there are a large number of FIN packets with the same characteristics, as shown in the flow sequence diagram below. Similarly, ACK scanning is that there are a large number of ACK packets. Because Windows systems and many BSD-based systems do not fully comply with the RFC 793 protocol and discard FIN+ACK probe packets, FIN/ACK scanning is not suitable for scanning some servers on Windows and Cisco.
N map detection bypasses IDS detection
We have analyzed some of the most common port scanning methods above. In actual applications, in order to collect information better and more comprehensively, there are often multiple combinations of custom scanning, such as using various combinations of SYN, ACK, UDP, etc. to send packets, or At the same time, combine some other parameters such as -sV (version detection) and -O (system detection) to obtain more information about the target system. Network security is always accompanied by the process of offensive and defensive confrontation. Network administrators will use firewalls or IDS/IPS to detect or prevent malicious scanning and network detection. Therefore, mainstream IDS products have added signature databases to Nmap traffic. The point of view is to find a way to bypass firewall rules or intrusion detection system rules.
We take some typical detection rules for port scanning in the Emerging Threats rules intrusion detection rules as examples to analyze the detection points that may be bypassed in the actual attack and defense process.
As shown in the figure, the ET rule set detects the rules of Nmap SYN scanning. The SYN message is mainly determined by checking the flags of the data packet. The window keyword is used to check the specific TCP window size. During the analysis, we did find that the default value of the Windows window value of Nmap's default sS parameter for SYN scanning is 1024. When the actual real business connects to a specific port, you can see that the window value is obviously different.
After understanding this detection mechanism, it is actually very easy to bypass this detection. Modify the Nmap source code tcpip.cc and adjust the tcp window size, while avoiding the window size such as 3072, 4096, etc. in the published rules.
In addition, if the -O parameter is added when Nmap is doing port scanning, it will send tcp, udp, icmp protocols, and scan the port while detecting operating system information. At present, IDS detection is generally based on the'C' character filled in the UDP data area String, as shown in the rule below.
The actual Nmap scanning packet capture analysis traffic is as follows,
Therefore, the source code osscan2.cc is modified as shown below. If 0X43 is modified to other values, the UDP and ICMP filling values will be changed accordingly, bypassing the IDS default detection rules.
In addition, if there is an HTTP protocol in the Nmap scanning detection service process, the User-Agent will also have obvious characteristics, which is also the most frequently encountered alarm content in daily security equipment monitoring.
Similarly, after studying the main source code architecture of Nmap, you can also find the place to configure UA information in nselib/http.lua, and modify it to normal UA information to avoid the detection of security devices based on UA characteristics.
Due to space limitations, only the most common scanning methods are selected for analysis and some mainstream detection features in the ET rule set are provided for your reference, mainly to provide ideas for offensive and defensive confrontation. We can find that detection based on single and static traffic characteristics is increasingly difficult to combat advanced attacks and unknown threats. This is also the direction in which various traffic-based security detection devices continue to optimize and evolve, based on abnormal network behavior, correlation analysis, and threats. Intelligence blessing and various threat modeling have basically become necessary functions and selling points of current intrusion detection systems based on traffic analysis.