All the following content comes from "Wireshark network analysis is so simple" (by Lin Peiman)
Wireshark network analysis is as simple as that
network analysis
>> Why is the IP of server A being pinged, but B is querying the MAC address of the default gateway?
Because B calculates that A belongs to different subnets according to its own subnet mask, cross-subnet communication requires the forwarding of the default gateway. To communicate with the default gateway, you need to obtain its MAC address.
>> Why did server B on a different network segment reply directly to server A's own MAC address?
Because the subnet is not considered when performing ARP reply, even if the ARP request comes from another subnet IP, it will still reply.
>> Why does the MAC address start with Vmware displayed on Wireshark?
Because the first 3 bytes of the MAC address indicate the manufacturer, here it indicates Vmware.
>> When A pings B, how should the network packet go?
If you have an IP in the same subnet as the other party, you can send packets directly from this IP to the other party; if you don’t have an IP in the same subnet as the other party, you will use the default gateway.
>> A is looking up B's MAC address through ARP broadcast, trying to bypass the default gateway to communicate directly with B. What does this mean?
This shows that there is a route on A that matches the same network segment as B, prompting A to directly communicate with B through eth1. Since the VLANs assigned to different subnets are also different, these ARP requests cannot reach B at all.
>> The trap of NAT
There is NAT (Network Address Translation) between the two networks. When the request sent by the client passes through the NAT device, the Source IP is changed, so the original release policy may become invalid.
Capture skills
>> Pack size
(1) Generally, the maximum length of each packet that can be captured (called "frame" is more accurate) is 1514 bytes, and after enabling Jumbo Frame (jumbo frame), it can reach more than 9000 bytes, and most of the time we only Requires IP header or TCP header is enough for analysis.
(2) Generally set a relatively large number: 80 bytes, which means that each packet only captures the first 80 bytes. In this way, the information of TCP layer, network layer and data link layer can be included
(3) You can use the "-s" parameter to achieve the same effect when using the tcpdump command to capture packets.
For example, the following command captures only the first 80 bytes of each packet on eth0, and saves the result to the /tmp/tcpdump.cap file. [root@server_1 /]# tcpdump -i eth0 -s 80 -w /tmp/tcpdump.cap
>> packet capture flag
In addition to grabbing small bags, it is best to mark each operation.
(1) ping-n1-l1
(2) Operation step 1
(3) ping-n1-l2
(4) Operation step 2
Check the "Data (1byte)" at the bottom of the packet, the number of bytes indicates which step
>> Server and client timing issues
(1) Adjust the time of Wireshark to the same format as the server: Click View–>Time Display Format–>Date and Time of Day of Wireshark to realize this setting; (2) If you capture packets on servers in other time
zones , and then download it to your computer for analysis. It is best to set the time zone of your computer to be the same as the server that captures the packet. This way, the time shown by Wireshark matches the time of the logs on the server.
>> filter
(1) If it is known that there is a problem with a certain protocol, you can filter it by the protocol name. Be sure to take into account dependencies between protocols when using protocol filtering. (For example, if the NFS share mount fails, the problem may occur in the mount protocol used when mounting, or in the portmap protocol before mount. In this case, you need to use "portmap||mount" to filter.) (
2 ) IP address plus port number is the most commonly used filtering method. In addition to manually entering filter expressions such as ip.addreq<IP address>&&tcp.porteq<port number>, Wireshark also provides a faster way: right-click the packet of interest and select Follow TCP/UDP Stream (select TCP Or UDP depends on the transport layer protocol) can be automatically filtered.
(3) Right-click the content of interest on Wireshark, and then select Prepare a Filter–>Selected, and the filter expression will be automatically generated in the Filter box. When there are complex requirements, you can also choose And, Or and other options to generate a combined filter expression. (If you choose Apply as Filter–>Selected instead of Prepare a Filter after right-clicking, the filter expression will be automatically executed after it is generated.) (
4) Enter "error" in Filter to search. Many application layer errors can rely on this method to lock the problem package.
>> analysis
(1) Click Analyze–>Expert Info Composite of Wireshark, and you can see different levels of prompt information under different labels. For example, retransmission statistics, connection establishment and reset statistics, and so on. We often need to rely on this feature when analyzing network performance and connection problems. (There is no error reported from Error and Warning, which means that there is no problem with the network. If there are retransmissions, out-of-order phenomena, etc., you should be able to see them in this window.) (2) Click Statistics–>Service
Response Time, and then select the protocol name, you can get the statistical table of response time. We often need this statistic when measuring server performance.
(3) Click Statistics–>TCP Stream Graph to generate several types of statistical graphs.
(4) Click Statistics–>Summary, you can see some statistical information, such as average traffic (Avg.MBit/sec), etc., which helps us to speculate on the load status. For example, the network packet is only 1.594Mbit/s, indicating that the traffic is very low.
>> tshark
The command line version of Wireshark is more powerful. Most of the output analysis text can be directly written into the analysis report, but Wireshark cannot generate such a report.
TCP protocol
>> Protocol description
TCP is connection based while UDP does not require a connection. They each support some application layer protocols, but some protocols are supported by both, such as DNS.
>> MTU
The network has a limit on the size of the packet, and its maximum value is called MTU, or "Maximum Transmission Unit".
The MTU of most networks is 1500 bytes, but some networks enable jumbo frames (Jumbo Frame), which can reach 9000 bytes.
An 8192-byte packet will have no problem entering a jumbo frame network, but it will be discarded or fragmented when it reaches a 1500-byte network.
Being dropped means that the transmission failed completely, because retransmitted packets will be dropped again.
And being divided means that the transmission efficiency is reduced.
>> MSS
When the TCP connection is established (three-way handshake), both parties will tell each other their MSS (Maximum Segment Size). The MSS plus the length of the TCP header and the IP header gives the MTU.
The size of the sent packet is determined by the party with the smaller MTU.
>> Seq&&Ack
Click on the "+" sign before the TCP layer, we can see a series of information such as Seq number and Ack number, which are used for network packet sorting, retransmission, flow control, etc.
(1) Seq: Indicates the sequence of the data segment. TCP provides ordered transmission, so each data segment must be marked with a sequence number. When the receiver receives out-of-sequence packets, it can reorder them with this sequence number. The size of a Seq number is based on the addition of the Seq number and length of the previous data segment.
(2) Len: The length of the data segment, note that this length does not include the TCP header.
(3) Ack: Acknowledgment number, the receiver confirms to the sender which bytes have been received. For example, A sends a data segment of "Seq:x Len:y" to B, and the confirmation number of B's reply is x+y, which means that it has received all the bytes before x+y. (TCP acknowledgments are cumulative.)
>> Logo description
(1) SYN: A packet carrying this flag indicates that a connection request is being initiated. Because the connection is bidirectional, both parties must send a SYN when the connection is established.
(2) FIN: A packet carrying this flag indicates that it is requesting to terminate the connection. Because the connection is bidirectional, when a connection is completely closed, both parties must send a FIN.
(3) RST: Used to reset a chaotic connection, or reject an invalid request.
>> Relative Sequence Number
In fact, the Seq number does not start from 0 during the handshake. The reason we see Seq=0 on Wireshark is because Wireshark has enabled Relative Sequence Number. If you want to turn off this function, you can set it in Edit–>Preferences–>protocols–>TCP.
>> netstat
If you encounter disconnection problems during work, you can use the netstat command to troubleshoot.
>> delayed acknowledgment
This is actually one of TCP's strategies for handling interactive scenarios. The principle of this strategy is this: if there is no data to send to the other party temporarily after receiving a packet, then delay for a period of time (the default is 200 milliseconds on Windows) before confirming. If there happens to be data to be sent during this time, the confirmation information and data can be sent in one packet.
>> Nagle Algorithm
Principle: Before the sent data is confirmed, if there is another small data generated, then collect the small data, fill up an MSS or send it after receiving the confirmation.
TCP window
>> TCP send window
There are also some limitations in reality: the receiver's cache (receiving window) may not be able to accept so much data at once; the bandwidth of the network may not be large enough, and sending too much in one go will lead to packet loss accidents.
Therefore, the sender needs to know which of the two limiting factors, the receiver's receiving window and the network, is stricter, and then send as many packets as possible within its limit. The amount of data that can be sent in one breath is the legendary TCP sending window.
(1) The interval between TCP timeout and retransmission is too long. Setting a smaller time can reduce the impact of retransmission on performance.
For example, the network is frequently congested, and most of the congestion points are above 32KB. If you limit the sending window to 32KB, you can avoid hitting the congestion point.
(2) In a real environment, the sending window can often reach dozens of MSS.
>> window size
(1) The TCP layer of each packet contains window size (that is, win=) information. This is not a sending window, but a declaration of its own receiving window to the other party.
(2) If the speed of data processing by the receiver cannot keep up with the speed of receiving data, the buffer will be full, resulting in a receiving window of 0.
(3) The sending window determines how many bytes can be sent in one go, and the MSS determines how many packets these bytes will be sent out.
>> Window Scale
(1) The function of Window Scale is to declare a Shift count to the other party. We use it as an index of 2, and then multiply it by the receiving window defined in the TCP header to get the real TCP receiving window.
(2) It should be noted that Wireshark calculates this result based on Shift count. If the three-way handshake is not captured when capturing packets, Wireshark does not know how to calculate, so we sometimes see some extremely small receiving windows inexplicably value. Sometimes the firewall cannot recognize Window Scale, so the other party cannot obtain the Shift count, which eventually leads to serious performance problems.
>> Other window algorithms
(1) The Westwood algorithm (proposed by Mr. Saverio)
calculates how many packets have been delivered to the receiver based on the Ack responded by the receiver, so as to more accurately estimate the bandwidth when congestion occurs, and finally determine the new congestion window based on the bandwidth .
(2) FlightSize
The introduction of FlightSize is just to get a safe critical window value, instead of pursuing an ideal window like Westwood+.
Its theoretical basis is not complicated: when the network condition is good, the RTT (round-trip time) of the data packet is relatively stable, and the congestion window can be increased at this time; when the network starts to be busy, the data packet starts to queue, and the RTT will change Large, then the congestion window needs to be reduced
TCP retransmission
>> congestion window
This strategy is to maintain a virtual congestion window at the sender, and use various algorithms to make it as close as possible to the real congestion point. The limitation of the network on the sending window is realized through the congestion window.
>> RTO
The time between sending out the original packet and retransmitting it is called RTO.
>>How to view retransmission
(1) Click the Analyze–>Expert Info Composite menu, and you can see them on the Notes tab. Click the + sign to see which packages have been retransmitted.
(2) Retransmission has a great impact on performance, and even a ratio of 0.5% will greatly degrade performance.
>> timeout retransmission
Timeout retransmissions have a serious impact on transmission performance.
One of the reasons is that the data cannot be transmitted during the RTO stage, which is equivalent to wasting a period of time; the
second reason is that the congestion window decreases sharply, which is equivalent to much slower transmission.
>> fast retransmission
When subsequent packets arrive at the receiver, the receiver will find that its Seq number is larger than expected, so it will Ack the expected Seq number every time it receives a packet, so as to remind the sender to retransmit.
When the sender receives 3 or more duplicate acknowledgments (Dup Ack), it realizes that the corresponding packet has been lost and immediately retransmits it. This process is called fast retransmission. It is called fast because it does not need to wait for a period of time like a timeout retransmission.
Because the distance of disordered order will not be too different, for example, the No. 2 package may run behind the No. 4 package, but it is unlikely to run behind the No. 6 package, so limiting it to 3 or more can be avoided to a large extent Trigger fast retransmission due to out-of-order.
>> quick recovery
The critical window value should be set to 1/2 of the amount of data that has not been confirmed when congestion occurs (but not less than 2 MSS). Then set the congestion window to the critical window value plus 3 MSS, and remain in the congestion avoidance phase. This process is called fast recovery.
>> Summary
(1) When there is no congestion, the larger the sending window, the better the performance. Therefore, under the condition that the bandwidth is not limited, the receiving window should be increased as much as possible, such as enabling the Scale Option.
(2) If congestion occurs frequently, then limiting the sending window can improve performance, because even one ten-thousandth retransmission has a great impact on performance.
(3) Timeout retransmission has the greatest impact on performance, because it has not transmitted any data for a period of time (RTO), and the congestion window will be set to 1 MSS, so try to avoid timeout retransmission.
(4) Fast retransmission has less impact on performance, because it has no waiting time, and the reduction in congestion window is not so large.
(5) SACK and NewReno are beneficial to improve retransmission efficiency and improve transmission performance. (Whether to enable SACK is negotiated during the TCP three-way handshake. As long as one of the two parties does not send "SACK_PERM=1", SACK will not be used after the connection is established.) (6) Packet loss for extremely small
files The impact is more severe than for large files. Because the number of packets needed to read and write a small file is very small, when a packet is lost, it is often not enough to make up 3 Dup Acks, so it can only wait for the timeout to retransmit. However, large files are more likely to trigger fast retransmission.
UDP protocol
>> Protocol description
(1) In the UDP protocol header, there are only a small amount of information such as the port number, packet length, and check code, which is a total of 8 bytes.
(2) UDP does not care about the size of the MTU of both parties like TCP. After it gets the data from the application layer, it directly adds the UDP header and hands it over to the next layer.
So what to do when the MTU is exceeded?
In this case, the sender's network layer is responsible for fragmentation, and the receiver assembles the fragments after receiving them. This process consumes resources and reduces performance.
(3) UDP has no retransmission mechanism, so packet loss is handled by the application layer.
>> Weaknesses of Fragmentation Mechanism
The reason why the receiver knows when to assemble the fragments is because there is a "More fragments" flag in each packet. 1 means that there are fragments in the future, and 0 means that this is the last fragment and can be assembled. If the hacker continues to send UDP packets with flag 1 quickly, the receiver will not be able to assemble these packets, and the memory may be exhausted.
NFS protocol
>> Protocol description
(1) The NFS server provides two shared directories /code and /document, which are respectively mounted on the local directories of multiple clients. When users read and write files in these local directories, they are actually reading and writing on the NFS server unconsciously.
(2) If you see that the portmap request has not been answered, you can consider the interception of port 111 by the firewall; if you find that the mount request is rejected by the server, you should check the access control of the shared directory.
(3) The access control of NFS to the client is realized through the IP address. When creating a shared directory, you can specify which IPs are allowed to read and write, which IPs are only allowed to read, and which IPs are not even allowed to mount
(4) The NFS protocol only recognizes the UID but not the user name. It is recommended that the relationship between the user name and UID be consistent on each client.
(5) Application layer: Since NFS is an RPC-based protocol, Wireshark divides it into two lines of NFS and RPC to display.
>> async and sync methods
(1) If the sync parameter is used during mount, the client will first send a WRITE Call, and then send the next Call after receiving the Reply, that is to say, WRITE Call and WRITE Reply appear alternately.
In addition, is there any way to see whether a write operation is async or sync in the package?
The answer is the "UNSTABLE" and "FILE_SYNC" flags on each WRITE Call, the former means async and the latter means sync.
(2) The amount of data written by each WRITE Call is also an important factor affecting the write performance. We can use the wsize parameter to specify how much data should be written each time when mounting. However, after the sync parameter is enabled on some clients, no matter how much wsize is defined, it will be forced to 4KB, resulting in very poor write performance.
Then why do some people still use the sync method?
The answer is that some special applications require the server to wait until the file is saved before replying with a WRITE Reply after receiving a sync write request. The sync operation meets this requirement. From this we can also deduce that COMMIT is not necessary for sync write operations.
(3) From the FILE_SYNC in the Write Call, we can know that although the sync parameter is not specified when mounting, noac forces the write operation into a sync mode, and the performance will naturally decrease.
CIFS protocol
>> Protocol description
(1) What sharing protocol is generally used on Windows? It is the SMB protocol maintained by Microsoft, also known as Common Internet File System (CIFS).
(2) There are three versions of the CIFS protocol: SMB, SMB2, and SMB3. Currently, SMB and SMB2 are more common.
(3) CIFS can only be based on TCP.
(4) The port number on the CIFS server is 445.
>> CIFS connection process
(1) The first CIFS operation is Negotiate (negotiation).
(2) After the version is negotiated, the CIFS Session can be established.
The main task of Session Setup is authentication, and the commonly used methods are Kerberos and NTLM.
(3) Click to open the two Tree Connect packages, the most valuable information is the Tree ID returned by the server. From then on, the client can use this ID to access the subdirectories and subfiles of the /dest share.
(4) Create is a very important operation in CIFS. Whether it is creating a new file, opening a directory, or reading and writing a file, Create is required.
(5) CIFS uses Oplock (opportunity lock) to solve this problem. Oplock has three forms: Exclusive, Batch and Level2. Exclusive allows read and write caching, Batch allows caching of all operations, and Level 2 only allows read caching.
>> Breakthrough of SMB3
(1) During the copy process of SMB3, we only transmit some instructions on the network, but the file content does not appear on the network, because the copy data is completely completed by the server itself.
(2) Another unprecedented improvement of SMB3 is the load balancing at the CIFS layer. Unlike other CIFS versions, one SMB3Session can be based on multiple TCP connections.
DNS protocol
>> Protocol description
Use DNS to resolve to an IP address, and then find the server through the IP address.
For example, if you turn on the company computer and log in to the operating system with a domain account, you will rely on DNS to find the Domain Controller to verify your identity.
>> Recursive query and iterative query
(1) Recursion: The client only sends a request once, asking the other party to give the final result.
(2) Iteration: The client sends a request. If the other party is not authorized to answer, it will return a list of other name servers that can answer the query, and the client will send requests to the returned list until it finds the name server responsible for the query. The nameservers for the domain name, from which the final result is obtained.
Adding the "+trace" parameter to the dig command can force the client to use an iterative query.
>> Record description
PTR record: Contrary to the function of A record, it can resolve from IP address to domain name.
SRV records: Windows domain administrators are particularly concerned with SRV records because they point to resources in the domain.
CNAME record: also known as Alias record, which means alias.
>> Round-robin mode
This feature can be widely used in load balancing. For example, if a website has 10 web servers, the administrator can create 10 records with the same name in DNS to point to the IPs of these servers. Since the order of results found by different clients is different, and the first IP in the results is generally selected, a large number of clients will be evenly distributed to 10 web servers.
>> security
(1) It is very dangerous for the DNS server to be maliciously modified to point to phishing websites, even if it is equipped with a regular DNS server, it is possible to be tricked. For example, a regular DNS server will become untrustworthy after encountering buffer poisoning;
(2) In addition to being used for deception, DNS can also be used as an offensive weapon. The famous DNS amplification attack is a headache. For example, when "dig ANY isc.org" is executed, DNS will reply with a large packet. In this packet, a source address to be attacked will be forged, and the address will inexplicably receive a reply of 3111 bytes from the DNS server. Taking advantage of this amplification effect, hackers can bring down a large website as long as they control a small number of computers.
FTP protocol
>> Protocol description
(1) FTP is often used to spread files, especially free software; another wide application is to collect logs. We can automatically send logs back to the manufacturer through FTP after the server fails.
(2) Control port 21 of the FTP server.
(3) FTP is transmitted in plain text
>> Control connection and data connection
The client connects to port 21 of the FTP server only to transmit control information, which we call "control connection".
When data needs to be transmitted, a TCP connection is re-established, which we call "data connection".
Not only when downloading files, but even when executing the ls command to list files, a new data connection needs to be created.
>>Passive mode and active mode
(1) If you have configured a firewall for FTP, you will also find that this method has brought about a more serious problem-since the three-way handshake of the data connection is initiated by the server (we call it the active mode), if the client If the firewall at the end blocks the connection request, the transmission will fail. At this time, you can try the passive mode of FTP.
(2) Passive mode means passive mode.
(3) The ftp command that comes with Windows seems to only support active mode.
HTTP protocol
>> Protocol description
Because massive files are not suitable for the traditional directory structure, cloud storage generally uses object storage—the client does not use its path and file name when accessing files, but its object ID.
>> HTTPS parsing
(1) The package captured when using HTTPS search, note that the server port is 443, and the keywords are also encrypted into "Encrypted Application Data".
(2) Export the key and import it into wireshark: Click Edit–>Preferences–>Protocols–>SSL–>RSA keys list of Wireshark. Then fill in the format of IP Address, Port, Protocol, and Private Key, and the content in https can be seen at this time.
(The key used for decoding can only be exported on the server side)
Kerberos protocol
>> Protocol description
(1) The identity authentication protocol is also called Kerberos, which has a very wide range of applications. For example, it will be used in the identity authentication of the Windows domain environment.
(2) The method adopted by Kerberos is to introduce an authoritative third party to be responsible for identity authentication. This third party is called KDC, and it knows the passwords of all accounts and resources in the domain.
(3) Send the generated string "{time stamp}Kclt", the information of account A, and a random string to KDC. This constitutes the Kerberos authentication request AS_REQ.
>> Replay attack
Hackers may intercept the string "{time stamp} Kclt" on the network, and then pretend to be account A to cheat authentication. This approach is called a replay attack. The camouflage process of a replay attack takes a while, so the KDC compares the decrypted timestamp with the current time, and if the difference is too large, it can be judged to be a replay attack. If you use time-independent characters to encrypt, you can't avoid replay attacks, which is why we must synchronize the time of all machines in the domain.