1: Overview of packet capture analysis
Packet capture analysis is the operation of capturing the data packets sent and received by network transmission. When doing a crawler, the data is not necessarily in the HTML source code, but may be hidden in some URLs, so we need to capture some data for Capture packets, analyze the URL where the corresponding data is hidden, then analyze the rules and crawl.
Two: Use the fiddler tool for packet capture analysis
(1) Set up fiddler:
This window is used to browse the captured URLs:
When you select a URL, specific information is displayed on the right:
The black text box below can be used to enter some fiddler commands.
(2) The use of the browser -- Firefox
First configure the Firefox browser:
a: open the browser
b: Click on the right
c: Select "Options"
d: Select "Advanced"
e: Click "Network"
f: Click "Settings"
Fiddler needs to use a proxy to capture packets, because fiddler uses a proxy server to work. For example, if we want to access a server in the Internet now, if we directly access the past in this way, then our data packets will be transmitted directly in this way, and we cannot see it. Then we can use fiddler as a proxy server, and then first of all whether all our data needs to be sent to the server in our Internet through fiddler, and then the corresponding information of the server in the Internet is also passed through fiddler and then passed to us locally, At this time, it is explained that at this time, fiddler, as an intermediate server, will have all the data packets. With these data packets, we will be very easy to do, and we can see all the data in fiddler. Packet, this is the principle of packet capture, it acts as a proxy server to intercept some information.
g: Click "Manually configure proxy" and fill in the following information: 127.0.0.1 is the default port 8888 of the local server
h: Click "OK"
So now our operations on the Firefox browser can be obtained in fiddler.
Three: Instructions in fiddler
Enter the clear command in the black box to clear it.