1. Set Headers
Some sites will not agree with the above procedure for direct access, to identify if there is a problem, then the site was not responding, so in order to work full simulation browser, we need to set some of the attributes of Headers.
Open our browser, debugger browser F12, the emergence of a new interface, in essence, this page contains many elements, but the content is not a one-time loading is complete, in essence, is the implementation of a good many requests, general is the first requests an HTML file, and then load the JS, CSS, etc., after many requests, the web page skeleton and muscles of the whole, the effect of the whole page will come out.
Split these requests, we look at a first request, as you can see, there is a Request URL, as well as headers, the following is the response.
Which, is to ask the identity of the agent, if there is no written request status, the server does not respond, it is possible to set the agent in the headers.