[Turn] Fiddler capture Guide: Tools in conjunction with Proxifier

This switched: https://blog.csdn.net/china_jeffery/article/details/93000824

This article describes how to use Fiddler to crawl HTTP and HTTPS protocol packets, while also briefed how to combine Proxifier tools to deal with Filddler not crawl to the package.

A, HTTP basic packet capture

Fiddler's official website to download and install: https://www.telerik.com/fiddler

To capture the browser, it will not go, open the software glance, the paper stresses capture on ordinary Windows desktop applications, click on the icon in the lower left corner of two small, let Fiddler capture into the state, and the role of in [All Processes].

Fiddler capture of principle, in fact, is equivalent to the windows set up a HTTP / HTTPS proxy, similar to setting up a proxy in IE browser, such as [Internet 选项] — [连接] — [局域网设置] — [高级]the setting agent  [127.0.0.1:8888], Fiddler in 8888providing HTTP / HTTPS proxy service port.

Two, HTTPS packet capture

Ethereal for HTTPS, enable the HTTPS Fiddler capture function, or only see the contents of HTTP requests, HTTPS requests because of ciphertext.
In the Fiddler Click [Tools] — [Options] — [HTTPS]check the following settings:

Click on [Actions] — [Trust Root Certificate] the system to a trusted root certificate Fiddler, which is the key to decrypt HTTPS packet capture, then you can happily watch the clear content of HTTPS requests.

Third, why not catch some applications of HTTP (s) of the package?

When you turn [All Processes] get caught, we run third-party programs, you will find some HTTP / HTTPS packets can be caught, while others can not catch, how is this going? That is because this set the global proxy Fiddler way, only valid for the following situations:

• IE、Chrome等浏览器。
• 程序使用Windows提供的WinInet库进行HTTP/HTTPS通信。
• 程序内嵌WebBrowser，比较常用的是IE控件和CEF。

例如如果应用程序中使用的是libcurl库进行HTTP(s)请求，则fiddler就抓不到包了（解决方法下面会介绍），因为libcurl没有使用windows的WinInet库，而是自己实现了http(s)协议的封装。

如果有程序源码，可以在源码中设置Libcurl使用fiddler作为本地代理，如：

curl_easy_setopt(curl, CURLOPT_PROXY, "127.0.0.1:8888");

• 1

四、处理Fiddler无法抓到包的情况

4.1 设置代理

上面说到了，Fiddler抓包的原理是在本机的8888端口开启HTTP/HTTPS代理，任何通过Fiddler代理的HTTP/HTTPS通信内容都会被解析，那么只要能给目标程序设置HTTP/HTTPS代理，目标程序的HTTP(s)通讯内容就可以被Fiddler抓到。

仔细查看软件设置，其实有些第三方软件比如 [百度网盘] 本身是可以设置HTTP/HTTPS代理的，只要设置为Fiddler的代理端口即可截获它的HTTP/HTTPS通讯内容。

4.2 结合Proxifier

但是有的第三方软件就是没有代理功能，怎么办？既然应用程序不支持设置代理，我们就借助其它软件给它设置代理，比如：
Proxifier： https://www.proxifier.com

以 [网易有道词典] 为例：

在Proxifier中添加[127.0.0.1:8888] 这个Fiddler提供的HTTPS代理服务器

设置Proxifier规则，让 [网易有道词典] 通过代理访问网络。

有一点设置通常容易被忽略，就是在Proxifier中，设置[Profile] — [Name Resolution] — 勾选 [Resolve hostnames through proxy]，让域名解析的工作交给代理服务器，而不是在Proxifier上解析。默认情况下Proxifier自行解析域名，比如www.baidu.com解析为180.97.33.108，然后发请求给Fiddler：

CONNECT 180.97.33.108:443 HTTP/1.1

• 1

这样Fiddler并不知道它请求的是哪个域名，于是返回给客户端的伪造证书时，伪造的是为180.97.33.108颁发的证书，有的客户端会做校验，发现这个证书是颁发给180.97.33.108的，而不是颁发给www.baidu.com的，然后报错处理。
修改Proxifier设置后，把域名解析的工作交给代理服务器，Proxifier会直接向Fiddler发送请求：

CONNECT www.baidu.com:443 HTTP/1.1

• 1

这样Fiddler就知道客户端请求的是 www.baidu.com，从而返回客户端伪造的www.baidu.com证书，客户端不报错，Fiddler才能顺利抓包解密。

具体设置步骤如下：

4.3 解决Host为“Tunnel to”的问题

按照4.2中介绍的方法，抓取python请求https://www.baidu.com的包，此时在fiddler中会显示“Tunnel to”，无法获取到包的内容，如图：

从python的提示中，可以看到是SSL证书验证错误，所以python requests的请求也失败了。

Fiddler has been able to catch and decrypting HTTPS content of the package, because the use of a means of Fiddler middle attacks, which means to be able to successfully implement, there is a precondition, that the client trusted root certificate Fiddler provided through before we [Actions] — [Trust Root Certificate] let the system after Fiddler's trusted root certificate, as well as most browsers WinInet-based HTTP communication library program, the operating system will trust the root certificate we Fiddler added. However, if a third party library using other HTTP communication, such as libcurl, JAVA the URLConnectionlibrary, C # a System.Net.Http, Python's requests, which typically comes with a set of HTTP library SSL trusted root certificates, they do not use the operating system's root SSL certificate, but we will not use Fiddler root certificate added to the operating system, so they verified the wrong one.

In Python, for example, this can be confirmed in the document requests:
https://2.python-requests.org/en/master/user/advanced/#ca-certificates

Requests bundled a set of root CAs that it trusted, sourced from the Mozilla trust store. The certificates were only updated once for each Requests version.

• 1

Details are given below two methods to solve this problem:

1. Disable certificate validation when requested.
2. When requested, specify SignSis.

Disable certificate validation request 4.3.1

With python as an example:

import requests
requests.get("https://www.baidu.net", verify = False)

• 1
• 2

Trust your certificate request 4.3.2

Access http://127.0.0.1:8888/root certificate download Fiddler, as:

then using third-party tools into corresponding libraries certificate types may be supported, where the python to requestsan example, using the openssl tool to convert it into a supported .pemformat:

openssl x509 -inform der -in FiddlerRoot.cer -out fiddler.pem

• 1

import requests
requests.get("https://www.baidu.net", verify = "./fiddler.pem") 

• 1
• 2