Unable to catch HTTPS package
"Windows Packet Capture Guide ①: Proxifier+Fiddler Forced Capture of Third-Party Programs"
Looking back on the previous article, we used Proxifier to direct all TCP traffic of third-party programs to Fiddler's HTTPS proxy, so Fiddler can parse the communication content of the HTTP/HTTPS protocol, but in the actual use process, we will find that the HTTP request parsing is not Problem, but some third-party programs cannot parse their HTTPS communication content, which means that the Tunnel to 443 port can be seen in Fiddler, but there is no following.
Analyze the reasons
Know yourself and the enemy, and you will not be imperiled in a hundred battles. To find out what's going on, the best way is to write a program yourself, make an HTTPS request, and then use this method to grab your own package to see what went wrong. So test it with the simplest Python code:
import requests
requests.get("https://www.csdn.net")
Then use Proxifier+Fiddler to force it to capture the package, and find that the Python program threw an exception,
Traceback (most recent call last):
File “C:\Python36\lib\site-packages\urllib3\contrib\pyopenssl.py”, line 453, in wrap_socket
cnx.do_handshake()
File “C:\Python36\lib\site-packages\OpenSSL\SSL.py”, line 1907, in do_handshake
self._raise_ssl_error(self._ssl, result)
File “C:\Python36\lib\site-packages\OpenSSL\SSL.py”, line 1639, in _raise_ssl_error
_raise_current_error()
File “C:\Python36\lib\site-packages\OpenSSL_util.py”, line 54, in exception_from_error_queue
raise exception_type(errors)
OpenSSL.SSL.Error: [(‘SSL routines’, ‘tls_process_server_certificate’, ‘certificate verify failed’)]
There is an error in the SSL certificate verification, so it is understandable that other third-party programs should also throw exceptions, and they cannot connect to the Internet normally and cannot work normally. As mentioned earlier, the reason why Fiddler can capture and decrypt the content of HTTPS packets is because Fiddler uses a man-in-the-middle attack method. To be able to implement this method successfully, there is a prerequisite that the client trusts the root certificate provided by Fiddler. After we let the system trust Fiddler's root certificate through [Actions] — [Trust Root Certificate], most browsers and programs for HTTP communication based on the WinInet library will trust the Fiddler root certificate we added in the operating system. But if third-party programs use other HTTP libraries for communication, such as VC programs using libcurl, JAVA programs using URLConnection in JDK or third-party OkHttp, C# using System.Net.Http, Python using requests, these HTTP libraries generally come with a A set of trusted SSL root certificates, they do not use the SSL root certificate that comes with the operating system, let alone the Fiddler root certificate we added to the operating system, so the verification goes wrong.
Using Python as an example, this can be confirmed in the requests documentation:
https://2.python-requests.org/en/master/user/advanced/#ca-certificates
Requests bundled a set of root CAs that it trusted, sourced from the Mozilla trust store. The certificates were only updated once for each Requests version.
Solution
Then there are two solutions, one is to let the HTTP client disable certificate verification:
import requests
requests.get("https://www.csdn.net", verify = False)
One is to let the HTTP client trust the Fiddler root certificate.
Visit http://127.0.0.1:8888 to download the Fiddler root certificate, and use openssl to convert it into a format supported by Python requests:
openssl x509 -inform der -in FiddlerRoot.cer -out fiddler.pem
Make Python's HTTP client trust it:
import requests
requests.get("https://www.csdn.net", verify = "./fiddler.pem")
Then Fiddler can successfully capture the HTTPS package of the program and decrypt it
However, do not have eggs?
So far, by writing our own programs and capturing our own packages, we have figured out why some HTTPS packets cannot be captured or decrypted when we capture third-party programs. We have also found two solutions, one is Is to let the target program disable SSL verification, one is to let the target program trust the Fiddler root certificate.
At this point, some friends are going to complain. The target program I want to capture is not written by me, and I don't have its code. How can I modify it to disable SSL verification or trust the Fiddler root certificate?
have a solution
When analyzing the target program, we roughly figure out what HTTP Client library the target program uses by decompiling the target program. Generally, few programs implement an HTTP Client by themselves, and most of them use the language standard library or the first one. Three-party open source HTTP Client library. For example, VC generally uses WinInet or WinHttp, C# generally uses System.Net.Http, and Java generally uses URLConnection or OkHttp, which are basically inseparable. After combined with decompilation, these HTTP Clients are usually open source, even if they are not Open source, the API is public, it is not difficult for us to find out how to disable SSL verification in this library and how to trust the specified root certificate. Then, the target program can disable SSL verification or trust the Fiddler root certificate by means of decompilation, recompilation, APIHook, Dll injection, Shellcode, etc. Then you can happily grab its bag and read its heart.
In subsequent articles, we will give some targeted examples to explain in detail how to do it.
This article was published by encoderlee on the CSDN blog: https://blog.csdn.net/CharlesSimonyi/article/details/90486208 Please indicate the source for reprinting