[Python3 Web crawler developed combat] 1.7.2-mitmproxy installation

Abstract mitmproxy is a HTTP and HTTPS packet capture program, similar to Fiddler, Charles function, but it operates in the form of the console.

In addition, there are two mitmproxy associated components, is a mitmdump, it is mitmproxy command line interface, it can be docked using the Python script, listening to achieve treatment; the other is mitmweb, it is a Web application, it is clear by observed mitmproxy capture request.

In this section, we take a look at the installation mitmproxy, mitmdump and mitmweb of.

1. Links

GitHub:https://github.com/mitmproxy/mitmproxy

Official Website: https://mitmproxy.org

PyPI:https://pypi.python.org/pypi/mitmproxy

The official document: http://docs.mitmproxy.org

mitmdump script: http://docs.mitmproxy.org/en/stable/scripting/overview.html

Download: https://github.com/mitmproxy/mitmproxy/releases

DockerHub:https://hub.docker.com/r/mitmproxy/mitmproxy

2. pip install

The easiest way to install or use pip, directly execute the following command to install:

pip3 install mitmproxy

This is the most simple and versatile installation to complete the installation after mitmproxy finished, and also comes with mitmdump mitmweb these two components are installed. If you do not want to install this way, you can also choose a special installation for each platform for installation or Docker listed thereafter.

3. Windows installation under

Releases to be a page on GitHub (link is: https: //github.com/mitmproxy/mitmproxy/releases/) for installation package, as shown in FIG 1-59. Download Page 1-59

For example, 2.0.2, you can choose to download the exe under Windows installation package mitmproxy-2.0.2-windows-installer.exe, simply double-click after downloading the installation package to install the most current version.

Note that the console does not support the interface mitmproxy on Windows, but you can use mitmdump and mitmweb.

4. Linux installation of

In Linux, you can download the compiled binary packages (Download https://github.com/mitmproxy/mitmproxy/releases/), this distribution package is generally the latest version, which includes the latest version of mitmproxy and built-in Python 3 environment, and the latest OpenSSL environment.

If your environment does not Python 3 and OpenSSL environment, it is recommended to use this way installation.

Once downloaded, you need to unpack and configure the environment variables:

tar -zxvf mitmproxy-2.0.2-linux.tar.gz
sudo mv mitmproxy mitmdump mitmweb /usr/bin

This allows the three executable files moved to / usr / bin directory. Under normal circumstances, / usr / bin directory already configured in the environment variable, so the next can directly call these three tool.

5. Mac installed under

Installation on Mac is very simple and straightforward to use Homebrew, the command is as follows:

brew install mitmproxy

After executing the command, mitmproxy to complete the installation.

6. Docker installation

mitmproxy also supports Docker, its DockerHub address is https://hub.docker.com/r/mitmproxy/mitmproxy/.

In Docker, mitmproxy installation command:

docker run --rm -it -p 8080:8080 mitmproxy/mitmproxy mitmdump

This port was launched in 8080 and mitmproxy mitmdump.

If you want to retrieve a CA certificate, you can choose to mount disk option, the command is as follows:

docker run --rm -it -v ~/.mitmproxy:/home/mitmproxy/.mitmproxy -p 8080:8080 mitmproxy/mitmproxy mitmdump

This CA certificate can be found at ~ / .mitmproxy directory.

It is also possible to start mitmweb on port 8081, the command is as follows:

docker run --rm -it -p 8080:8080 -p 127.0.0.1:8081:8081 mitmproxy/mitmproxy mitmweb

More start-up mode can refer to the installation instructions Docker Hub.

7. certificate configuration

For mitmproxy, if you want to intercept HTTPS requests, you need to set the certificate. mitmproxy after installation will provide a CA certificate, as long as the client trust certificates mitmproxy offer, you can get the specific content of HTTPS requests by mitmproxy, otherwise mitmproxy is unable to resolve HTTPS requests.

First, run the following command to generate the CA certificate, and start mitmdump:

mitmdump

Next, we can find the CA certificate in .mitmproxy directory under the user directory, shown in Figure 1-60.

1-61.jpg

Figure 1-60 certificate file

A total of five certificates, Table 1-1 briefly describes these five certificates.

Table 1-15 certificates and their descriptions

image.png

Double-mitmproxy-ca.p12, will be introduced intro the certificate, as shown in FIG 1-61.

1-62.jpg

Figure 1-61 Certificate Import Wizard

Click "Next" button, set a password prompt will appear, as shown in Figure 1-62.

1-63.jpg

Figure 1-62 Password Settings Tips

There is no need to set a password, click on the "Next" button.

Next, the storage area need to select the certificate, as shown in FIG 1-63. Here click on the second option "Place all certificates in the following store" and click the "Browse" button, select the certificate store location "Authorities Trusted Root Certificate" is, then click on the "OK" button, then click " Next "button.

1-63.jpg

FIG selected certificate storage region 1-63

Finally, if there is a security warning pops up, as shown in Figure 1-64, click "Yes" button.

1-65.jpg

Figure 1-64 Security Warning

This will complete the configuration of CA certificates in Windows.

Mac

Mac double-click mitmproxy-ca-cert.pem to bring up the keychain management page, and then find mitmproxy certificate to open its setup options, select "Always Trust" button, as shown in Figure 1-65.

Figure 1-65 Certificate Configuration

1-65.jpg

iOS

Transmitting mitmproxy-ca-cert.pem files to the iPhone, we recommend the use of mail transmission and can click on the iPhone and recognize the installed accessory, as shown in Figure 1-66.

1-67.jpg

Figure 1-66 Certificate Installation page

After clicking "Install" button will jump to install the profile page, click on the "Install" button, then there will be a warning message shown in Figure 1-67.

1-68.jpg

Figure 1-67 Installation Warning page

Continue to click "Install" button in the upper right corner, and after a successful installation will be installed prompt, as shown in Figure 1-68.

1-69.jpg

Figure 1-68 Installation successful page

If your iOS version is 10.3 or less, the flow trusted CA certificate here has been completed.

If your iOS version 10.3 and above, but also in the "Settings" → "General" → "on the machine" → "certificate trust settings" full trust mitmproxy the switch is open, as shown in FIG 1-69. At this point, the process of configuring trusted CA certificate on iOS is over.

1-70.jpg

Figure 1-69 Certificate trust settings

Android

On an Android phone, also you need to send a certificate mitmproxy-ca-cert.pem file to the phone, for example, copy the file directly.

Next, click the certificate, a prompt window will appear, as shown in Figure 1-70.

1-71 (1).jpg

Figure 1-70 Certificate Installation page

Then enter the name of the certificate, and then click the "OK" button to complete the installation.

 

Source: Huawei cloud community  Author: Cui Shu Jing Qing only seek

Guess you like

Origin blog.csdn.net/devcloud/article/details/93722181