In addition to Web pages, reptiles can also grab the data of APP, APP pages to load it, you first need to get the data, then the data is generally through an interface request to the server to get, because the client does not like APP browser developer tools directly see the request background intuitively, so the APP, its main use some data fetch packet capture technology.
Packet capture tools presented in this book have Charles, MitmProxy, MitmDump, APP simple interface analysis we find out the law by Charles or MitmProxy simulation program can be directly used to grab, but if you encounter more complex interface we need to use MitmDump docking Python to crawl to the requests and responses in real-time processing and preservation, since to do another acquisition on the scale of operations to be automated rather than manual APP to collect, so here also need a tool called Appium, it can be like Selenium automatic control of APP, APP An automated simulation clicks, the pull-down operation.
In this section we look at Charles, MitmProxy, MitmDump, Appium installation method.
1.6.1 Charles installation
Charles is a network capture tool, making APP packet capture will be used when compared Fiddler it, Charles is more powerful, and better cross-platform support, so here we use Charles as the major mobile end of the capture tools for analyzing data packets moving APP, APP data capture auxiliary work is completed.
1. Links
- Official website:https://www.charlesproxy.com
- Download Link:https://www.charlesproxy.com/...
2. Download Charles
Charles's official website is:https://www.charlesproxy.com , we can download the latest stable version of the official website, the link is:https://www.charlesproxy.com/ ..., it supports Windows, Linux, Mac three major platforms.
3. certificate configuration
Now many pages are moving in the direction of HTTPS, HTTPS communication protocol used more and more widely, if a communication application of the APP HTTPS protocol, and that it will communicate the data is encrypted, the conventional method is cut package does not recognize the request for internal data. Python learning qun seven hundred eighty-four 5, July Eight, two hundred and fourteen tools, sharing all kinds of combat operations
after the installation is complete, if we want to do HTTPS request packet capture, then also need to configure the relevant SSL certificate, if not, then not crawl HTTPS configuration .
Click Help-> SSL Proxying-> Install Charles Root Certificate, you can enter the certificate installation page.
Then double-click the certificate to find Charles, the trust can always trust, shown in Figure 1-48:
Figure 1-48 Certificate configuration
so that you have successfully installed the certificate.
iOS
If your phone is iOS system, the certificate can be configured in accordance with the following operation.
First, look at the computer's Charles Agent is open, click Proxy-> Proxy Settings to open the current proxy settings page, make sure the current HTTP proxy is open, as shown 1-49:
Figure 1-49 proxy settings
, for example, where the proxy port 8888, can also modify their own.
Next to mobile phones and computers connected under the same local area network, such as the current computer's IP is 192.168.1.76, then the first set the phone's agent for 192.168.1.76:8888, set in Figure 1-50:
图 1-50 代理设置
设置完毕之后电脑上就会出现一个提示窗口，询问是否信任此设备，如图 1-51 所示：
图 1-51 提示窗口
点击 Allow 即可，这样手机就和 PC 连在了同一个局域网内了，而且设置了 Charles 的代理，这样 Charles 即可抓取到流经 APP 的数据包了。
接下来我们再安装 Charles 的 HTTPS 证书。
在电脑上打开 Help->SSL Proxying->Install Charles Root Certificate on a Mobile Device or Remote Browser，如图 1-52 所示：
图 1-52 证书安装页面入口
即可看到如下提示，如图 1-53 所示：
它提示我们在手机上设置好 Charles 的代理，我们刚才已经设置好了，然后在手机的浏览器中打开 chls.pro/ssl 下载证书，我们在手机上打开这个链接。
打开之后便会弹出证书的安装页面，如图 1-54 所示：
图 1-54 证书安装页面
我们点击安装，然后输入密码即可完成安装，如图 1-55 所示：
图 1-55 安装成功页面
如果你的 iOS 版本是 10.3 以下的话，此处信任 CA 证书的流程就已经完成了。Python学习q-u-n七八四，七五八，二一四教程视频，工具，各类实战操作分享
如果你的 iOS 版本是 10.3 及以上，还需要在设置->通用->关于本机->证书信任设置将证书添加完全信任，如图 1-56 所示：
在这里将 Charles 的证书的完全信任开关打开即可，如图 1-56 所示：
Figure 1-56 Certificate trust settings
so trusted CA certificate configuration process ended on iOS.
Android
If your phone is Android system, you can configure a certificate in accordance with the following operation.
Android Charles also need to set up a proxy agent, shown in Figure 1-57:
Figure 1-57 proxy settings
will appear on the computer after set up a prompt window asking whether to trust this device, as shown in Figure 1-58:
Figure 1-58 Prompt window
click Allow to.
Next, the same as the iOS device, open chls.pro/ssl on a mobile browser, then there will be a prompt box, Python learning qun seven hundred eighty-four 5, July Eight, two hundred and fourteen tools, all kinds of combat operations share. As shown in FIG 1-59:
evernotecid: // D603D29C-DFBA-4C04-85E9-CCA3C33763F6 / appyinxiangcom / 23,852,268 / ENResource / of p75
Figure 1-59 Certificate Installation page
, we add a name for the certificate, and then click OK to complete the installation of the certificate.
4. Conclusion
Charles This article describes the installation and configuration process certificate, later we will introduce the basic use of Charles.