Pay attention to the WeChat public account: Brother K crawler, continue to share technical dry goods such as advanced crawler, JS/Android reverse!
statement
All the content in this article is for learning and communication only. The captured content, sensitive URLs, and data interfaces have been desensitized. Commercial and illegal uses are strictly prohibited. Otherwise, all consequences arising therefrom have nothing to do with the author. If there is any infringement , please contact me to delete immediately!
reverse goal
- Goal: Unlimited debugger for an air quality monitoring platform and dynamic encryption and decryption of request data and return data
- Homepage:
aHR0cHM6Ly93d3cuYXFpc3R1ZHkuY24v - interface:
aHR0cHM6Ly93d3cuYXFpc3R1ZHkuY24vYXBpbmV3L2FxaXN0dWR5YXBpLnBocA==
write in front
This site is updated frequently. Before Brother K, many bloggers have written analysis articles on the site. Recently, a reader asked about the encryption of request data and the decryption of returned data, and found that the encryption and decryption JS has become dynamic. The solutions mentioned in the previous articles are not very good, but on the whole it is not difficult, but it is a little more troublesome to deal with, and there are some small details that need to be paid attention to.
In the "About System" of the website, you can see that this site seems to be maintained by individual developers. It has been in existence as early as 2013. In the friendship sponsorship list, you can see that most of them are environmental, surveying and mapping, public health Relevant university majors and research institute personnel can guess that these data are very helpful for their research. Coupled with the frequent anti-crawling updates, it can be seen that the webmaster is suffering from reptiles, and Brother K does not want to give it to the webmaster. Add a burden. After all, we should support this kind of site and let him maintain it for a long time. Therefore, in this issue, Brother K only analyzes the logic and a small part of the code, and does not put the complete code. If there are relevant professionals who really need to capture data for research , you can contact me in the background of the official account.
Bypass infinite debugger
Right-click F12, it will prompt that the right-click is disabled, it does not matter, use the shortcut key Ctrl+Shift+ior the upper right corner of the browser, more tools, developer tools, can still be opened.
method one
After opening the console, you will enter the first infinite debugger, follow a stack up, you can see a try-catch statement, you will find that it will keep going to catch and call the setTimeout()method . This method is used in the specified milliseconds Call the function or calculate the expression after counting. Note that the debugger is passed to the constructor, so here we have two ways to get rid of the debugger. Hook the constructor or setTimeout.
// 两种 Hook 任选一中
// Hook 构造方法
Function.prototype.constructor_ = Function.prototype.constructor;
Function.prototype.constructor = function (a) {
if(a == "debugger") {
return function (){};
}
return Function.prototype.constructor_(a);
};
// Hook setTimeout
var setTimeout_ = setTimeout
var setTimeout = function (func, time){
if (func == txsdefwsw){
return function () {};
}
return setTimeout_(func, time)
}
Then I came to the second infinite debugger, which is also the same as the stack, and found that there is a setInterval timer and a construction method constructor. Similarly, we can Hook off the constructor or setInterval. Note: The timer also detects the window height and width. Even if you have passed the constructor or setInterval, if you don't take out the developer tools separately, it will not work, and it will continue to output "illegal debugging detected".
// Hook setInterval
var setInterval_ = setInterval
setInterval = function (func, time){
if (time == 2000) {
return function () {};
}
return setInterval_(func, time)
}
We have observed that in fact, these two infinite debuggers can be overcome by the Hook constructor, so Fiddler can directly inject the code of the Hook constructor:
Method Two
When we encounter the second infinite debugger, we can also directly follow the stack to a city_realtime.php page, there are two eval statements in it, execute the statement in the first eval and you will find that we are in the VM The debugger code seen in the virtual machine, so in theory, this page can be directly replaced here, remove the eval statement, there will be no infinite debugger, but Brother K will tell you first, it can't work now, because there is a certain load in it. JS, this JS will be used in the subsequent encryption and decryption, but this JS is dynamic and will change every 10 minutes. We will use this page to obtain dynamic JS later, so it cannot be replaced! Just to mention this idea here!
Method three
Of course, there is also the simplest method. Right-click to select Never pause here, and never pause here. You also need to take out the developer tool window separately, otherwise it will always output "Illegal debugging detected".
Packet capture analysis
On the real-time monitoring page, we click to query a city by the way, and we can see that the requested Form Data and the returned data are encrypted, as shown in the following figure:
Encrypted entry
Since it is XHR, we follow the stack directly, and it is easy to find the encrypted location:
You can see the passed data key-value pair: {hXM8NDFHN: p7crXYR}, the key is hard-coded in this JS, the value is pU14VhqrofroULds()obtained , this method needs to pass two parameters, the first is the fixed value GETDATA, the second is the city Name, let's follow up to see what this method is:
Some parameters such as appId, timestamp, city, etc. have done some MD5 and base64 operations, and the returned param is the value we want. It doesn't seem difficult. Let's find out how to decrypt the returned encrypted data. We noticed that the ajax request has a success keyword. Even if we don't understand JS logic, we can guess that it should be the processing operation after the request is successful. , as shown in the following figure: the incoming dzJMI is the returned encrypted data, after passing through the db0HpCYIy97HkHS7RkhUn()method , the decryption is successful:
Follow up the db0HpCYIy97HkHS7RkhUn()method , you can see that it is AES+DES+BASE64 decryption, and the incoming key key and offset iv are defined in the header:
Dynamic JS
After the above analysis, our encryption and decryption logic has been completed, but if you debug more, you will find that this encryption and decryption JS is dynamically changing, and the defined key key and offset iv are changed every time. It will change. If you set a breakpoint in this code, stay for too long, and suddenly find that the breakpoint fails and cannot be broken, it means that JS has changed, and the current code has failed.
Let's just pick two different JS down (hint: JS will change every 10 minutes, and a detailed analysis will be given later), and using PyCharm's file comparison function (select View - Compare With in turn), we can summarize the following changes. (changes in variable names don't count):
- The values of the first 8 parameters: two aes key and iv, two des key and iv;
- When the encrypted param is generated, the appId is changed. The final encryption is divided into AES, DES and no encryption. There are three cases (this is the easiest place to ignore, I didn't notice here, the request may prompt the invalid appId):
- When the request is finally sent, the data key-value pair, the keys of which are also changed:
We found the change, so how do we get this JS? Because this JS is in the VM virtual machine, we have to find its source and where it came from. We can see a special JS when we grab the package, similar to encrypt_xxxxxx.js. Simple, what is returned is a piece of eval-wrapped code:
We are already familiar with eval, just remove eval and let him execute it, and we can see that it is the JS we need:
Here is a small detail. If you use the console, you will find that it has been printing img tags, affecting our input. Here you can directly follow up and go to the breakpoint to temporarily stop it from running. You don’t need to do other operations to waste time:
Do you think you're almost done here? Wrong, the same encrypt_xxxxxx.js also has a mystery:
- The name of encrypt_xxxxxx.js is dynamic, and the following v value is a second-level timestamp, which will change every 600 seconds, that is, ten minutes. This JS can be found on the city_realtime.php page. Remember the bypass we mentioned earlier Can't the infinite debugger replace this page? We want to get dynamic JS through this page, so it cannot be replaced!
- The JS returned by encrypt_xxxxxx.js, not all the plaintext code can be obtained by executing eval once. It is a combination of eval and base64. The first time is eval, but it may be later. It is possible to need base64, it is possible to base64 twice, and it is possible that eval is required after two times of base64. In short, in addition to the first time of eval, whether base64 and eval are required later, and the number of times and the order they are required, are all irrelevant. definite! To give a few examples:
Some people may ask here, how can you tell that it is base64? Very simple, enter directly in the console of the website page dswejwehxt, click to see this function, which is base64:
Then for the case where the content of encrypt_xxxxxx.js is uncertain, we can write a method. After obtaining encrypt_xxxxxx.js, execute eval if you need to execute eval, execute base64 if you need to execute base64, until there is no eval and base64, you can use characters respectively String eval(functionand dswejwehxt(to determine whether eval and base64 are needed (of course there are other ways, such as ()the number of , etc.), the sample code is as follows:
def get_decrypted_js(encrypted_js_url):
"""
:param encrypted_js_url: encrypt_xxxxxx.js 的地址
:return: 解密后的 JS
"""
decrypted_js = requests.get(url=encrypted_js_url, headers=headers).text
flag = True
while flag:
if "eval(function" in decrypted_js:
# 需要执行 eval
print("需要执行 eval！")
replace_js = decrypted_js.replace("eval(function", "(function")
decrypted_js = execjs.eval(replace_js)
elif "dswejwehxt(" in decrypted_js:
# 需要 base64 解码
base64_num = decrypted_js.count("dswejwehxt(")
print("需要 %s 次 base64 解码！" % base64_num)
decrypted_js = re.findall(r"\('(.*?)'\)", decrypted_js)[0]
num = 0
while base64_num > num:
decrypted_js = base64.b64decode(decrypted_js).decode()
num += 1
else:
# 得到明文
flag = False
# print(decrypted_js)
return decrypted_js
local rewrite
Through the above functions, we get the dynamic JS, so can we directly execute the JS we got back? Of course not, you can execute it locally, you can find that CryptoJS, Base64, hex_md5 in it all need to be filled, so here we have two ways:
- After getting the decrypted dynamic JS, the dynamic JS and our own methods such as Base64 and hex_md5 form a new JS code, and execute the new JS code to get the parameters. We also need to pay attention here because the other method names are dynamic. , so you have to find a way to match the correct method name to call, so this method personally feels a little troublesome;
- We write a JS locally. After getting the decrypted dynamic JS, we need to encrypt the key, iv, appId, data key name, and param whether AES or DES is required. These information are matched, and then passed to our own writing. JS, call our own method to get the encrypted result.
Although both methods are very troublesome, Brother K can't think of a better solution for the time being. Friends who have better ideas can leave a message and talk about it.
Take the second method as an example, our local JS example (main.js):
var CryptoJS = require("crypto-js");
var BASE64 = {
encrypt: function (text) {
return CryptoJS.enc.Base64.stringify(CryptoJS.enc.Utf8.parse(text))
},
decrypt: function (text) {
return CryptoJS.enc.Base64.parse(text).toString(CryptoJS.enc.Utf8)
}
};
var DES = {
encrypt: function (text, key, iv) {
var secretkey = (CryptoJS.MD5(key).toString()).substr(0, 16);
var secretiv = (CryptoJS.MD5(iv).toString()).substr(24, 8);
secretkey = CryptoJS.enc.Utf8.parse(secretkey);
secretiv = CryptoJS.enc.Utf8.parse(secretiv);
var result = CryptoJS.DES.encrypt(text, secretkey, {
iv: secretiv,
mode: CryptoJS.mode.CBC,
padding: CryptoJS.pad.Pkcs7
});
return result.toString();
},
decrypt: function (text, key, iv) {
var secretkey = (CryptoJS.MD5(key).toString()).substr(0, 16);
var secretiv = (CryptoJS.MD5(iv).toString()).substr(24, 8);
secretkey = CryptoJS.enc.Utf8.parse(secretkey);
secretiv = CryptoJS.enc.Utf8.parse(secretiv);
var result = CryptoJS.DES.decrypt(text, secretkey, {
iv: secretiv,
mode: CryptoJS.mode.CBC,
padding: CryptoJS.pad.Pkcs7
});
return result.toString(CryptoJS.enc.Utf8);
}
};
var AES = {
encrypt: function (text, key, iv) {
var secretkey = (CryptoJS.MD5(key).toString()).substr(16, 16);
var secretiv = (CryptoJS.MD5(iv).toString()).substr(0, 16);
secretkey = CryptoJS.enc.Utf8.parse(secretkey);
secretiv = CryptoJS.enc.Utf8.parse(secretiv);
var result = CryptoJS.AES.encrypt(text, secretkey, {
iv: secretiv,
mode: CryptoJS.mode.CBC,
padding: CryptoJS.pad.Pkcs7
});
return result.toString();
},
decrypt: function (text, key, iv) {
var secretkey = (CryptoJS.MD5(key).toString()).substr(16, 16);
var secretiv = (CryptoJS.MD5(iv).toString()).substr(0, 16);
secretkey = CryptoJS.enc.Utf8.parse(secretkey);
secretiv = CryptoJS.enc.Utf8.parse(secretiv);
var result = CryptoJS.AES.decrypt(text, secretkey, {
iv: secretiv,
mode: CryptoJS.mode.CBC,
padding: CryptoJS.pad.Pkcs7
});
return result.toString(CryptoJS.enc.Utf8);
}
};
function getDecryptedData(data, AES_KEY_1, AES_IV_1, DES_KEY_1, DES_IV_1) {
data = AES.decrypt(data, AES_KEY_1, AES_IV_1);
data = DES.decrypt(data, DES_KEY_1, DES_IV_1);
data = BASE64.decrypt(data);
return data;
}
function ObjectSort(obj) {
var newObject = {};
Object.keys(obj).sort().map(function (key) {
newObject[key] = obj[key];
});
return newObject;
}
function getRequestParam(method, obj, appId) {
var clienttype = 'WEB';
var timestamp = new Date().getTime()
var param = {
appId: appId,
method: method,
timestamp: timestamp,
clienttype: clienttype,
object: obj,
secret: CryptoJS.MD5(appId + method + timestamp + clienttype + JSON.stringify(ObjectSort(obj))).toString()
};
param = BASE64.encrypt(JSON.stringify(param));
return param;
}
function getRequestAESParam(requestMethod, requestCity, appId, AES_KEY_2, AES_IV_2){
var param = getRequestParam(requestMethod, requestCity, appId);
return AES.encrypt(param, AES_KEY_2, AES_IV_2);
}
function getRequestDESParam(requestMethod, requestCity, appId, DES_KEY_2, DES_IV_2){
var param = getRequestParam(requestMethod, requestCity, appId);
return DES.encrypt(param, DES_KEY_2, DES_IV_2);
}
We match the Python code example of various parameters in JS (matching the encryption method of 8 keys, iv values, appId and param):
def get_key_iv_appid(decrypted_js):
"""
:param decrypted_js: 解密后的 encrypt_xxxxxx.js
:return: 请求必须的一些参数
"""
key_iv = re.findall(r'const.*?"(.*?)";', decrypted_js)
app_id = re.findall(r"var appId.*?'(.*?)';", decrypted_js)
request_data_name = re.findall(r"aqistudyapi.php.*?data.*?{(.*?):", decrypted_js, re.DOTALL)
# 判断 param 是 AES 加密还是 DES 加密还是没有加密
if "AES.encrypt(param" in decrypted_js:
request_param_encrypt = "AES"
elif "DES.encrypt(param" in decrypted_js:
request_param_encrypt = "DES"
else:
request_param_encrypt = "NO"
key_iv_appid = {
# key 和 iv 的位置和原来 js 里的是一样的
"aes_key_1": key_iv[0],
"aes_iv_1": key_iv[1],
"aes_key_2": key_iv[2],
"aes_iv_2": key_iv[3],
"des_key_1": key_iv[4],
"des_iv_1": key_iv[5],
"des_key_2": key_iv[6],
"des_iv_2": key_iv[7],
"app_id": app_id[0],
# 发送请求的 data 的键名
"request_data_name": request_data_name[0].strip(),
# 发送请求的 data 值需要哪种加密
"request_param_encrypt": request_param_encrypt
}
# print(key_iv_appid)
return key_iv_appid
An example of our Python code for sending a request and decrypting the return value (taking Beijing as an example):
def get_data(key_iv_appid):
"""
:param key_iv_appid: get_key_iv_appid() 方法返回的值
"""
request_method = "GETDATA"
request_city = {"city": "北京"}
with open('main.js', 'r', encoding='utf-8') as f:
execjs_ = execjs.compile(f.read())
# 根据不同加密方式调用不同方法获取请求加密的 param 参数
request_param_encrypt = key_iv_appid["request_param_encrypt"]
if request_param_encrypt == "AES":
param = execjs_.call(
'getRequestAESParam', request_method, request_city,
key_iv_appid["app_id"], key_iv_appid["aes_key_2"], key_iv_appid["aes_iv_2"]
)
elif request_param_encrypt == "DES":
param = execjs_.call(
'getRequestDESParam', request_method, request_city,
key_iv_appid["app_id"], key_iv_appid["des_key_2"], key_iv_appid["des_iv_2"]
)
else:
param = execjs_.call('getRequestParam', request_method, request_city, key_iv_appid["app_id"])
data = {
key_iv_appid["request_data_name"]: param
}
response = requests.post(url=aqistudy_api, headers=headers, data=data).text
# print(response)
# 对获取的加密数据解密
decrypted_data = execjs_.call(
'getDecryptedData', response,
key_iv_appid["aes_key_1"], key_iv_appid["aes_iv_1"],
key_iv_appid["des_key_1"], key_iv_appid["des_iv_1"]
)
print(json.loads(decrypted_data))
Running result, successful request and decrypt return value:
