【作者主页】：吴秋霖
【作者介绍】：Python领域优质创作者、阿里云博客专家、华为云享专家。长期致力于Python与爬虫领域研究与开发工作！
【作者推荐】：对JS逆向感兴趣的朋友可以关注《爬虫JS逆向实战》，对分布式爬虫平台感兴趣的朋友可以关注《分布式爬虫平台搭建与开发实战》
还有未来会持续更新的验证码突防、APP逆向、Python领域等一系列文章
1. 写在前面
某些网站在请求的时候都会带上一个sign签名参数，某宝的几个H5网站几乎都有sign，对于加密参数，也不是说非要逆向去实现，不同的背景下有各种不同的方案可以选择
目标站点：
Cmh0dHBzOi8vc2FsZS4xNjg4LmNvbS8=
2. 抓包分析
打开网站随便搜索，抓包一下数据接口，可以看到请求参数如下所示：
简单的过一下请求参数，这里除了sign是加密参数，其他的基本都是固定参数加上非加密参数，所以接下来去分析sign的加密
3. 断点分析
参数关键词搜索打上断点，可以看到sign参数的加密看起来并不是很难，如下代码所示：
把加密的关键代码拿出来，分析一下，如下所示：
j = h(d.token + "&" + i + "&" + g + "&" + c.data)
h是加密方法，d.token（下面细说），i是当前的时间戳，g是请求头内的appKey参数值，也是固定的，最后c.data是请求参数内的data参数
通过&字符串拼接上面多个变量再去调用h实现对sign参数的加密
加密所需要的参数捋清楚之后，开始分析h加密方法，最终的sign参数计算就是它出来的。鼠标点击跳转到h函数，如下所示：
代码并不多，将它扣下来，如下所示：
function h(a) {
function b(a, b) {
return a << b | a >>> 32 - b
}
function c(a, b) {
var c, d, e, f, g;
return e = 2147483648 & a,
f = 2147483648 & b,
c = 1073741824 & a,
d = 1073741824 & b,
g = (1073741823 & a) + (1073741823 & b),
c & d ? 2147483648 ^ g ^ e ^ f : c | d ? 1073741824 & g ? 3221225472 ^ g ^ e ^ f : 1073741824 ^ g ^ e ^ f : g ^ e ^ f
}
function d(a, b, c) {
return a & b | ~a & c
}
function e(a, b, c) {
return a & c | b & ~c
}
function f(a, b, c) {
return a ^ b ^ c
}
function g(a, b, c) {
return b ^ (a | ~c)
}
function h(a, e, f, g, h, i, j) {
return a = c(a, c(c(d(e, f, g), h), j)),
c(b(a, i), e)
}
function i(a, d, f, g, h, i, j) {
return a = c(a, c(c(e(d, f, g), h), j)),
c(b(a, i), d)
}
function j(a, d, e, g, h, i, j) {
return a = c(a, c(c(f(d, e, g), h), j)),
c(b(a, i), d)
}
function k(a, d, e, f, h, i, j) {
return a = c(a, c(c(g(d, e, f), h), j)),
c(b(a, i), d)
}
function l(a) {
for (var b, c = a.length, d = c + 8, e = (d - d % 64) / 64, f = 16 * (e + 1), g = new Array(f - 1), h = 0, i = 0; c > i;)
b = (i - i % 4) / 4,
h = i % 4 * 8,
g[b] = g[b] | a.charCodeAt(i) << h,
i++;
return b = (i - i % 4) / 4,
h = i % 4 * 8,
g[b] = g[b] | 128 << h,
g[f - 2] = c << 3,
g[f - 1] = c >>> 29,
g
}
function m(a) {
var b, c, d = "", e = "";
for (c = 0; 3 >= c; c++)
b = a >>> 8 * c & 255,
e = "0" + b.toString(16),
d += e.substr(e.length - 2, 2);
return d
}
function n(a) {
a = a.replace(/\r\n/g, "\n");
for (var b = "", c = 0; c < a.length; c++) {
var d = a.charCodeAt(c);
128 > d ? b += String.fromCharCode(d) : d > 127 && 2048 > d ? (b += String.fromCharCode(d >> 6 | 192),
b += String.fromCharCode(63 & d | 128)) : (b += String.fromCharCode(d >> 12 | 224),
b += String.fromCharCode(d >> 6 & 63 | 128),
b += String.fromCharCode(63 & d | 128))
}
return b
}
var o, p, q, r, s, t, u, v, w, x = [], y = 7, z = 12, A = 17, B = 22, C = 5, D = 9, E = 14, F = 20, G = 4, H = 11,
I = 16, J = 23, K = 6, L = 10, M = 15, N = 21;
for (a = n(a),
x = l(a),
t = 1732584193,
u = 4023233417,
v = 2562383102,
w = 271733878,
o = 0; o < x.length; o += 16)
p = t,
q = u,
r = v,
s = w,
t = h(t, u, v, w, x[o + 0], y, 3614090360),
w = h(w, t, u, v, x[o + 1], z, 3905402710),
v = h(v, w, t, u, x[o + 2], A, 606105819),
u = h(u, v, w, t, x[o + 3], B, 3250441966),
t = h(t, u, v, w, x[o + 4], y, 4118548399),
w = h(w, t, u, v, x[o + 5], z, 1200080426),
v = h(v, w, t, u, x[o + 6], A, 2821735955),
u = h(u, v, w, t, x[o + 7], B, 4249261313),
t = h(t, u, v, w, x[o + 8], y, 1770035416),
w = h(w, t, u, v, x[o + 9], z, 2336552879),
v = h(v, w, t, u, x[o + 10], A, 4294925233),
u = h(u, v, w, t, x[o + 11], B, 2304563134),
t = h(t, u, v, w, x[o + 12], y, 1804603682),
w = h(w, t, u, v, x[o + 13], z, 4254626195),
v = h(v, w, t, u, x[o + 14], A, 2792965006),
u = h(u, v, w, t, x[o + 15], B, 1236535329),
t = i(t, u, v, w, x[o + 1], C, 4129170786),
w = i(w, t, u, v, x[o + 6], D, 3225465664),
v = i(v, w, t, u, x[o + 11], E, 643717713),
u = i(u, v, w, t, x[o + 0], F, 3921069994),
t = i(t, u, v, w, x[o + 5], C, 3593408605),
w = i(w, t, u, v, x[o + 10], D, 38016083),
v = i(v, w, t, u, x[o + 15], E, 3634488961),
u = i(u, v, w, t, x[o + 4], F, 3889429448),
t = i(t, u, v, w, x[o + 9], C, 568446438),
w = i(w, t, u, v, x[o + 14], D, 3275163606),
v = i(v, w, t, u, x[o + 3], E, 4107603335),
u = i(u, v, w, t, x[o + 8], F, 1163531501),
t = i(t, u, v, w, x[o + 13], C, 2850285829),
w = i(w, t, u, v, x[o + 2], D, 4243563512),
v = i(v, w, t, u, x[o + 7], E, 1735328473),
u = i(u, v, w, t, x[o + 12], F, 2368359562),
t = j(t, u, v, w, x[o + 5], G, 4294588738),
w = j(w, t, u, v, x[o + 8], H, 2272392833),
v = j(v, w, t, u, x[o + 11], I, 1839030562),
u = j(u, v, w, t, x[o + 14], J, 4259657740),
t = j(t, u, v, w, x[o + 1], G, 2763975236),
w = j(w, t, u, v, x[o + 4], H, 1272893353),
v = j(v, w, t, u, x[o + 7], I, 4139469664),
u = j(u, v, w, t, x[o + 10], J, 3200236656),
t = j(t, u, v, w, x[o + 13], G, 681279174),
w = j(w, t, u, v, x[o + 0], H, 3936430074),
v = j(v, w, t, u, x[o + 3], I, 3572445317),
u = j(u, v, w, t, x[o + 6], J, 76029189),
t = j(t, u, v, w, x[o + 9], G, 3654602809),
w = j(w, t, u, v, x[o + 12], H, 3873151461),
v = j(v, w, t, u, x[o + 15], I, 530742520),
u = j(u, v, w, t, x[o + 2], J, 3299628645),
t = k(t, u, v, w, x[o + 0], K, 4096336452),
w = k(w, t, u, v, x[o + 7], L, 1126891415),
v = k(v, w, t, u, x[o + 14], M, 2878612391),
u = k(u, v, w, t, x[o + 5], N, 4237533241),
t = k(t, u, v, w, x[o + 12], K, 1700485571),
w = k(w, t, u, v, x[o + 3], L, 2399980690),
v = k(v, w, t, u, x[o + 10], M, 4293915773),
u = k(u, v, w, t, x[o + 1], N, 2240044497),
t = k(t, u, v, w, x[o + 8], K, 1873313359),
w = k(w, t, u, v, x[o + 15], L, 4264355552),
v = k(v, w, t, u, x[o + 6], M, 2734768916),
u = k(u, v, w, t, x[o + 13], N, 1309151649),
t = k(t, u, v, w, x[o + 4], K, 4149444226),
w = k(w, t, u, v, x[o + 11], L, 3174756917),
v = k(v, w, t, u, x[o + 2], M, 718787259),
u = k(u, v, w, t, x[o + 9], N, 3951481745),
t = c(t, p),
u = c(u, q),
v = c(v, r),
w = c(w, s);
var O = m(t) + m(u) + m(v) + m(w);
return O.toLowerCase()
}
接上面，我们来说一说d.token的值是如何生成的，通过分析发现它跟我们请求头Cookie中的_m_h5_tk的前面部分一致，如下对比图所示：
token是可以复用的，不需要每个请求都重新获取一次token。但需要注意的是token是有有效期的，如果返回"令牌过期"，就需要重新获取
4. 爬虫实现
截止于此，上面所有加密所需关键参数都已经完成，接下来开始编写爬虫代码，实现sign加密并请求网站获取数据，如下所示：
import time
import requests
import execjs
def generate_sign(token, g, data):
timestamp = int(time.time() * 1000)
j = f"{
token}&{
timestamp}&{
g}&{
str(data)}"
with open("h.js", "r", encoding="utf-8") as f:
js_code = f.read()
return execjs.compile(js_code).call("h", j)
def main():
token = "" # Token（自行按照上面方法获取即可）
g = "12574478" # 固定值
data = '' # 根据请求参数自行获取即可
sign = generate_sign(token, g, data)
headers = {
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36",
"Referer": "https://sale.1688.com/",
"Cookie": "", # 请替换为你的Cookie值
}
params = {
"jsv": "2.6.1",
"appKey": "",
"t": str(int(time.time() * 1000)),
"sign": sign,
"v": "1.0",
"type": "jsonp",
"isSec": 0,
"timeout": 20000,
"api": "mtop.taobao.widgetService.getJsonComponent",
"dataType": "jsonp",
"jsonpIncPrefix": "mboxfc",
"callback": "mtopjsonpmboxfc3",
"data": data,
}
url = "https://h5api.m.1688.com/h5/mtop.taobao.widgetservice.getjsoncomponent/1.0/"
response = requests.get(url, headers=headers, params=params)
print(response.text)
if __name__ == "__main__":
main()
上面使用的execjs直接调用扣下来的JS代码，当然咱们也可以尝试使用Python去还原上面的JS加密代码，如下：
import struct
import hashlib
def h(message):
def left_rotate(n, b):
return ((n << b) | (n >> (32 - b))) & 0xFFFFFFFF
def process_chunk(chunk, h0, h1, h2, h3):
w = [0] * 64
for i in range(16):
w[i] = struct.unpack(b'>I', chunk[i * 4:i * 4 + 4])[0]
for i in range(16, 64):
w[i] = left_rotate(w[i - 16] ^ w[i - 8] ^ w[i - 14] ^ w[i - 3], 1)
a, b, c, d = h0, h1, h2, h3
for i in range(64):
if 0 <= i <= 15:
f = (b & c) | ((~b) & d)
g = i
elif 16 <= i <= 31:
f = (d & b) | ((~d) & c)
g = (5 * i + 1) % 16
elif 32 <= i <= 47:
f = b ^ c ^ d
g = (3 * i + 5) % 16
elif 48 <= i <= 63:
f = c ^ (b | (~d))
g = (7 * i) % 16
f = (f + a + k[i] + w[g]) & 0xFFFFFFFF
a = d
d = c
c = b
b = (b + left_rotate(f, s[i])) & 0xFFFFFFFF
h0 = (h0 + a) & 0xFFFFFFFF
h1 = (h1 + b) & 0xFFFFFFFF
h2 = (h2 + c) & 0xFFFFFFFF
h3 = (h3 + d) & 0xFFFFFFFF
return h0, h1, h2, h3
message = message.encode('utf-8')
original_length = len(message)
message += b'\x80'
while (len(message) + 8) % 64 != 0:
message += b'\x00'
message += struct.pack(b'>Q', original_length * 8)
h0, h1, h2, h3 = 0x67452301, 0xefcdab89, 0x98badcfe, 0x10325476
k = [0] * 64
s = [7, 12, 17, 22, 5, 9, 14, 20, 4, 11, 16, 23, 6, 10, 15, 21]
for i in range(64):
k[i] = int(2 ** 32 * abs(math.sin(i + 1)))
for i in range(0, len(message), 64):
chunk = message[i:i + 64]
h0, h1, h2, h3 = process_chunk(chunk, h0, h1, h2, h3)
return '{:08x}{:08x}{:08x}{:08x}'.format(h0, h1, h2, h3)
好了，到这里又到了跟大家说再见的时候了。创作不易，帮忙点个赞再走吧。你的支持是我创作的动力，希望能带给大家更多优质的文章