Kuaishou short video crawler, signature algorithm implementation

1. The packet capture tool grabs a request

POST /rest/n/feed/nearby?app=0&kpf=ANDROID_PHONE&ver=6.5&c=HUAWEI_KWAI&mod=HUAWEI%28HWI-AL00%29&appver=6.5.5.9591&ftt=&isp=CUCC&kpn=KUAISHOU&lon=102.698614&language=zh-cn&sys=ANDROID_9&max_memory=384&ud=0&country_code=cn&oc=HUAWEI_KWAI&hotfix_ver=&did_gt=1584622753889&iuid=&net=WIFI&did=ANDROID_9ba4839bf09a1834&lat=25.002707 HTTP/1.1

type=10&page=1&token=&count=20&id=9&refreshTimes=0&coldStart=false&source=1&browseType=1&seid=60ed7899-e25e-4b9b-b971-3f75b4df00fd&os=android&sig=75e33af6cb4a795c039e0f94a9bd27bf&client_key=3c2cd3f3

2. Analysis

        You can see that in the requested parameters, there is a field called sig. The main work we need to do is to calculate the sig through the parameters. Through reverse analysis, it is found that the sig calculation method is:

Put the parameters in the url into map1;
put the parameters in the form into map2;
put the elements in map1 and map2 into the arraylist in the form of key=value

Sort
the arraylist ; splice the elements in the arraylist into a string str in order;
convert str into a bytearray;
call CPU.getClock() and pass in str to calculate the signature;
CPU.getClock() is a native method, in libcore Implemented in .so

3. Put the code

from sig import signature
import requests
para = {
    "app":"0",
    "kpf":"ANDROID_PHONE",
    "ver":"6.5",
    "c":"HUAWEI_KWAI",
    "mod":"HUAWEI(HWI-AL00)",
    "appver":"6.5.5.9591",
    # "ftt":"",
    "isp":"CUCC",
    "kpn":"KUAISHOU",
    # "lon":"102.698614",
    "language":"zh-cn",
    "sys":"ANDROID_9",
    "max_memory":"384",
    "ud":"0",
    "country_code":"cn",
    "oc":"HUAWEI_KWAI",
    # "hotfix_ver":"",
    "did_gt":"1584622753889",
    # "iuid":"",
    "net":"WIFI",
    "did":"ANDROID_9ba4839bf09a1834",
    # "lat":"25.002707"
}
post = {
    "type":"10",
    "page":"1",
    "token":"",
    "count":"20",
    "id":"9",
    "refreshTimes":"0",
    "coldStart":"false",
    "source":"1",
    "browseType":"1",
    "seid":"60ed7899-e25e-4b9b-b971-3f75b4df00fd",
    "os":"android",
    "client_key":"3c2cd3f3"
}
j = signature.WeChat_YY_yhzf.sig_post("https://apissl.ksapisrv.com/rest/n/feed/nearby",para,post)
header = {"Content-Type":"application/x-www-form-urlencoded"}

resp = requests.post(j["para"], data=j["data"],headers=header)
print(resp.text)

The result of the request is as follows:

Get it done, perfect. Other interfaces can be processed similarly, communication and learning can be +v:YY_yhzf