Pay attention to the WeChat public account: Brother K crawler, continue to share technical dry goods such as advanced crawler, JS/Android reverse!
reverse goal
The goal of this reverse process comes from the extra web intermediate question of My Love Crack 2022 Spring Festival Problem Solving and Red Envelopes. My Love Crack will have a red envelope distribution every year (send My Love Coins), and everyone needs to use their reverse skills to analyze the content. To get a password red envelope, there are a total of five questions this year, one for bonus points, two for Windows, one for Android, and one for Web. This article analyzes the Web questions. My love stipulates that the password and discussion should not be leaked before the event ends. Share the analysis process, so this article was released after the event.
The title of this web topic is: Xiao D's favorite video website has recently shut down. Before shutting down, he used Fiddler and Web Archive to save a video of a host, but he found that the saved file could not be played. Can you help Xiao D get his memories back? (Choose one of .saz and .wacz to solve the problem)
In order to prevent Wu Ai from closing the problem-solving channel in the later stage, Brother K saved a copy of the .saz and .wacz files, which can be obtained by replying to Wu Ai in the background of the official account !
-
Event address: https://www.52pojie.cn/thread-1582582-1-1.html
-
Web question address: https://www.52pojie.cn/home.php?mod=task&do=view&id=17
HLS Streaming Media Transport Protocol
This question involves the HLS streaming media transmission protocol. Let me briefly introduce it first, and those who understand it can skip it directly.
The full name of HLS is HTTP Live Streaming, which is an HTTP-based adaptive bitrate streaming media transmission protocol. It is a dynamic bitrate adaptive technology developed by Apple. It includes an M3U(8) index file, several TS video stream files. If the video stream If the file is encrypted, there will also be a key encrypted string file.
M3U8 files are a type of M3U except that the text stored in the file is encoded using UTF-8 characters, and in rare cases, M3U8 files may be saved with the M3UP extension. An M3U8 file is a playlist file used by various audio and video playback programs. It contains the path or URL of a media file or media folder, and related information about the playlist.
The full name of TS is MPEG2-TS, TS is Transport Stream, also known as MPEG-TS, MTS, TP. The feature of this format is that it can be decoded independently from any segment of the video stream.
For files in TS format, if they are unencrypted, ordinary players can play them directly, or they can be converted to other formats using tools such as FFmpeg. FFmpeg can also directly process M3U8 files, automatically decrypt, merge and convert TS files, and of course there are other The gadgets written by the big guy, drag in the M3U8 file and handle it directly for you.
An example of the approximate format of the contents of an M3U8 file is as follows:
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-ALLOW-CACHE:YES
#EXT-X-KEY:METHOD=AES-128,URI="https://www.example.com/m3u8.key"
#EXT-X-TARGETDURATION:5
#EXTINF:4.200000,
https://www.example.com/hls/live_00000.ts
#EXTINF:4.166667,
https://www.example.com/hls/live_00001.ts
#EXTINF:3.600000,
https://www.example.com/hls/live_00002.ts
#EXTINF:2.516667,
https://www.example.com/hls/live_00003.ts
#EXTINF:4.166667,
https://www.example.com/hls/live_00004.ts
#EXTINF:4.166667,
https://www.example.com/hls/live_00005.ts
#EXTINF:4.166667,
https://www.example.com/hls/live_00006.ts
#EXTINF:1.716667,
https://www.example.com/hls/live_00007.ts
#EXT-X-ENDLIST
The meaning of each label is as follows:
#EXTM3U: m3u file header, which must be placed on the first line to mark it;#EXT-X-VERSION: Compatible version of the playlist file. If this flag is not present, it defaults to the first version of the protocol;#EXT-X-MEDIA-SEQUENCE: Each media URI in the playlist has a unique integer sequence number. The sequence number of a URI is equal to the sequence number of the URI preceding it plus one;#EXT-X-ALLOW-CACHE: Indicates whether the client can cache downloaded media segments for later replay;#EXT-X-KEY: The TS segment can be encrypted. This tag specifies the encryption method (METHOD), the URI of the key, and the offset IV and other information. Without this tag, it means that it is not encrypted;#EXT-X-TARGETDURATION: the maximum duration of each TS media file, in seconds;#EXTINF: Detailed information of each media file, including media duration, media URL address, etc.;#EXT-X-ENDLIST: Indicates that the media clip will no longer be added to the playlist file, usually at the end of the file.
For the complete format and standard tags, please refer to the introduction to the Playlist file in the HLS standard protocol: https://datatracker.ietf.org/doc/html/draft-pantos-http-live-streaming-08
SAZ analysis
In the Fiddler software, the SAZ format is used to save and read HTTP/HTTPS request information. When opening the file, you can notice some important requests: script.bundle.js, live.m3u8, drm, and eight ts video stream files.
Let's take a look at the m3u8 file first, you can see that it is AES-128 encryption, and the encrypted key file address is key://live, as shown in the following figure:
Under normal circumstances, if you want to decrypt ts, you must request the address of the key, and then decrypt the ts after you get the key. Obviously, the key address of this question is not a legal URL address. Of course, the packet capture record of this question may be out of question. It was faked, because this Host is 52tube.mmxxii, and it is not a legitimate domain name. The most important thing is that there is no key://livesuch , so there is a high probability that the real address is hidden in JS, and it comes from another aspect. Think, if this is a complete packet capture record, no matter what the real key address is, it will definitely appear in the record!
Experienced friends should be able to see at a glance that the request for drm is most likely to be a key operation. First, the keyword drm often appears in ts decryption. Friends who have done more should have seen it a lot. The success returned by the second ping request does not look like the key by its name and return value, and only drm is left. Check that the return value is garbled, check the Hex value, 32-bit hexadecimal data, and the normal key It should be 16-bit hexadecimal data, so if you use this data as a key to decrypt it, it will definitely fail.
Here we should have the following conjecture: the data returned by drm can get the correct key after script.bundle.js secondary processing.
JS reverse
We save the script.bundle.js of the captured package record, right click, save - response - response body, and save it locally.
After formatting, there are 15000+ lines of code, and it cannot be dynamically debugged. Where can I find the encryption entry? You can try it boldly:
- JS may detect that there is a key URI in m3u8, send /api/drm/ this request, you can directly search
/api/drm/orkey://livelocate; - drm is a post request with two parameters h and id, which can directly search
post, locate,idandhlocate the approximate location.
The following suspicious code snippets can be found by searching:
Refine the key code:
function n(t) {
return [...new Uint8Array(t)].map((t => t.toString(16).padStart(2, "0"))).join("")
}
function s(t, e) {
let r = new Uint8Array(t.length);
for (let i = 0; i < t.length; i++) r[i] = t[i] ^ e[i];
return r
}
let e = "/api/ping/",
i = "/api/drm/";
class a extends t.DefaultConfig.loader {
let e = await async function() {
let t = new Uint8Array(16);
crypto.getRandomValues(t);
let e = n(t.buffer) + Date.now() + Math.random();
return new Uint8Array((await async function(t) {
const e = (new TextEncoder).encode(t);
return await crypto.subtle.digest("SHA-256", e)
} (e)).slice(0, 16))
}();
var r = new URLSearchParams;
r.append("h", n(e.buffer)),
r.append("id", t);
var a = {
method: "POST",
headers: {
"Content-Type": "application/x-www-form-urlencoded"
},
body: r
};
let o = await fetch(i, a),
l = await o.arrayBuffer();
if (32 !== l.byteLength) throw new Error("Invalid response");
let u = new Uint8Array(l.slice(0, 16)),
c = new Uint8Array(l.slice(16, 32));
return s(s(u, e), c)
}
It can be seen that in fact, after sending the /api/drm/request and getting the result, the 16-bit data before and after are taken successively, and then processed by the s method, and finally the returned s(s(u, e), c)key should be the correct key. The key here is the value of e. There is a method above. , take the current time + random value, encrypt it with SHA-256, and then take the first 16 bits.
You can think about it here. The value of e is not fixed, so the last key should also be not fixed. There are countless keys corresponding to the same TS. I have never seen it anyway. If you don’t believe me, try to generate e using that method. You will find that the final key is wrong.
Take a closer look at the place where the value of h is assigned when sending a post request: r.append("h", n(e.buffer)), the n method is to convert to hexadecimal, then we directly push the value of h backwards, from hexadecimal to decimal, this is the correct value of e ! Then the value of l is the 32-bit hexadecimal data returned by the /api/drm/request and converted to decimal, and the rest is easy to say, just rewrite the JS code to get the correct key:
function s(t, e) {
let r = new Uint8Array(t.length);
for (let i = 0; i < t.length; i++)
r[i] = t[i] ^ e[i];
return r
}
function getKey(){
// /api/drm/ 请求表单的 h 值，16进制数据
const h = ["7b", "10", "31", "1e", "6e", "31", "0f", "0d", "f0", "68", "d9", "ed", "e1", "04", "75", "a8"];
// /api/drm/ 请求返回的32位16进制数据
const drm = ["08", "A5", "E6", "C2", "C2", "61", "A8", "AC", "B4", "D7", "9C", "49", "AF", "16", "0A", "3A", "DA", "4E", "5C", "EA", "E1", "6F", "ED", "46", "EB", "6F", "49", "8C", "9B", "63", "D5", "3B"]
// 转换为10进制数据，为 e 和 l 赋值
const e = [];
const l = [];
for (let i=0; i<h.length; i++)
{
e.push(parseInt(h[i],16))
}
for (let i=0; i<drm.length; i++)
{
l.push(parseInt(drm[i],16))
}
const u = new Uint8Array(l.slice(0, 16));
const c = new Uint8Array(l.slice(16, 32));
const keyArray = s(s(u, e), c);
const keyHex = new Buffer.from(keyArray).toString('hex');
const keyBase64 = new Buffer.from(keyArray).toString('base64');
console.log("keyArray: ", keyArray)
console.log("keyHex: ", keyHex)
console.log("keyBase64: ", keyBase64)
}
getKey()
// 输出
// keyArray: Uint8Array(16) [
// 169, 251, 139, 54, 77,
// 63, 74, 231, 175, 208,
// 12, 40, 213, 113, 170,
// 169
// ]
// keyHex: a9fb8b364d3f4ae7afd00c28d571aaa9
// keyBase64: qfuLNk0/Suev0Awo1XGqqQ==
TS decrypt merge transform
Through JS reverse, we got the key in hexadecimal and base64 form, which can be decrypted in any form. Here are two methods for decrypting, merging, and converting TS media streams.
The first method is to use the FFmpeg tool, an open source computer program that can be used to record, convert digital audio, video, and stream it. Official website address: https://ffmpeg.org/ , download the compiled program, and add the bin directory to the environment variable. The tool can also be obtained directly by replying to M3U8 in the background of the K-brother crawler official account.
First of all, we need to save the m3u8 file and the ts media stream to the same folder. Because it is a fake Host, it cannot be accessed and saved directly in the browser. You can directly in Fiddler, right-click, save - response - response body, save it locally, As shown below:
The next step is to save the key file, which requires that the key file must be hexadecimal data. If you directly save the key as a string, decryption will also fail. There are special tools for editing hexadecimal files, such as HxD, 010 editor, winhex, etc., take HxD as an example, create a new file, write the hexadecimal data of the key we obtained through JS reverse, and save it as a .key file, as shown in the following figure:
Then modify the address and name of the key in the m3u8 file. It is recommended to put the key, m3u8 and ts files in the same folder, so that there is no need to add resource paths in the m3u8 file, and it is not easy to make mistakes.
Then in the current folder, open the command line and enter the command: ffmpeg -allowed_extensions ALL -i live.m3u8 -c copy live.mp4, the ts can be automatically decrypted and converted into .mp4 format:
The second method is to use a third-party gadget written by the boss. Here I recommend the M3U8 batch downloader written by Xiaoyao Yixian, the boss of my love. See the original post for the download address and usage method: https://www.52pojie.cn/ thread-1374045-1-1.html , you can also reply to M3U8 in the background of Brother K's reptile official account to get it.
We can directly drag in the processed M3U8 file and process it automatically:
You can also choose Other - Tools - Merge Assistant, add all TS files, and process them automatically after entering the key:
After processing, the mp4 file is in the output folder of the software directory by default. After decryption, it is an animation. Looking back, you will find the flag: flag{like_sub_52tube}is the correct answer.
