Any business will face a large number of hackers after operating for a period of time. The relationship between verification code and various crawlers is like cat and mouse, and they will continue to play games forever. Based on eleven years of experience in fighting against black industry, Jiexian divides the cracking methods of black industry into three categories:
1. Batch cracking verification is achieved by identifying the answer to the verification code picture, is the picture answer recognition;
2. After understanding the communication process, directly send a request with relevant parameters for interaction,that is, protocol cracking;
3. Use various client simulators to simulate real people passing verification, that is, simulator cracking.
For the above three cracking methods, Jiexian summarized the corresponding verification code attack and defense points one by one:Picture answer game, are used to protect Jiexian customer-related scenarios and interfaces safety. Environment detection and Protocol cracking detection
In the previous two issues, we have introduced the first major attack and defense point with black products from picture resource traversal and picture answer identification respectively: the picture answer game.
In this issue, we continue to introduce the second major attack and defense point against black industry: protocol cracking confrontation. Protocol cracking is the most commonly used cracking method for black products. Let’s first talk about how to detect protocol cracking.
One,Route
What is protocol cracking
In order to achieve the purpose of brushing through verification, all black-produced crawlers will simulate the request characteristics of real people and directly access the communication interface by forging corresponding requests and parameters. This attack method is called protocol cracking.
In order to better understand protocol cracking, let’s take an example: in June 2023Mayday Concert Abnormal Order Incident , which involves a similar problem of forged parameters.
After the two passionate ticket-grabbing battles in Wuhan and Shenyang on May Day, in order to maintain the fairness of ticket-grabbing, a certain ticketing platform F banned third-party bidding, third-party ticket-grabbing and packet-capturing software, and those who did not use the official APP/mini-program. Due to a single violation of the script, a total of 455 abnormal orders (a total of 754 tickets) were refunded, which caused dissatisfaction among many fans.
Regarding the complaints from "innocent" fans whose tickets were refunded, the platform responded: Normal orders placed through the official APP/mini-program should include the following complete order flow parameters, while abnormal orders lack some core parameters.
To put it simply, the ticketing platform has set a non-required phishing parameter in the request parameters in advance, through the official APP/ Normal order requests from the mini program will automatically carry this parameter. Although the script packet capture tool will simulate a normal request and seems to be the same as a normal request, it will skip this core parameter because it is not required.
As shown in the picture above, the abnormal order that does not use the official APP/mini program normally lacks the meaningless phishing parameter "119533". This is how this ticketing platform detects script cracking.
When the black product simulates a real person request, as long as we set a new fishing parameter, the originally fixed script of the black product will be "exposed" due to the lack of this new parameter. Although black products missing core parameters can still pass the verification normally (same as abnormal orders running scripts can still be successfully submitted and paid), we have been able to intuitively distinguish the black product account Distinguish it from the normal account, so as to know the whereabouts of illegal products in real time for subsequent processing (the same as the platform's forced refund of tickets). Therefore, the method of adding new parameters to mark abnormal accounts can greatly reduce the losses caused by the cracking of hidden black industry protocols, and achieve entrapment strikes from defense to countermeasures.
2. AttackerPerspective (Black Production)
From the perspective of the attacker, how exactly does the black industry crack the protocol and forge parameters?
Attack process
All requests for verification codes are HTTP requests. HTTP (Hypertext Transfer Protocol) is a simple request-response protocol used for client-server communication. It specifies what kind of messages the client may send to the server and what kind of response it gets.
All crawlers will simulate the protocols of real people. It is our shadow and will simulate all the request characteristics of our real people. Black products who want to crack the front-end and back-end communication usually forge HTTP protocol request parameters.
The black industry’s specific process for cracking protocols and forging parameters is as follows:
Step 1: Use browser F12 to open the developer tools panel and capture network packets.
Step 2: Enter the target page, observe the network request list, and find the key requests. (Key requests will vary depending on different businesses, so specific analysis is required)
Step 3: Open the key request details, observe the request protocol, and analyze the request parameters.
1. Actively input parameters, such as account password, email, mobile phone number, answer, etc.
2. Parameters with clear semantics, such as timestamp, client type, business id, business serial number, etc., usually do not change after multiple requests, or can be extracted from previous requests.
3. Request header parameters, such as Referer, Ua, Cookies, etc.
4. Signature and encryption parameters.
Step 4: Reverse parameter generation logic.
The key to protocol cracking is the last category Signature and encryption parameters. The generation of such parameters is accompanied by one-way hashing, encryption and other logic, and coupled with client code obfuscation, the parameter generation logic will be hidden deeply, and it is necessary to cooperate with breakpoints and call stacks for debugging and analysis.
Step 5: Script disguises client logic.
Use code to fix IP proxy, parameter forgery, HTTP request and other logic into a script. It is separated from the client and skips page rendering and interaction processes to maximize efficiency.
Ways to make profits
From the perspective of black industry, why is protocol cracking their most commonly used cracking method?
We mentioned earlier that image answer recognition requires batch crawling and downloading of image galleries, manual coding, exhaustive recognition/training models, etc. A series of processes are required to obtain verification answers, which requires black production to invest a lot of time and energy in the early stage. It takes 8.33 hours for black products to download a batch of 300,000 verification pictures; and manual coding to obtain the answers costs thousands of yuan and consumes Time8~10 days, the efficiency is relatively low.
What I will talk about lateremulator cracking is also limited by environmental conditions. It requires hackers to have a good understanding of browsers and use various Automated testing tools, such as Selenium, operate the chromium kernel to realize automated dragging, clicking and other operations. The attack process is more complex and the technical threshold is higher.
In contrast,the biggest advantage of protocol cracking is low cost and high efficiency. Black products do not need to spend a lot of money. They only need to enter the target page and simulate the core key parameters to crack the verification code. The execution efficiency of the entire process is higher. After the script is written and fixed, it can be purchased for ranging from a few hundred to tens of thousands of yuan per time Sell it to people in need and earn huge profits in a short period of time, and the income is far higher than the cost. Therefore, most hackers will choose protocol cracking when attacking.
Of course, while protocol cracking is highly efficient, it also has an inevitable drawback: it is difficult to forge parameters. Protocol cracking requires hackers to parse the front-end source code of the verification code in order to forge the parameters sent by the front-end, so a certain technical threshold is also required.
3.The perspective of the attacked party (customer)
From the customer's perspective, if a protocol cracking attack is encountered, what abnormal changes will occur to the data?
Impact of protocol cracking on customers
G, a customer in the information industry, suffered long-term crawler attacks to crawl information in the information query scenario. On August 22, 2023, G company’s backend query volume suddenly increased sharply, and the number of verification requests increased significantly, reaching about 7,000 interactions per hour. There were a large number of abnormal data attacks during this period, which caused huge losses to the customer's business.
And, from the IP dimension, among the more than 40,000 interactions between 0:00 and 9:00, the maximum number of visits to a single IP reached 28 times< a i=2>, indicating that the crawler has controlled the IP access frequency! This will pose a greater threat to customer business.
CT hit volume: Positioning protocol cracked
After being attacked by a crawler, Company G immediately found Jiyan for help. After checking the background data, Jiexin security experts found that the volume of verification requests, verification interactions, and verification defenses were all very high, indicating that black products frequently requested verification and retried after verification failed, or re-collected verification resource information for coding attacks. Judging from these verification amounts alone, there is not much difference from the data of the previous picture answer recognition attack, and it is impossible to accurately identify the attack method of black products.
Background data that suffered image answer recognition attacks
Background data encountering protocol cracking
After being unable to discern any obvious difference from the image answer recognition attack from these verification quantities, Jiexian security experts opened the purple one aloneCT (captcha token) hits, this is Jiexian’s unique “abnormal marking” function, which is specially used to locate illegal protocol cracking. The principle is mainly to detect changes in front-end js and transformation parameters to detect whether there is a black product that has cracked the front-end protocol.
If the number of CT hits is high, it means that the black industry is conducting a protocol cracking attack and hits ourfront-end "abnormal mark".
View CT hits individually and locate protocol cracks
After discovering that the number of CT hits reached 20,000, Jiexian security experts began to block and intercept protocol cracking requests at 11 o'clock on August 23. The number of successes began to decrease, and the number of failures and interactions began to increase, indicating that black products were indeed being processed at this time. Perform protocol cracking.
During this period, the crawler continued to organize reverse cracking, and the interception effect lasted until September 5. On the same day, the number of verification passes and the amount of verification interactions continued to increase, indicating that the crawler had already The crack was completed for this version of the verification protocol.
4.Defender’s perspective (extreme experience)
We learned earlier that since the principle of cracking black protocols is to simulate real person requests and forge real person agreements, how can the defender defend it? The key is to: Distinguish these fake and dangerous requests.
defensive ideas
How to distinguish black products pretending to be normal users? Although the black industry has simulated real people in terms of behavior, equipment, IP and other dimensions, and seems to have a real identity, there is one most obvious difference: the starting point of the black industry is always to obtain the maximum benefit. ization, so the process will be automated and the cracked and reused protocols will be solidified into a set of automated scripts. If there are no problems with the script or no one reports any abnormalities, no cost will be spent on updating.
Therefore, we can use this to design traps:In order to pass the verification, black products will simulate real-person requests, so we only need to add a phishing parameter to the request. Since the script used by hackers after cracking the protocol has already been solidified, it will not carry this new phishing parameter, so the parameters will be missing and the identity will be exposed. This principle is very similar to the ticketing platform’s detection of abnormal orders mentioned at the beginning of this article. Requests issued by script capture tools often lack core phishing parameters.
defense strategy
Aiming at the illegal protocol cracking, Jiexian’s response strategy is based on the existing request parameters, in the front end< a i=3>Add a new fishing parameter. If it is a user from a normal browser, this new parameter will be automatically carried after the update; while black products that have cracked the protocol will continue to use the original solidified protocol and will not Carrying new parameters. This can effectively detect black products: if the request does not carry new parameters and only carries old parameters, then it is very likely that it is a black product that has cracked our protocol before. .
The process of adding new parameters and distinguishing black products is the "abnormal mark" function in our protocol cracking detection. Through the parameter Change comparison allows us to distinguish abnormal black products. Moreover, the detected black products can still pass the verification normally, but we can intuitively distinguish the black product accounts from normal accounts, so as to secretly observe the whereabouts of the black products in real time.
"Exception Marking"The function is also very simple to implement:
Step 1: Just click on the backend of Jiexian to configure new parameters (the example version is v1.7.4-c6515a). The background usually reserves about 50 new parameters in advance so that they can be dynamically updated at any time after being cracked.
Step 2: Under the v1.7.4-c6515a version, users who normally make requests through the client will automatically carry the newly configured parameters "9pI3": "7k9E".
The black production request initiated by the script will lack the newly configured parameters "9pI3": "7k9E" and only carry the old parameters of the previous batch, which is different from the normal request.
Breaking is always inevitable. Black products will definitely crack our front-end protocol, and will constantly change IPs, devices, and modify behavioral parameters. The reason is Because the front-end code has been disclosed to the public, whether it is ours or any other competing product, including Google's recaptcha, everyone will inevitably be cracked.
Passive defense will always lag behind the black production, and the "soldiers will block the water and the soil will cover it" approach will make the defender exhausted and operational pressure is huge. Therefore, we must jump out of the various data parameter dimensions of black products imitating real people and actively distinguish black products.
problem solved
Currently, the "abnormal mark" function (captcha token, CT) in the fourth generation verification code of Jiexian can be changed through the front-end js And transform parameters to detect whether there is a black product that has cracked the front-end protocol and whether the verification interaction protocol needs to be readjusted. As long as the protocol is cracked, it will definitely be recognized by the next update of Jiexian.
In response to the crawler problem encountered by G customers, Jiexian passed the protocol crack detection and dynamically updated parameters. After the crawler reversely cracked the previous version of the verification protocol, it started to update the new verification protocol at 9:30 on September 12. You can see the update. Changes in pre- and post-protocol crack hits.
After that, the protocol crack detection hits continue to take effect. Even if the crawler cracks again, the parameters can be updated at any time to complete the crack detection, and the customer's data finally returns to normal.
technological breakthrough
In addition to helping G customers solve crawler problems, we have also made dynamic parameter updates into a corresponding backend tool, which can help all kinds of customers respond quickly to solve problems when they arise. The JiExperience technology that customers learn about in daily operations may be just the tip of the iceberg, and our real development strength behind the verification code is the more solid iceberg under the water.
In the entire process of protocol cracking and identification, Jiexian has achieved the following technical breakthroughs:
1) JS confusion
It has self-defense capabilities by obfuscating js code, converting JavaScript code into a form that uses automatic analysis attacks and prevents reverse engineering.
2) Parameter encryption
The front-end parameters are encrypted with complex encryption algorithms such as SHA256, and the trajectory and answer location are encrypted. The parameters cannot be read at all, nor can the meaning of the values passed in the parameters be known, and cannot be cracked by forging parameters.
3) Parameter confusion
JiExperience also added some confusion parameters to the front-end parameters. Some of the data is actually collected data, and some of the data is honeypot parameters. The cracker cannot distinguish which are necessary parameters and which are honeypot parameters.
4) Agreement update
Jiexian updates the protocol from time to time every day, and the entire network can be updated with one click and take effect within seconds. The protocol update instantly marks black script requests and collects a large amount of black sample data.
5. Conclusion
Data information always has lag. Most manufacturers still use existing databases/black and white lists to identify problematic accounts when distinguishing black products. However, the database is often based on past data. For the latest black products, or after the black products change their equipment, accounts, and IPs, it will be difficult to identify problematic accounts. It cannot be identified immediately, so relying on the security database cannot identify all black products.Verification manufacturers have long been difficult to solve the problem of unblocked accounts of black products. .
And now, through protocol cracking detection, Jiexian has been able to accurately identify leaked black products for the first time in the industry. In 2022, Jiexian’s CT “abnormal marking” function will be officially launched, which can mark illegal production requests through dynamic changes in front-end parameters, providing an effective basis for the next step of banning and handling. Up to now, Jiexian’s CT “abnormal marking” function has marked 81.074 billion abnormal cases for service customers. production request.
Again, any business will face a large number of hackers after operating for a period of time, because whether it is the environment or behavior, the final form returned to the server is parameters, and hackers can still calibrate normally through cracks. Verified request parameters to fake real users. Therefore, deploying the verification code as a defensive tool cannot be a one-and-done solution. Verification codes and various black products will continue to play games forever.
As a defensive side, how can we win in this game? The first is to fully understand the attacker's technology; the second is to have more and more efficient defense methods than the attacker on the basis of knowing yourself and the enemy. As we mentioned earlier, various tools and technologies can help customers provide more defense solutions, detect earlier and respond faster during defense. Jiexian adheres to the belief of "thinking for the world, and using it for the world". We do not shy away from talking about cracking, but always stay ahead of black products and actively compete with black products for a better customer experience.
Extremely patented wall
In the next issue, we will continue to introduce the second part of protocol cracking confrontation: POW anti-brute force cracking. Protocol cracking pursues "efficiency". During the protocol cracking process, black products must complete a large number of POW calculations in order to complete a large number of verification interactions, and the difficulty of POW calculations is in the hands of Jiexian. If you are also interested in the next issue, please pay attention to us~
Add Eva WeChat to join the Jiexian readers exchange group