Takahata-style three-grip device：缓存、降级和限流
- Cache: The purpose is to improve the system access speed and increase the capacity that the system can handle. It can be said to be a silver bullet against high concurrent traffic;
- Downgrade: When there is a problem with the service or it affects the performance of the core process, it needs to be temporarily blocked and reopened after the peak or the problem is solved;
- Current limiting: Some scenarios cannot be solved by caching and downgrading, such as scarce resources (flash sales, rush sales), writing services (such as comments, orders), frequent and complex queries (last pages of comments), so there needs to be a way to limit the number of concurrency/requests in these scenarios, that is, current limiting.
Implement service isolation and downgrade based on Hystrix: Provide 线程 pool and 信号量 isolation to reduce the complexity of different services Provide an elegant degradation mechanism for the mutual influence caused by competition among resources.
- Based on thread pool: The advantage is that using thread pool isolation can completely isolate third-party applications, and the request thread can be quickly put back. The request thread can continue to accept new requests, and if a problem occurs, the thread pool isolation is independent and will not affect other applications. When the failed application becomes available again, the thread pool is cleaned up and can be resumed immediately without requiring a lengthy recovery. Separate thread pools improve concurrency. The disadvantage is thatincreases computational overhead (CPU). The execution of each command involving queuing, scheduling and context switching is run on a separate thread.
- Based on semaphore: Record how many threads are currently running. When a request comes in, first judge the value of the counter. If the number of threads exceeds the set maximum number, the request will be rejected. If it does not exceed, it will pass. At this time, the counter will be +1. After the request returns successfully, the counter will be -1. The biggest difference from thread pool isolation is that the thread that executes the dependent code is still the request thread. The size of the semaphore can be dynamically adjusted, but the size of the thread pool cannot. .
Solution to current limiting:
- **Current limiting algorithm:** Such as: token bucket, leaky bucket, counter
- Application layer solution
- Access layer solution (such as Nginx): Nginx access layer current limiting can be implemented using Nginx’s own two modules: the connection number current limiting module ngx_http_limit_conn_module and the leaky bucket algorithm. The request current limiting module ngx_http_limit_req_module. You can also use the Lua current limiting module lua-resty-limit-traffic provided by OpenResty for more complex current limiting scenarios.
- Others (Web front-end optimization): Website dynamic and static separation, JS/CSS compression technology, CDN (CDN acceleration means adding a caching mechanism between the user and our server, Dynamically obtain IP addresses based on geographical location, allowing users to access the nearest server.)
Current limit calculation method:
- Counter: If the value of the counter is greater than 10 and the time interval between the request and the first request is within 1 minute, then there are too many requests. If the time interval between the request and the first request is greater than 1 minute, and the value of the counter is still there Within the current limit range, reset the counter.
- Sliding window counter: Compared with counters, it solves the "critical value" problem.
- Token bucket algorithm: Add tokens to the bucket at a fixed rate. Whether the request is processed depends on whether there are enough tokens in the bucket. When the number of tokens reduces to zero, the new request is rejected (RateLimiter.create(1.0);rateLimiter .tryAcquire(500, TimeUnit.MILLISECONDS);)
- Leaky bucket algorithm: outflow requests at a constant fixed rate, and incoming requests at any rate. When the number of incoming requests accumulates to the leaky bucket capacity, new incoming requests will be rejected.
RateLimiter is an implementation class based on the token bucket algorithm provided by guava. It can use AOP to implement current limiting (annotation: @ExtRateLimiter(value = 10.0, timeOut = 500), rateLimiter.tryAcquire(timeOut, TimeUnit.MILLISECONDS))
Common Web Vulnerabilities:
- XSS attack: Use Javascript script injection to attack. Solution: Special characters in the script are converted into html source code for display.
- SQL injection attack: It exploits the incorrect processing of user input by the application and injects malicious SQL code into the user input to execute malicious database queries. Solution: Use #{} (parsed as a parameter placeholder) instead of ${} (pure string replacement).
- HTTP hotlink prevention: Website A has a picture, which is directly introduced by website B through the img tag attribute, directly stealing the image display from website A. Solution: Determine the value of the record source in the Referer field of the http request header.
- CSRF (Cross-site Request Forgery): Exploiting a trusted website by disguising requests from a trusted user. Solution: Use graphic verification codes to prevent machine simulation interface request attacks. When calling core business interfaces, such as payment, ordering, etc., it is best to use mobile phone SMS verification or face recognition to prevent other users from using Token to forge requests. .
- API idempotent problem: Data may be submitted repeatedly due to repeated clicks, network resending, or nginx resending. Solution: Use token plus redis. In the request header, pass the token parameter, and the token can only be used once.
- Forgot Password Vulnerability: If the verification code number is relatively short, it is easy to use brute force to attack. Solution: Use graphic verification code interception to prevent machine simulation.
- Upload file vulnerability: Upload the Trojan file and get WEBSHELL directly. Verify the file format, set the file permissions, and prohibit execution permissions under the file.
OAuth2.0: A standard protocol for authorization that allows third-party applications to access resources in a restricted manner without obtaining the user's credentials (such as username and password) . The process is as follows:
- Step 1: User agrees to authorize and obtain code
- Step 2: Exchange the code for web page authorization access_token
- Step 3: Refresh access_token (if needed)
- Step 4: Pull user information (need to have scope snsapi_userinfo)
One-way hash encryption:
- MD5: It is a one-way hash algorithm developed by RSA Data Security Company. It is non-reversible. The same plaintext produces the same ciphertext
- MD5 salting: Each time the password is saved to the database, a random 16-digit number is generated, the 16-digit number is added to the password and the MD5 digest is obtained, and then In the abstract, these 16 digits are combined according to rules to form a 48-bit string.
- SHA: Operate data of any length to generate a 160-bit value;
Symmetric encryption: The sender and recipient use a single key that they share. This key is used for both encryption and decryption. It is called secret encryption. key (also called a symmetric key or session key). Common ones include:
- DES (Data Encryption Standard): Block encryption, the algorithm is derived from Lucifer, as the NIST symmetric encryption standard; 64 bits (56 valid bits, 8 parity bits), group algorithm
- 3DES: 128-bit, grouping algorithm
- IDEA (International Data Encryption Algorithm): 128 bits, faster than DES, grouping algorithm
- Blowfish: 32-448 bits, algorithm public, grouping algorithm
- RC4: stream cipher with variable key length
- RC5: block cipher, variable key length, maximum 2048 bits
- Rijndael: 128-bit/196-bit/256-bit
- AES (Advanced Encryption Standard): an upgraded version of DES, the algorithm comes from Rinjindael
Case: Assume that Alice and Bob know each other. In order to ensure that the communication messages are not intercepted by others, the two people have agreed on a password in advance to encrypt the messages sent between them. In this way, even if someone intercepts the message, they cannot know it without the password. The content of the message. Confidentiality is thus achieved.
Asymmetric encryption: One is used to encrypt information, and the other is used to decrypt information. Common ones include:
- RSA
- Elgamal
- knapsack algorithm
- Half
- D-H
- ECC (elliptic curve cryptography)
Example:
HTTPS: To put it simply, it adds secure HTTP, that is, HTTP+SSL. Differences from HTTP:
- HTTPS servers need to apply for a certificate from the CA to prove the purpose of their servers;
- HTTP information is transmitted in clear text, and HTTPS information is transmitted in cipher text;
- HTTP and HTTPS have different ports, one is port 80 and the other is port 443.
Several encryption algorithms of HTTPS:
- The asymmetric encryption algorithm used during the handshake is used to encrypt the request and response after the handshake;
- Symmetric encryption used when transmitting information;
- The hash algorithm (digital signature) is used to ensure data integrity.
HTTPS security process:
