Java 2023 seckill project day(1) face-to-face experience

1. Spike project

1.1 How to design the seckill system

• high availability
• High performance: support high concurrent access
• Consistency: seckill must maintain data consistency

1.2 Database

• Character set: utf8mb4
  can store emoji expressions

2. Business

2.1 Login

2.2.1 Password Encryption

• MD5(MD5(password+fixed salt)+random salt)
• The front end uses a fixed salt for the first encryption, and the back end uses a random salt for the second encryption.

2.2.2 Password parameter verification

• pom

 <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-validation</artifactId>
        </dependency>

• custom annotation

@Target({
    
    ElementType.METHOD, ElementType.FIELD, ElementType.ANNOTATION_TYPE, ElementType.CONSTRUCTOR, ElementType.PARAMETER, ElementType.TYPE_USE})
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Constraint(
        validatedBy = {
    
    
            IsMobileValidator.class
        }
)
public @interface IsMobile {
    
    
    //该字段必填

    boolean required() default true;

    String message() default "手机号码格式错误";

    Class<?>[] groups() default {
    
    };

    Class<? extends Payload>[] payload() default {
    
    };
}

package com.example.seckilldemo.validator;

import com.example.seckilldemo.utils.ValidatorUtil;
import org.thymeleaf.util.StringUtils;

import javax.validation.ConstraintValidator;
import javax.validation.ConstraintValidatorContext;

/**
 * 手机号码校验规则
 *
 * @author: LC
 * @date 2022/3/2 3:08 下午
 * @ClassName: IsMobileValidator
 */
public class IsMobileValidator implements ConstraintValidator<IsMobile, String> {
    
    

    private boolean required = false;

    @Override
    public void initialize(IsMobile constraintAnnotation) {
    
    
//        ConstraintValidator.super.initialize(constraintAnnotation);
        required = constraintAnnotation.required();
    }

    @Override
    public boolean isValid(String s, ConstraintValidatorContext constraintValidatorContext) {
    
    
        if (required) {
    
    
            return ValidatorUtil.isMobile(s);
        } else {
    
    
            if (StringUtils.isEmpty(s)) {
    
    
                return true;
            } else {
    
    
                return ValidatorUtil.isMobile(s);
            }
        }
    }
}

2.2.3 Distributed sessions

• The request is distributed to different tomcat servers through the distributor. Some tomcats store user-related session information, while others do not. When the request is distributed to a tomcat server without session information, the user needs to log in again.

2.2.3.1 Solutions

• Session replication: it will increase storage, only tomcat configuration is required
• Back-end centralized storage will increase complexity

2.2.4 Parameter Parser

• configuration

package com.example.seckilldemo.config;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.method.support.HandlerMethodArgumentResolver;
import org.springframework.web.servlet.config.annotation.EnableWebMvc;
import org.springframework.web.servlet.config.annotation.InterceptorRegistry;
import org.springframework.web.servlet.config.annotation.ResourceHandlerRegistry;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurer;

import java.util.List;

/**
 * MVC配置类
 *
 * @author: LC
 * @date 2022/3/3 2:37 下午
 * @ClassName: WebConfig
 */
@Configuration
@EnableWebMvc
public class WebConfig implements WebMvcConfigurer {
    
    

    @Autowired
    private UserArgumentResolver userArgumentResolver;
    @Autowired
    private AccessLimitInterceptor accessLimitInterceptor;

    @Override
    public void addArgumentResolvers(List<HandlerMethodArgumentResolver> resolvers) {
    
    
//        WebMvcConfigurer.super.addArgumentResolvers(resolvers);
        resolvers.add(userArgumentResolver);
    }

    //静态资源展示
    @Override
    public void addResourceHandlers(ResourceHandlerRegistry registry) {
    
    
        registry.addResourceHandler("/**").addResourceLocations("classpath:/static/");
        //swagger 和 knife4j
        registry.addResourceHandler("swagger-ui.html").addResourceLocations("classpath:/META-INF/resources/");
        registry.addResourceHandler("doc.html").addResourceLocations("classpath:/META-INF/resources/");
        registry.addResourceHandler("/webjars/**").addResourceLocations("classpath:/META-INF/resources/webjars/");
    }

    @Override
    public void addInterceptors(InterceptorRegistry registry) {
    
    
        registry.addInterceptor(accessLimitInterceptor);
    }
}

• Parameter parsing implementation

package com.example.seckilldemo.config;

import com.example.seckilldemo.entity.TUser;
import com.example.seckilldemo.service.ITUserService;
import com.example.seckilldemo.utils.CookieUtil;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.MethodParameter;
import org.springframework.stereotype.Component;
import org.springframework.web.bind.support.WebDataBinderFactory;
import org.springframework.web.context.request.NativeWebRequest;
import org.springframework.web.method.support.HandlerMethodArgumentResolver;
import org.springframework.web.method.support.ModelAndViewContainer;
import org.thymeleaf.util.StringUtils;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

/**
 * 自定义用户参数
 *
 * @author: LC
 * @date 2022/3/3 4:46 下午
 * @ClassName: UserArgumentResolver
 */
@Component
public class UserArgumentResolver implements HandlerMethodArgumentResolver {
    
    

    @Autowired
    private ITUserService itUserService;

    @Override
    public boolean supportsParameter(MethodParameter parameter) {
    
    
        Class<?> parameterType = parameter.getParameterType();
        return parameterType == TUser.class;
    }

    @Override
    public Object resolveArgument(MethodParameter parameter, ModelAndViewContainer mavContainer, NativeWebRequest webRequest, WebDataBinderFactory binderFactory) throws Exception {
    
    

        return UserContext.getUser();

        //        HttpServletRequest nativeRequest = webRequest.getNativeRequest(HttpServletRequest.class);
//        HttpServletResponse nativeResponse = webRequest.getNativeResponse(HttpServletResponse.class);
//        String userTicket = CookieUtil.getCookieValue(nativeRequest, "userTicket");
//        if (StringUtils.isEmpty(userTicket)) {
    
    
//            return null;
//        }
//        return itUserService.getUserByCookie(userTicket, nativeRequest, nativeResponse);
    }

}

2.3 Exception handling

2.3.1 ControllerAdvicer+ExceptionHandler

Handle exceptions thrown by the controller.

2.4 Spike

2.4.1 Logic

2.4.1 Judgment before flash kill

• If it is within the seckill range, there will be a seckill button on the front end
• Determine whether the user is logged in
• Judging whether the stock of flash sale products is sufficient
• To determine whether the user has repeatedly placed an order, check the seckill order table
• Perform seckill, reduce inventory, then generate seckill order form, and jump to the order details page

2.4.2 Perform seckill

• Modify the library from the need to judge the library from greater than 0 before modifying it: use the update statement

boolean seckillGoodsResult = itSeckillGoodsService.update(new UpdateWrapper<TSeckillGoods>()
                .setSql("stock_count = " + "stock_count-1")
                .eq("goods_id", goodsVo.getId())
                .gt("stock_count", 0)
        );

• Generate Lightning Deal Details Order

2.5 Business Process

• The user logs in and enters the user name and password, the password is encrypted with a fixed salt at the front end, and then transmitted to the back end, and the back end verifies the password and user name. After the verification is passed, cookie information is generated for the user, the cookie is returned, and the user information is stored in redis.

• After successful login, go to the flash sale product list page, select the flash sale product, enter the product details page, and click the flash sale product to perform the flash sale.
  

• The flash sale is showing that it is in progress

• Seckill successfully enters the product order details page
  

• product detail page

2.6 redis information

• 
  The key timeout time of the verification code during seckill is 300s

captcha+userID+goodsID

2.7 Flash Kill Process

• Determine user login information
• Judging whether the flash sale products in the memory tag are enough: Controller implements the afterPropertiesSet method of InitializingBean, reads the flash sale product information into the memory tag, and stores the flash sale product information in redis.

 HashMap goodsStockMap = new HashMap<Long, Boolean>();

• Judging whether the seckill is repeated: query the order information in redis, if there is no, there will be no repeated seckill.
• Redis pre-decreases the inventory, and sends a rabbitmq message after the pre-reduction succeeds.
• The mq receiver consumes mq messages, judges whether to buy repeatedly, judges the inventory of the product, and then generates a second kill order, and generates a second kill order for product details. Store the seckill order information in redis.

2.8 Interface Current Limiting

• Limit the current for the uri requested by the user and the user's id, enter the maximum number of current limits, and implement the current limit through redis.
• Get the key for the first time, the key is null, set the key and the value is 1
• For each subsequent visit, the number of times will be increased by 1, and if the number of times exceeds, the return information will be written into response

 private void render(HttpServletResponse response, RespBeanEnum respBeanEnum) throws IOException {
        response.setCharacterEncoding("UTF-8");
        response.setContentType("application/json");
        PrintWriter printWriter = response.getWriter();
        RespBean bean = RespBean.error(respBeanEnum);
        printWriter.write(new ObjectMapper().writeValueAsString(bean));
        printWriter.flush();
        printWriter.close();
    }

Three, the problem

3.1 The front-end cannot transfer flash kill inventory

The data passed by the front end may be modified

3.2 Ask why the newly created seckill table does not add fields to the original table

• If there are many flash sale activities, it is troublesome to modify the original table every time, and the original price and flash sale price of the item may be used at the same time, so it is easy to make mistakes when modifying the original table.

3.3 Packaging

• The maven plugin is required here
• Remove test
  mvn clean
  mvn package
• java -jar jar package running

 <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <configuration>
                    <excludes>
                        <exclude>
                            <groupId>org.projectlombok</groupId>
                            <artifactId>lombok</artifactId>
                        </exclude>
                    </excludes>
                </configuration>
            </plugin>
        </plugins>
    </build>

4. Pressure test

4.1 Test product details interface

4.1.1 Test conditions

Generate 5000 users and store the user's cookie in redis.

4.1.2 Query file list

4.2 Single-service spike test

4.2.1 The original seckill test does not have any optimization strategy

• jmeter result
• oversold
• There is no transaction added at this time, and it is not judged whether the inventory is greater than 0 before deducting the inventory
  
  

4.2.2 Tests without using mq

• Do not use mq, do not use memory tags, do not use redis to pre-decrease inventory
  

• Do not use mq, do not use memory tags, use redis to pre-decrease inventory
  

• Do not use mq, use memory mark, use redis to pre-decrease inventory
  

4.2.3 Test after using mq

• Use mq, use memory tags, use redis to pre-decrease inventory
  
• 50000 thread test results
  

4.3 Multi-service spike test

Five, spike optimization

5.1 Page Optimization

5.1.1 Page cache (the entire html is returned, the front and back ends are not separated)

After the page is rendered, it is cached in redis, and the expiration time T is set, so that the user gets the page in T. In this way, the page does not need to be rendered every time within T time, which reduces the time. T can not be too long, can not be too short.

5.1.2 URL cache

It is similar to page caching, except that the url will have dynamic parameters, so it will be cached a little more. Cache according to different dynamic parameters.

5.1.3 Static page (separation of front and back end)

The backend only needs to pass data to the frontend, and the backend does not need to return the entire html page at this time.

5.2 Single-service seckill backend optimization

5.2.1 Inventory reduction optimization and correction

• add business
• Judging that the inventory is > 0, the inventory reduction operation is performed
• Add a unique index to prevent users from placing repeated orders. This repeated order is to prevent users from placing orders with the same user id and product id. The above prevention of repeated orders is to prevent users from placing multiple different orders.

5.2.2 User repeated seckill problem

• There are two types of repeated flash sales, one is for the same product, and the unique index of user ID and product ID is established to prevent repeated flash sales.

5.2.3 Using redis optimization

• Add the product information to redis before the seckill, pre-decrease the inventory , and generate a seckill message at the same time, send it to the mq queue, and return the information to the front end. The front end displays the page in the seckill according to the returned structure.
• Use memory tags to reduce access to redis. It used to be necessary to access redis when the inventory was insufficient. Now set additional variables to access the variables first. When the inventory of the variable is insufficient, it returns directly, and there is less access to redis.

5.2.4 Acquisition of mq seckill information

• If the database queries the order of seckill, the seckill is successful and returns 1
• If the database cannot query the order of the flash sale and the inventory of the product in redis is less than or equal to 0, the flash sale will fail
• Spike is in progress

6. Technical role

6.1 redis

• Store product inventory information
• Pre-decrement inventory operation
• Store the order information of the user's seckill, and judge the repeated seckill
• Auxiliary verification code verification

6.2 ThreadLocal

• Each ThreadLocal has its own ThreadLocalMap, which stores data in the ThreadLocalMap in the form of key and value.
• The interceptor parses and stores the user information through the threadlocal object, and takes out the user information in the parameter parser for parameter parsing. At this time, the threadlocal object is used to store a copy of the user information of each request processing thread to realize the isolation of data access.
  threadlocal role

6.3 Rabbitmq

• Asynchronous order
• Shaving peaks and filling valleys

6.4

7. Security Optimization

7.1 Hide interface address

7.2 Verification code

7.3 Interface current limiting

7.3.1 Current limit standard

70%~80% of the maximum qps

7.3.2 General Current Limit

• Implement current limit on users based on user id
• custom annotation

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public @interface AccessLimit {
    
    

    int second();

    int maxCount();

    boolean needLogin() default true;
}

8. Questions

8. 1 Products sold less

Initially generating two copies of the same product information in redis will result in fewer products being sold. But the inventory is still at 0? ? ?

Nine, sql statement

9.1 Empty table statement

DELETE  FROM t_order WHERE goods_id = 1;
DELETE FROM `t_seckill_order` WHERE goods_id = 1;