Coupon Program Introduction

      The time is 2018.5.4. While waiting for the defense, the blogger wanted to pay attention to some technologies, so he found a relatively mature background project on github. I plan to read the source code carefully. Since the project is open source, I will introduce this project directly based on the introduction on git. There are ideas worth learning from. I'll sort it out.

The git address of the project is:

https://github.com/lipeiNet/peiyu-mem

Introduction

It mainly realizes the promotion of coupons. First, create the event, then create the coupon group, and use the preprocessing method to make coupons in advance. In the first version, the basic business of the function is mainly realized. Then in branch implementation, large numbers and high concurrency issues.

Branch 1.1

1: To solve the problem of duplication of coupon codes, the original method was to obtain all coupons in the database, and then compare whether they are duplicated. If the amount of database data reaches one million, it will be very slow, and there will be frequent coupon-making failures, etc. Therefore, this version abandons the original random number model and uses Twitter's snowflake algorithm to avoid uniqueness, but still retains the coupon prefix and suffix.
2: The original asynchronous thread is changed to a thread pool, thinking that there are a large number of threads occupying memory space when there is high concurrency.
3: The for loop mode is used to make coupons in batches, and the coupons are inserted by distribution. One batch is currently set at 5000.
4: Join the message queue (using rabbitMQ). If a batch fails to be added, put the failure are put into the paired column, remedied through the queue, and high availability has been reached. Avoid re-importing large quantities of coupons back and forth to the message queue. Refuse to resolve abnormal information and return to the message queue. Configure 2 consumers to avoid one of the service exceptions, and message processing will appear in an infinite loop.

branch 1.2

1: Add operation log
Purpose : Track hot data, query log to quickly track slow queries or slow operations in the application, and lay the foundation for subsequent optimization 2
: Add exception log
Data can be quickly modified
3: Adopt technology to do it through aop and rabbitmq middleware, which reduces the efficiency problem caused by log problems to the program

No optimization efficiency statistics

Use database mysql
data: add 25 valid activities, each activity has 2 coupon groups, and the number of coupons produced under each coupon group is 50,000. There are 2.5 million records in the coupon table.
Business : A member who satisfies these 25 activities at the same time needs to send 50 coupons.
Statistics: After 10 counts of statistics, the entire coupon issuance process consumes about 306s, of which it takes 6s to obtain coupons each time. If multiple cycles will inevitably lead to performance bottlenecks, it takes about 0.5s to
update coupon status. From the above, we can see that our performance problem is mainly in obtaining coupons. Therefore, the 1.3 version mainly solves this problem through the program

branch 1.3

Purpose: Improve performance through program code and database optimization.
Specific solutions:
1: All coupons in the coupon group that were previously acquired are now modified to only get 100 coupons each time (test statistics show that sending 50 coupons consumes 106s, and each coupon It takes more than 2 seconds to obtain coupons each time, and the overall performance is improved by nearly 3 times)
2: Optimize sql and add a composite index (statistically, it takes 2.5 seconds to send 50 coupons, and it takes about 2.5 seconds to obtain coupons each time.) It is 0.015s, and the overall performance is improved by nearly 42 times)
3: Add local cache (if the coupons obtained at one time are put into the map first, then if there are any coupons next time, there is no need to obtain coupons from the library. Statistics found : 10 items, 50 coupons for each item. The efficiency time
without local cache is 7.5s. After adding local cache, it takes about 5.5s, and the overall performance is improved by 2s.)
Effect analysis:
4: For coupon issuance, batches are used. Updated to replace the for loop (from ~5.5s above to ~4.8s)

branch 1.4

Purpose: Issue coupons through asynchronous and message queues
Specific solutions:
1: Issue coupons asynchronously, which can improve the utilization of CPU, and at the same time ensure stability through message queues, and avoid exceptions that lead to the successful return of the front-end coupons, but When an exception occurs during asynchronous coupon making, the efficiency is improved by about 0.5s when 500 coupons are issued.
2: Refactor the code
once . The responsibility of this method is to obtain activities, and at the same time, each small method has a return value as much as possible, which can increase the readability of the code

The architecture of the entire project belongs to SOA architecture, multi-module project. The details are as follows:

Among them, peiyu-men belongs to the parent module, and a series of required jar packages for each module are defined in the pom file.

The commen module contains some common system constants and tool classes (date string conversion tool, list tool, snowflake algorithm tool, etc.)

The dao module is to persist data.

The domain module is some entities that facilitate data transmission, data display, and facilitate the correspondence between data and database.

The manager module is equivalent to the Controller layer in the MVC architecture. It is mainly to receive the business request of the Service, operate the business process, complete the interaction with the DAO layer, or display data to the Service layer.

The rabbitmq module mainly completes the asynchronous message queue. I will introduce his introduction and use later.

The redis module is mainly used for cache processing.

The rpc module is mainly to complete the remote call between modules. Dubbo is used to realize the remote call, mainly the producer exposes the interface and the consumer requests the interface.

The service module is to receive requests.

A webapp is a large running web site.

The project module is my own test, this can be ignored.