Inventory pre-occupation architecture upgrade plan design - transaction inventory center | Jingdong logistics technical team

background introduction

With the rapid development of the logistics industry and the implementation of the integrated supply chain model, it poses a huge challenge to system throughput and system stability. Inventory, as the top priority of the supply chain, is more obvious. The data for the past three years shows that:

Access merchants increased by 37.64% year-on-year, and product categories increased by 53.66% year-on-year

The number of goods increased by 46.43% year-on-year, and the number of warehouses increased by 18.87% year-on-year

Through the analysis of the past traffic of big promotions, the growth rate of minute-level traffic is 75%. In the big promotion warehouse, it is reported that the three-party orders are not delivered in time, and the throughput and performance of inventory reservation are one of the factors leading to the backlog of orders. At present, the inventory uses the mysql database as a means of pre-occupying orders. With the construction of an integrated supply chain and the continuous access of key KA merchants, the existing inventory structure has risks and defects in business support.

In addition, business growth and traffic growth are expected to increase by 5-10 times in the next 3 to 5 years. In order to avoid business losses caused by system performance and technical architecture defects, a lightweight inventory architecture is imperative.

// Glossary:

Inventory pre-occupation: It means that after a consumer takes an order for a commodity, the inventory is temporarily reserved for the order, and the reserved inventory is the pre-occupied inventory.

Architecture Principles

Architecture : It is problem-oriented and a means of solving problems. Problems of inventory system: Non-functionality: 1. High concurrency 2. System stability (disaster recovery) 3. Data consistency Functionality: 1. Business complexity 2. Data consistency

system design

Design ideas

Where is the bottleneck of the current inventory system? : Anti-write traffic, the database becomes a bottleneck point.
How to solve the system bottleneck? : The database is replaced by the high-concurrency component Redis.
What problems need to be solved by using Redis? : Anti-overselling, asynchronous writing to the database to ensure eventual consistency.

overall design

Loading part : Inventory performance bottleneck is pre-occupied. The traditional architecture mainly relies on database transactions to maintain data consistency and read and write data; the new architecture design transplants the data carrying part to Redis, and uses Redis' high-performance throughput to solve data reading and writing in high-concurrency scenarios .
Data write-back : Redis carries out load shaving and subsequent data is only used for bookkeeping, and ultimately sacrifices the short-term consistency of data to achieve the purpose of peak shaving.
The difference : the old version of the inventory preemption design only relies on data for data processing, and the new version of the design relies on slicing configuration to build data and switch to Redis, and uses Redis' high read and write to perform peak shaving operations.

detailed design

Main process:

Inventory initialization : the race condition uses the Redis watch command to implement lock waiting and solve the problem of data inconsistency in concurrent scenarios.
LUA Executor : Encapsulate atomic operation instructions/multiplex instructions into LUA scripts to reduce network overhead.
Compensation mechanism: i> When all business exceptions in the execution process occur, a reverse operation request will be initiated synchronously; ii> After the reverse operation is abnormally executed, an asynchronous reverse operation task will be submitted; **iii>** After the asynchronous task is abnormally executed, the dependent The monitoring system scans abnormal documents or abnormal inventory and modifies the abnormal inventory

Backtracking and writing : After the task is placed in the database, the mq assembly parameter is sent to call the data write-back service, and the data write-back service operates the inventory quantity; at the same time, the redis data is written back to release the pre-occupied inventory data; the data status of the task database is updated

data structure

Inventory record index : {deptNo|goodsNo|warehouseNo}|stockStatus|stockType|goodsLevel
hashTag：{deptNo|goodsNo|warehouseNo}|stockStatus|stockType|goodsLevel
Available inventory quantity : usableKey: {Inventory record index}
Deduct inventory : usableSubtractKey: {inventory record index}, record Redis to DB to reduce inventory during execution
Pre-occupancy anti-duplication key : operateKey: {inventory record index: odd number} anti-duplication key anti-concurrent duplicate request
Rollback Anti-reload : rollbackOperateKey: {Inventory Record Index}
Ullage reserved inventory : ullageOperateKey: {inventory record index}
Deduct inventory document record : hSetrecord: {Inventory record index}

key	preempt	Shortage Preemption	rollback	Writeback
Available inventory	-	-	+	constant
Deduction of inventory	+	+	-	-
pre-occupancy anti-duplication key	+	+	-	constant
rollback anti-heavy	constant	constant	+	constant
Shortage Reserved Inventory	constant	+	reverse	constant
Deduct inventory document record	+	+	-	-

Redis&DB

First, compare redis and slave database data, and verify the main database if there is a difference
During the comparison process, the sku detail row in the DB is locked (for update), and the comparison logic is DB available inventory == (Redis available inventory + Redis reserved amount)
If there is a difference, an alarm will be issued and the expiration of the available amount of the SDK will be triggered, and the pre-occupied amount will be corrected at the same time

Disaster recovery plan

// For system fault tolerance/degradation, monitoring mechanism (space for stability, two copies of redis, 3 failure counts), traffic distribution materials, 618 large traffic, peak data severance. Inconsistent data, multiple merchants, no more than 5 points.

Persistence of pre-occupied tasks: mysql needs to persist the core attribute field data: business department, commodity code, warehouse code, grade, inventory type, inventory status, pre-occupied inventory, task status; after the scheduling execution is completed, the stockTask status needs to be updated as Finish

initialization:

(1) lock db

(2) sum stockTask

(3) Use DB available inventory to initialize Redis available inventory, and stockTask reserved amount to initialize Redis reserved amount

(4) Redis inventory rollback, if the reserved key does not exist, the key does not need to be rolled back

performance results

23 years 618 big promotion

cut rules

hot and cold data

OMS Inventory Cooling and Heating Units

Pre-occupied architecture upgrade cut-off key key monitoring

Inventory pre-occupation structure upgrade cut-off merchants

Structure upgrade cut business details 2

Businesses that have been cut

reverse tangent

The following lists existed in the original design

Prohibition of cut-off business: the priority is higher, once in the list, cut-off is prohibited

Batch inventory merchants: Batch inventory management merchants, this part of the capacity has not yet been built

Dynamic pledge merchants: logistics financial business, currently this part of the capacity has not yet been established. Merchants on the cut-off list: this part is cut-off merchants

The original cutting process:! Scaling is prohibited ->! Batch stock ->! In the dynamic pledge->scaling list, those merchants that pass the above verification are staking merchants.

In the original process, in incremental merchants, merchants need to be manually configured into the sizing list before the slicing operation can be performed. The operation for new merchant scenarios remains unchanged, and the logical inventory list in the original process is a pain point: enable configuration of logical inventory In the division master data, it is not on the inventory side.

In the new version of the sizing process, the sizing list is optimized, and the original sizing list merchants are split into two lists: a non-logical inventory list and a logical inventory list, of which:

Illogical inventory list: including merchants that can be cut

Logical inventory list: Logical inventory merchants, this part cannot be quantified

The new process of the original process optimizes the list of cut-off merchants and splits it into two lists: non-logical inventory list and logical inventory list

Build the model (batch inventory & memory model to be continued)

Redis storage data structure

MD generation rule toolset

◦ Logical inventory MD5 tool

     StringBuffer md5Key = new StringBuffer();
     md5Key.append(logicWarehouseStock.getGoodsNo()+"_"+logicWarehouseStock.getWarehouseNo()+"_"+logicWarehouseStock.getOwnerNo()+
             "_"+logicWarehouseStock.getDeptNo()+"_"+logicWarehouseStock.getStockType()+"_"+logicWarehouseStock.getGoodsLevel());
     if(StringUtils.isBlank(logicWarehouseStock.getFactor1())){
         md5Key.append("_0");
     }else {
         md5Key.append("_"+logicWarehouseStock.getFactor1());
     }
     if(StringUtils.isBlank(logicWarehouseStock.getFactor2())){
         md5Key.append("_0");
     }else {
         md5Key.append("_"+logicWarehouseStock.getFactor2());
     }
     if(StringUtils.isBlank(logicWarehouseStock.getFactor3())){
         md5Key.append("_0");
     }else {
         md5Key.append("_"+logicWarehouseStock.getFactor3());
     }
     if(StringUtils.isBlank(logicWarehouseStock.getFactor4())){
         md5Key.append("_0");
     }else {
         md5Key.append("_"+logicWarehouseStock.getFactor4());
     }
     if(logicWarehouseStock.getYn()== null){
      md5Key.append("_1");
     }else {
         md5Key.append("_"+logicWarehouseStock.getYn());
     }
     md5Key.toString().hashCode()

Batch Inventory MD5 Tool

public void fillMd5Value(){
        StringBuffer md5Key = new StringBuffer();
        md5Key.append(warehouseNo);
        md5Key.append("_");
        md5Key.append(goodsNo);
        md5Key.append("_");
        md5Key.append(goodsLevel);
        md5Key.append("_");
        md5Key.append(stockType);
        //遍历类字段不遍历map是为了控制MD5的组成顺序
        Class clazz = BatchAttrStock.class;
        Field[] fields = clazz.getDeclaredFields();
        try {
            int batchFieldCount = 0 ;
            for (Field field : fields){
                BatchAttrEnum attrEnum = BatchAttrEnum.batchFieldEnumMap.get(field.getName());
                //不是批属性的字段不进入MD5的组成
                if (attrEnum == null){
                    continue;
                }
                batchFieldCount ++;
                field.setAccessible(true);
                Object value = field.get(this);
                if (value == null ){
                    md5Key.append("0");
                    continue;
                }
                if(field.getType().toString().contains("String")){
                    md5Key.append(value);
                    continue;
                }
                if(field.getType().toString().contains("Date")){
                    Date timeField = (Date) value;
                    md5Key.append(timeField.getTime());
                    continue;
                }
                throw new RuntimeException(attrEnum.getField()+"填充MD5异常");
            }
            //默认50个批属性长度,长度不够0补齐
            int remainLength = 50 - batchFieldCount;
            String str = String.format("%0"+remainLength+"d", 0);
            md5Key.append(str);

        }catch (Exception e){
            throw new RuntimeException("填充MD5异常.");
        }

        md5Key.append(yn);
        String md5Value =  MD5Util.md5(md5Key.toString());
        setMd5Value(md5Value);
    }

MD&ID&Attribute Save Tool

The length of this article is limited, and the remaining two issues will be shared.

Author: JD Logistics Jinpeng

Source: Reprinted from Yuanqishuo Tech by JD Cloud developer community, please indicate the source