Risk Control Decision Engine - Decision Flow Construction Actual Combat

introduction

This article mainly focuses on the construction of the decision tree arrangement capability in the risk control decision engine. The decision-making engine is the brain of risk control, and the decision tree arrangement ability and experience are the means to build the brain. How to build an efficient , smooth , stable and reliable decision tree arrangement ability is a major challenge for the risk control decision-making engine. This article will share with you the experience of building it in the past.

background

The initial construction of any system is definitely not in the direction of " one step in place ", but the architecture designer tries to build in the direction of scalability and maintainability in the later stage. A good underlying design is not afraid of crazy iterations in the later stages of the product, and it is easy to change and adjust. Poor " cramming " codes may eventually develop into " shit mountains " in order to realize the functions as soon as possible at the time, and the maintenance costs are getting higher and higher.

MVP Small Step Iteration 1.0

Goals at this stage: Minimum Viable Product (MVP); Xiaobu iterates and goes online quickly; one person plays multiple roles.

In the early days of the establishment of the risk control department, there were few personnel and lacked UED and front-end. After all, the risk control itself did not just need visual design and front-end, but mainly the back-end R&D and strategic operations to fight against black production. At this time, in order to launch the decision tree function as soon as possible, the R&D personnel directly placed the decision tree static configuration file in the code layer resource directory resource(the specific implementation is decomposed below), and each change needs to be published. The construction of the engine itself is not perfect, and there are many functions that need to be added. It is commonplace to release several versions a week, and everyone can accept it at this stage.

"Rotation from Static" 2.0

Goals at this stage: no version release, rapid production changes; considerations related to stability.

With the gradual growth of the department team and the standardization of the R&D process, the risk control strategy operators have more and more urgent needs for the response timeliness and visualization capabilities of decision-making arrangement . They are dissatisfied with the status quo that R&D needs to publish a version to deploy new decision-making capabilities . Black production is efficient, but R&D and release require planning and time. At the same time, there are certain risks in releasing the version. If something goes wrong, it needs to be rolled back immediately. At this time, the time for the strategy to go online will be delayed.

Based on the above, we consider that it is time to open up the ability to directly visualize the decision tree arrangement in the production environment, but we do not have front-end students, and we may not be familiar with the decision engine process specification if we find other departments, and the communication cost is still high. That compromised a solution: move the static configuration file to the DB storage, and the configuration can be displayed on the front-end in the form of text characters. No complicated front-end design is required, and only simple form text box filling is required to meet the demands of R&D and modification decision-making flow. In this way , the original static configuration can be "moved" and can be configured directly in production, which greatly improves the efficiency of production deployment .

Visual decision flow orchestration 3.0

The goal of this stage: efficient, stable and intelligent visual decision tree arrangement capability product construction

There are more and more business lines connected to risk control, and R&D personnel are busy with variable development iterations corresponding to risk scenarios. At this time, part of their energy needs to be allocated to modify the decision tree. For operations, the decision tree of version 2.0 is just a string, not a tree. There is no way or dare to modify the strategy operation, and the risk of making mistakes is too great. Considering that the volume and mode of the entire risk control are already very stable, there is also a certain amount of time to consider making decision-making arranging a visual product delivery strategy for use by personnel. After all, the adjustment of the decision tree itself is also one of the responsibilities of the strategy, which needs to be precipitated into a highly available product .

We refer to BPMNthe front-end style design specifications of the workflow in the industry, extract the elements that need to be used in the risk control decision-making tree, and build our own decision-making engine with intelligent orchestration capabilities and visualized drag-and-drop nodes, which can be completely delivered to the strategy personnel for self-configuration and use .

design implementation

Technology selection

A decision tree is actually a variant of DAG (Directed Acyclic Graph) . The nodes in the graph have different attributes and functions at the business level.

So how to store this DAG structure? Using two-dimensional array storage cannot meet the requirements of node attributes and edge attributes. First, the boundary cannot be defined, and the tree may be very large. Second, if the attributes are implemented by an association table, it will be very fragmented and cannot be seen directly.

In fact, a graph can be represented by a linked list , and the storage structure of the linked list is represented by JSON or XML . It is conceivable that if JSON is used to express, the hierarchical nesting relationship will be very cumbersome. After all, JSON is used to serialize data. In terms of presentation, it is more convenient and intuitive to add attributes in XML.

data structure

An example of a simple decision tree is as follows:

The above decision tree is represented by XML data structure as follows:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<flow id="test01" desc="建议决策流">

  <!-- 开始节点 -->
  <start id="start">

    <!-- 链接到下一节点 -->
    <link to="black01"/>
  </start>

  <!-- 名单节点 -->
  <nameList id="black01" desc="黑名单">

    <!-- 名单属性：名单类型：黑/白/灰；领域类型；适用范围 -->
    <field key="type" val="black"/>
    <field key="domain" val="10001,10002"/>
    <field key="scope" val="deviceHash,phone,uid"/>
    <link to="split01"/>
  </nameList>

  <!-- 分流节点 -->
  <split id="split01" desc="是否为微信渠道">

    <!-- 条件分支 -->
    <condition order="0" desc="是" expr="system == 'wechat'" to="strategy01"/>
    <condition order="10" desc="否" to="strategy02"/>
  </split>

  <!-- 策略节点 -->
  <strategy id="strategy01" desc="微信专属策略">

    <!-- 关联专属策略元数据 -->
    <field key="strategyGuid" val="25F7C71A5F834F24A12C478CEE4CB9EB"/>
    <link to="end"/>
  </strategy>
  <strategy id="strategy02" desc="非微信渠道策略">
    <field key="strategyGuid" val="0FC8A95A4D6A4F169C77950BB4A98D80"/>
    <link to="end"/>
  </strategy>

  <!-- 结束节点 -->
  <end id="end" desc="结束"/>
</flow>

The above data structure very intuitively represents the current decision tree data structure that needs to be drawn. Compared with JSON data representation, XML is more flexible, more convenient to expand, and can have a better balance in horizontal and depth .

Decision Flow Analysis

XML is a very mature technology implemented. There are many open source implementations of parsing XML on the market. I use common-digesterparsing for the above data structure, and the following dependencies can be introduced into the POM:

<!-- https://mvnrepository.com/artifact/commons-digester/commons-digester -->
<dependency>
    <groupId>commons-digester</groupId>
    <artifactId>commons-digester</artifactId>
    <version>1.8.1</version>
</dependency>

The entity relationship is as follows:

The XML data is parsed as follows:

@Data
public class FlowEntity {
    
    
    private String id;
    private String desc;

    private INode startNode;

    private Map<String, INode> nodeMap = new HashMap<>();
}

Digester digester = new Digester();

// parse flow node
digester.addObjectCreate("flow", FlowEntity.class);
digester.addSetProperties("flow");

// parse start node
digester.addObjectCreate("flow/start", StartNode.class);
digester.addSetProperties("flow/start");

// 在 FlowEntity 实现 addNode 方法，将当前节点录入
digester.addSetNext("flow/start", "addNode");
digester.addObjectCreate("flow/start/link", LinkBranch.class);
digester.addSetProperties("flow/start/link");

// 在 StartNode 实现 addLink 方法，将当前边录入
digester.addSetNext("flow/start/link", "addLink");

// parse split node
digester.addObjectCreate("flow/split", SplitNode.class);
digester.addSetProperties("flow/split");
digester.addSetNext("flow/split", "addNode");
digester.addObjectCreate("flow/split/condition", ConditionBranch.class);
digester.addSetProperties("flow/split/condition");

// 在 SplitNode 实现 addCondition 方法，将当前条件录入
digester.addSetNext("flow/split/condition", "addCondition");

// 省略...

InputStream inputStream = new ByteArrayInputStream(xmlResource.getBytes());
return (FlowEntity) digester.parse(inputStream);

Where addNodethe logic is to store all nodes in a nodeMapstructure, and if the current node is the start node, assign to
startNodethe node.

After the XML is parsed, the association relationship has not yet been established. After polling each node, connect the nodes with each other , and verify that the nodes exist enough to ensure that they can be associated into a tree.

public void assembleToNode(Map<String, INode> nodeMap) {
    
    
    if (Objects.isNull(nodeMap)) {
    
    
        return;
    }

    if (!nodeMap.containsKey(this.to)) {
    
    
        throw new RuntimeException(String.format("%s to: %s can't find node from nodeMap", this.desc, this.to));
    }

    this.toNode = nodeMap.get(this.to);
}

Decision Flow Execution

The execution of the decision only needs to startNodestart from the execution and recursively execute until the only exit is found and popped out. Note that the strategy interface has an output decision result. If it is rejected, the process execution can be directly interrupted at this time and the result can be returned.

@Override
public void execute(FlowContext context) {
    
    

    // 出口
    if (this instanceof EndNode) {
    
    
        return;
    }

    // 递归执行
    this.execute(context);
}

Among them, SplitNodethe node execution needs to calculate the conditional expression . As long as a condition is met, the node to go down can be determined. The implementation of subclass coverage is as follows:

Note: I posted a separate article on conditional expressions before. If you are interested, please pay attention to it. You can find it in my historical article archives , and I won’t explain it here.

@Override
public void execute(FlowContext context) {
    
    

    Validate.notEmpty(condition, "node id: {} desc: {} [condition] is empty", this.getId(), this.getDesc());

    // 主动判断
    Optional<ConditionBranch> target = condition.stream().filter(c -> c.evaluate(context)).findFirst();

    // TODO: 考虑返回默认兜底分支节点
    if (!target.isPresent()) {
    
    
        throw new RuntimeException("node id: {} ConditionBranch expr execute find nothing, please check your expr condition");
    }

    target.get().getToNode().execute(context);
}

StrategyNodeThe principle of node execution is SplitNodethe same as that of . You only need to override the implementation method by the subclass to execute the corresponding rule engine, and get the decision result to judge the direction, which is not listed here.

After designing the storage structure of the decision tree as above, and then cooperating with the style based on the BPMN flow diagram built by the front-end students, and customizing the node information and expressions required for risk control , an ideal tree can be built at any time (I mentioned it here, but the front-end students have paid a lot for silky arrangement and auxiliary verification, of course, this is not the focus of this article).

Summarize

This article shares the thinking and construction process of the decision-making flow diagram in the decision-making engine, from the minimum available product launch to support business development to the work area where visual orchestration capabilities are precipitated. Of course, this article only shows the thinking and construction process of the general decision-making flow. It shows that various challenges will still be encountered in the business, such as performance requirements , cost control , etc. There are many challenges. I will share them one by one in the follow-up. Welcome to pay attention.

Wonderful past

Welcome to pay attention to the official account: Cuckoo Chicken Technology Column
Personal technology blog: https://jifuwei.github.io/