First, the business system design overview
1. What is the business system
Internet companies often product direction divided into two categories, C ends A and B, C mainly for end customers and consumers of the system, the B-side of the range is relatively vague, to the system vendor or business use, to internal business personnel system, are collectively referred to as the B-side system. Starting point and focus and the C-terminal end of the B system construction is completely different. C-terminal system emphasis on user experience, emphasizing the emotional, continuous data analysis and optimization, with a different button placement must be carefully designed, demonstration, clients are personal; B-end system emphasis on processes, modular, emphasizing the abstract and structural , pay attention to overall planning and system design, clients are organizations and institutions.
If the B-side system further split can be divided into two categories, the first category is the business end, common in bilateral mode of platform-based Internet companies, such as Taobao sellers management systems, business management background beauty group; the second category is the internal business systems to support the business, management, business operation.
- Common business systems including ERP (EnterpriseResource Planning), CRM (CustomerRelationship Management), SCM (Supply ChainManagement), WMS (WarehouseManagement System), TMS (TransportationManagement System), OA (Office Automation), HRM (Human ResourceManagement) and so on.
- From a software point of view, all of the software system is divided into two categories, the first category is a system capable of producing real-time traffic data, called OLTP (Online TransactionProcessing) system, the second is for data processing, treatment, exploration, mining show system, called OLAP (Online AnalyticalProcessing) system
Overall, the system has a four-point business value to the enterprise: to enhance the management and control capability, control operational risks, reduce operating costs and increase sales.
2, the importance of designing business systems
Not the same for different enterprises emphasize business focus, specific customization of the system of business is very important.
3, how to design a business system
Generally, a service system is constructed from 0 to 1, go through the following link.
(1) Business Design
PM carding and business owners together to develop business processes, systems, mechanisms, understand business problems and identify software system solutions.
(2) the overall system design
PM combine business aspirations and goals, complete system-level design, including the definition and evolution of abstract business blueprint, system boundaries, system functions, the overall application architecture design, how to splice the existing system with the company, interaction .
(3) system design details
PM complete details of all program design, including modeling, roles, interfaces, and other privileges. Which is the hardest part of modeling, modeling determines the quality of the future system flexibility, scalability. Modeling requires a comprehensive understanding of the business, a strong ability to abstract summarized.
(4) the implementation and acceptance
PM landed responsible for the final project, on the system after the line is to be an ongoing iterative optimization, deeply involved in product operations and data analysis.
If the system is designed from scratch, you must fully implement the above aspects, especially in architecture and design model, it is the most important.
Above Reference Source:
http://www.woshipm.com/pd/586436.html
https://36kr.com/p/5105245
Fourth, data reconciliation system
Focus on what we want to belong to design business systems: data reconciliation, data reconciliation generally used in the payment system, the two sides agreed to maintain funding system is very important. The other system is also very important for core data for interactive, so here we need from architecture to design a highly available data reconciliation system.
What is the capital of reconciliation?
- The concept for the Accounting: Refers to check work-related accounts in order to ensure the correctness of the books and records carried out, so that the account card match, match ledger, accounts match the reality.
- The concept of paying agencies: funds reconciliation, reconciliation in the center, the system will save the bank accounts and return water pipeline clearing and settlement documents for reconciliation, accounting system data consistency checking and clearing bank data, to ensure payment bodies of the reserve bank accounts daily amount is expected to occur consistent with the actual amount. That is the actual liquidation of funds such as bank reconciliation recharge, whether the result of the processing bank charge refund, cash and other services agreement.
The role of reconciliation center?
- Reconciliation is the main processing system modules, main business is liquidated reconciliation. Reconciliation work platform deployed in the center, were accepted accounting data entry system and settlement system for reconciliation process.
- Reconciliation is the center of the most important responsibilities of blending water clearing bank payment systems and water accounted for checking balances reflect changes in actual silver deposit accounts with the internal system users to pay the balance of the change is balanced. For already certified true bank clearing and payment systems recorded water flow, respectively, into the corresponding historical stream banks recorded flow reservoir reservoirs and history.
- For the blending results in the bank clearing system accounted for more than water flowing water, and the operator does not clear the source of funds, we need to temporarily clear the funds losses are treated according to the set of classification rules. Then we think that part of the funds have systematically recorded, it can flow into the history of the reservoir.
- Since the actual increase in the silver deposit balance of the account in this case, the corresponding user is an internal balance of the payment system also increased, such as: T day losses offs may be recognized in the future T + N, and the subsequent behavior offs It is a detail of the offloading processing service water, and we should not be water-offs accounting day T + N as the entry of the generated water is no longer necessary to reflect the reconciliation center.
From now extended analogy to other payment data reconciliation system ( references )
to other reference material on the payment system: data 1 , data 2 Data reconciliation between systems based on our existing system, the following key elements, sometimes due the company did not provide as much financial and human support, all of the research and development role requires a person to solve, so here is my role from the business side, PM, product managers, architects, development, testing to solve them.
(1) Business Design
Business Background (business research): joint marketing projects across the main shop around big promotion activities across the self / businesses, marking, coupons, promotional single product, take stock and other big promotion activities to create and collect items to the current fight benchmarking activities received a million product quantity data, synchronization to the downstream marketing system or hundreds of millions of promotional items received much data is synchronized to downstream systems, this process large volumes of data for long periods of time, we need to ensure that both sides have systems data between idempotent. He had appeared before the network jitter, and no data loss due to data reveal all the details and circumstances are not synchronized to downstream systems.
OP: current core system for processing logic programs - "downstream data synchronization, the synchronization system successfully updated in real time status data, and synchronization data inserted into a log table, the data fails to be synchronized ES. Due to the distributed system, the transaction is more complex, currently relies on exceptions and MQ retry mechanism, the results prone, garbage data and failed to find inconsistencies and make effective solutions. Consider the current situation the need for data reconciliation, data independent of the expansion in core stability assist system, and to achieve from the existing core business systems.
(2) the overall system design
- Reconciliation content and data sources
- Content reconciliation: main product sku whether the present system and the downstream system data consistent.
- Source: the system database table called the downstream data transmission system,
data is returned in real time of the system call downstream system.
- Reconciliation business processes
- The main function of the center of reconciliation
Tips:
- 1, accounted reception:
joint marketing core business systems to close product SKU information sent MQ (redo) - "Data Reconciliation System monitor consumption -" Storage jimdb- 2, data is stored:
- jimdb (redis) stored as a clearing pool, and an account check diff recorded data, and verification.
- Elastic Database JED (MySQL), as a result of checking the log storage water reconciliation
- 3, the receiving account:
joint marketing promotions core business systems in real-time call information system returns the result to send MQ (done) - "Data Reconciliation System monitor consumption -" Storage jimdb- 4, data clearing: joint marketing activities into three categories trigger liquidation event
- Daily settlement: daily big promotion income products and synchronize according to the rules, plenty of time. 24h liquidation trigger event before the event.
- 限时清算：主要应对收品量小于千万，时间限制8h左右的情况。活动开始前5h进行清算事件触发。
- 紧急清算：主要应对收品量具大但是由于管理员忘记或者其他原因未能提前发布，品量大于千万级别。活动开始前12h进行清算事件触发。
- 5、异常预警：
数据对账系统进行清算入账数据和出账数据，对比结果不一致，异常预警邮件和短信通知研发，并进入审核节点。对比结果一致，接入数据存储，作为流水日志存储。- 6、审核流水：
研发收到报警，进入审核节点根据活动和异常sku信息定位数据不一致原因，根据预警和人工介入排查信息定位是否需要同步业务方和产品，或者从新跑数据调用同步促销系统。- 7、数据展示：
根据流水日志存储，进行数据展示界面，提供数据分析给产品和研发更好的保障系统数据的安全性。
- 对账流程图
- 对账中心功能模块分析
- Q1:
清算策略执行，异常预警，人工审核，重新跑数据时间周期多久？是否有足够的时间可控范围内
A1:
目前需要预估数据，根据jimdb运维反馈，单个key的数据比对不要超过5000，如果2亿数据，需要比对40w次，性能时间未能预估，需要压测执行参考性能。
- Q2:
数据对账系统的可用性
A2:
设计初衷，数据对账不止同步sku数据，其他所有系统交互都可以接入数据对账系统，数据对账系统只关心入账和出账比对结果，脱离核心业务系统逻辑，属于扩展系统。
- Q3:
数据对账系统的数据安全性
A3:
数据对账系统的数据安全性主要依赖于jimdb，通过rdb快照和aof日志恢复数据，但是不能保证100%数据恢复。
- 意外数据恢复逻辑
- 5.1. Accidental data recovery logic
with reference to said data reconciliation data security system.- 5.2. Reconciliation and abnormal restore logic
by manual review, or re-run the data to make timely emergency measures plan depends on your situation.
(3) details of the system design
3.1) simplifies the preliminary project schedule
3.2) establish the information shared folders GitHub or SVN (continuous delivery and information sharing):
(4) the implementation of acceptance
