As long as payment is involved in the Internet industry, there will inevitably be a need for reconciliation. Almost all Internet companies’ businesses involve payment to some extent, and larger companies even have their own third-party payment companies as standard. , so reconciliation is universal. The reconciliation system is the most important part of the payment system and the last line of defense to ensure the safety of transactions and funds. In most Internet companies, there are generally independent reconciliation systems for processing, such as: e-commerce platforms, Internet finance, third-party payment companies, etc.
Reconciliation is a part of the payment system, so before reconciliation, let's understand the relevant business knowledge
business knowledge
what is reconciliation
The traditional reconciliation is to check the accounts, which refers to the work of checking and checking the relevant data in the account books in order to ensure the correctness and reliability of the records in the account books. In bank or third-party payment, account reconciliation is actually a process of confirming transactions within a certain period. Generally, the bank or third-party payment company sorts out the transactions of the previous day on the next day, and generates a statement for Platform merchants download and settle accounts payable to the platform merchants. At the next level, in the Internet finance industry or e-commerce industry, reconciliation is actually to confirm the correctness of the transactions and funds with the payment provider (bank and the third party payment) within a fixed period, and to ensure the transaction and funds of both parties. Consistently correct.
Generalized reconciliation, all data interactions across applications should theoretically be reconciled. Therefore, reconciliation can also be divided into information flow reconciliation and capital flow reconciliation. Information flow reconciliation is also generally used for reconciliation of its own internal systems, such as reconciliation of payment data in the payment system and business data in the business system to ensure the consistency of capital transactions and business transactions. Fund flow reconciliation is the fund transaction reconciliation between the payment system and the bank or third-party payment system.
Reconciliation method
- One-way reconciliation: generally use a third-party payment institution or bank flow to reconcile with its own system to prevent the problem of dropped orders;
- Two-way reconciliation: Two-way checking of the flow between two applications, such as the order and the financial system, it is necessary to ensure that the financial system has a successful payment record, and the order system is also successful; it is also necessary to ensure that the order system records a successful record, and the financial system is also successful. .
We generally use two-way reconciliation for reconciliation
Reconciliation related issues
Inconsistency between the daily cut-off points of different systems: Rolling reconciliation
Error handling: Replenishment, compensation (refund)
Related concepts
billing and reconciliation
For each transaction, the records of each participant must be consistent with no deviation. The job of the account reconciliation system is to find discrepancies in the records, that is, account balance; and then resolve these differences manually or automatically, that is, account balance.
Long and Missing Orders
Reconciling accounts with the bank based on the platform transaction, and finding that the transaction within the cycle, the platform has this order but there is no order in the third-party payment, which becomes the platform's long-term payment. The long payment on the platform is generally due to the situation that the user crosses the sky when making payment. For example, the user creates an order at 23:58 and pays at 00:03 a.m. the next day. In the case of reconciliation based on bank transactions, if the bank has this order but the platform does not have this order, it means that the platform is missing an order. Missing orders on the platform are rare, and are generally directly transferred to manual processing.
Account system
In the general payment system, it will be divided into login account and payment account. Payment account refers to the certificate of the owner's rights and interests of funds used for transactions in the payment system; login account refers to the user's login certificate and personal information in the system. A user can have multiple login accounts, and a login account can have multiple payment accounts, such as a change account, a stored value card account, and so on. Generally, payment accounts are not shared between multiple login accounts. The transaction of reconciliation is generally the payment account to participate in the transaction.
Transactions and Accounts
Account setup, generally begins with trading. The realization of the transaction must be supported by the account, and the account is the basic element of the transaction. From a payment system perspective, the flow of funds involved in a transaction is the flow of funds from one account to another. The party that initiates the transaction is called the transaction subject, which can be a person or an institution.
Clearing and Settlement
Clearing mainly refers to the currency receipt and payment between different banks. It can be considered as the sending, receiving and checking and confirmation of payment instructions by the initiating bank and the receiving bank before settlement. The result is a comprehensive exchange of settlement tools and payment information, and the establishment of final settlement. position.
Settlement refers to the process of performing corresponding accounting treatment on the positions to be settled in the originating bank and the receiving bank, completing the transfer of funds, and notifying both the payer and the payer. At present, most bank settlement business is completed mainly through two types of accounts: one is the agency account opened with each other between banks, and the other is the account opened with the central bank, independent financial institutions such as UnionPay, or third-party payment institutions.
Liquidation: Calculate the time and amount of money receivable and payable by each party. Settlement: According to the result of the settlement, the actual fund transfer operation is carried out to the parties at the specified time
Capital pool
The user's reserve funds (such as recharge) are uniformly placed in the bank account of the enterprise, and the enterprise can dispose of these funds at will, which is the fund pool. Correspondingly, the third-party escrow, the user's reserve funds are placed in the virtual account opened by the enterprise in the third-party payment institution for the user, and the enterprise cannot withdraw these funds at will. Internet finance now fully requires access to bank depository, that is, the bank will create a fund account for each user to protect the user's funds, and Internet finance companies cannot arbitrarily transfer the amount in these fund accounts.
reconciliation system
Reconciliation Design
In the design stage of the reconciliation system, the reconciliation system is divided into four modules, each of which is responsible for its own functions.
- File acquisition module: download or read the reconciliation file of each channel
- File parsing module: Create different parsing templates, and obtain corresponding parsing templates for parsing according to channels and file types
- Reconciliation processing module: business logic processing of reconciliation
- Error handling module: Process orders in the error pool
Generally, a timing task is designed, which is triggered at a fixed time point every day. The timing-driven scheduling class respectively calls four modules to process the reconciliation. Some banks will actively push the reconciliation statement, and then trigger the reconciliation process through http callback.
Reconciliation algorithm
1. Process:
1. Obtain reconciliation files from upstream channels (banks, UnionPay and other financial institutions), and the program parses and stores them line by line;
2. During program processing, based on the table of upstream reconciliation files, the program reads line by line and communicates with us Compare the transaction records of the system with the accounting records (if there is an accounting system, a reasonable solution should be to compare the accounting records) to find out the difference records;
3. Based on the transaction records of our system and the accounting records, the program is performed line by line Read and compare the upstream reconciliation file to find out the difference records.
2. Defects:
1. Querying relevant data during the reconciliation process. If the amount of data is huge, it will greatly affect the performance of the database , and the expansion of the reconciliation logic is extremely troublesome;
2. The efficiency of the line-by-line comparison algorithm is low, but there is no good room for optimization in the algorithm . If the database INTERSECT and MINUS are used, the pressure on the database is also high;
3. In the case of large business volume (for example, there are hundreds of upstream channels that need to be checked, each with hundreds of thousands of transaction records), the reconciliation server and database server The load is high. Even if the read-write separation is adopted, the pressure of using the reading library during account reconciliation is the same;
4. Importing batch files, the efficiency of entering the library line by line is relatively low (each time it is necessary to establish a network connection and close the connection).
3. Improvement:
1. For network transmission, try to operate in batches to reduce network consumption and time consumption;
2. Use Redis and other NOSQL databases to reduce the pressure on database servers. At the same time, it is also easy to expand. One Redis server is not enough, and you can add servers for account reconciliation without limit;
3. Use the sdiff function of the Redis set collection, and use the high performance of the Redis sdiff algorithm to compare the upstream records with mine differences in the records.
Reconciliation process
1. Download the statement
Most banks require the access party to provide ftp service, and the bank regularly pushes the statement to the ftp server provided by the access party; some banks will provide the download service of the statement, both through ftp/http, ftp method Most of them are; in addition, the statement of online banking is quite special. Generally, it is necessary to log in to the background management system of online banking for settlement, download it manually, and import the statement into the reconciliation system after the settlement is completed.
The technical implementation can be made into a factory mode. Different payment channels have different download classes. If it is an http interface, the file will be written to the statement. If it is an ftp server, the statement in the server will be downloaded to the local directory with parsing. . The code mainly involves ftp tool class, http(s) tool class, and related IO reading and writing.
In terms of technology selection, HTTP(S) can use apache httpclient to realize connection pool and resumable upload, and FTP can also use Apache Commons Net API. But no matter which one it is, you need to set the number of retries and the connection timeout. The setting of the number of retries and the interval needs to be careful. Too frequent retries will easily kill the server. If the time interval is too large, subsequent processing steps will be blocked. 5 to 10 minutes is a suitable retry interval.
2. Create a batch
On the one hand, batches are created to prevent repeated reconciliation, and on the other hand, the reconciliation result information needs to be stored in the batch at the end of the reconciliation.
3. Parse the file
The parsing file is mainly to parse the downloaded reconciliation file into a data type that we can reconcile and store it in the library. The parsed files have different types in different channels, so they can also be designed into different parsing templates, and the factory mode can be used to parse files of different formats into a unified data type that can be reconciled. The parsed file types generally include: json, text, cvs, excle, etc., and some banks will encrypt the bills or provide the zip package format. Here, additional zip tools and encryption and decryption tools need to be developed for processing.
The main information contained in the reconciliation file includes: merchant order number, transaction serial number, transaction time, payment time, payer, transaction amount, transaction type, and transaction status.
4. Reconciliation processing
Reconciliation processing is also the core logic of reconciliation, which is implemented in the following steps:
- Query all orders successfully traded on the platform
- Query all trading orders on the platform
- Query the data in the platform cache pool
- Query the order of successful bank transaction
- Begin reconciliation based on the platform data, and the platform long-term payment will be credited to the buffer pool
- Start reconciliation based on the data of the bank channel
Reconciliation logic based on platform orders: Based on all successful orders on the platform, traverse all the data of bank orders, find orders with the same order number, and compare whether the amount and handling fee of the orders are consistent. If it is consistent, skip it; if it is inconsistent, the platform order will enter the error pool; if the transaction is not found in the bank order, the order will enter the cache pool, and the platform long payment will be recorded. At the same time, the reconciliation-related amount and the number of orders are counted.
Reconciliation logic based on bank orders: based on bank transaction data, traverse all platform transactions (including unsuccessful orders), find orders with the same order number but inconsistent payment status, and store the amount in the error pool for comparison. . If the order is not found in the platform transaction, then traverse the cache pool to find the corresponding platform order to verify whether the amount is consistent, and the inconsistency enters the error pool. If the corresponding order is still not found in the buffer pool, it will directly enter the error pool and record the platform's missed order. At the same time, the reconciliation-related amount and the number of orders are counted.
5. Reconciliation statistics
According to the reconciliation process, the statistics related information include: reconciliation completion time, whether the reconciliation is successful, the amount of the reconciliation and the number of orders, the amount of errors and the number of orders, the amount of the buffer pool and the number of orders, etc.
6. Error handling
In a general system, error handling is divided into two types, one is handled manually, and the other is handled automatically by the system.
There are mainly the following situations:
The local is not paid, the payment channel is paid. This is mainly caused by the asynchronous notification sent by the channel is not correctly received locally. The general processing is to modify the local state to paid, and do the follow-up processing of the response, such as notifying the business party.
The local payment has been paid, and the payment channel has been paid, but the amount is different, which requires manual verification.
The payment has been made locally, but there is no record in the payment channel; or there is no record locally, but there is a record in the payment channel. In addition to excluding cross-day factors, this situation is very rare and needs to be dealt with after understanding the specific reasons.
sample code
There are not many open source codes for online reconciliation systems, but one is not bad: Dragon Fruit Payment. There is a relatively complete reconciliation process in Dragon Fruit Payment. In the code, the reconciliation process is written by taking the reconciliations of WeChat and Alipay as examples. You can refer to it.
Finally, how is your company's reconciliation system implemented, and what pitfalls did you encounter? Welcome to discuss with me.
refer to:
Payment system reconciliation processing
Account system architecture design related ideas record
Reconciliation System – Project Overview
Author: Pure Smile
Source: http://www.ityouknow.com/
http://blog.csdn.net/ityouknow/article/details/73274779
The copyright belongs to the author, please indicate the source for reprinting