The following article comes from InfoQ digital latitude and longitude.
InfoQ digital latitude and longitude:
The official account of InfoQ Geek Media. For digital managers, practitioners, and insights, it provides digital enterprise cases, policy interpretations, and research reports, and serves as a "recorder" in the digital era.
Author | Zhao Yuying
Guest | Wu Chunlan, Technical Director of Central European Wealth
This article interviews Wu Chunlan, Director of Wealth Technology at CEIBS, to discuss the challenges faced by the digital transformation of the wealth management industry, including talent, security and technology infrastructure. In the database migration, China-Europe Fortune solved the old problems of MySQL by using the distributed database TiDB, emphasizing the importance of the HTAP converged architecture in terms of performance and resource management. The article points out that digital transformation needs to span the levels of thinking, organization, process, and platform to adapt to increasingly efficient and innovative needs. It takes about 12 minutes to read the full article.
Key points of this article:
- In recent years, the field of wealth management has been affected by changes in the internal and external environment, which has put forward higher requirements for technical base capabilities such as agility, efficiency, and timely innovation;
- Digital transformation is not a technical issue, but involves four levels: thinking, organization, process, and platform;
- The digital transformation of enterprises in the wealth management field mainly faces three challenges: talent, security, and infrastructure (technology); it will take time for artificial intelligence technology to be implemented on a large scale within enterprises in the wealth management field;
- Before selecting a database, enterprises need to first determine their needs;
- Before the database migration, the main problems encountered by the old MySQL system were that DDL for large tables was time-consuming, sub-database and sub-tables were labor-intensive, and single-node writing was prone to bottlenecks. Finally, the above problems were solved through the distributed database TiDB;
- When evaluating the effects before and after database migration, pay attention to the invisible costs such as operation and maintenance and resources;
- The HTAP converged architecture plays an important role in performance and accurate resource consumption;
Ideas, difficulties and practices of digital transformation and upgrading of China-Europe wealth
InfoQ: The pace of financial management companies and fund companies has obviously accelerated in recent years. What is the driving force behind this? What are the pain points of digital transformation in the financial management industry?
Wu Chunlan : In recent years, the internal and external environments of fund companies have undergone relatively large changes. Since the rise of Yu'e Bao in 2013, the entire Internet business has developed rapidly, which has brought several obvious changes to the market: The first change is the rapid expansion of the user base. To serve users well, the speed of technology iteration needs to be faster. quick. For example, the data of some companies with Internet attributes refreshes quickly. In order to achieve this effect, the entire company has made relatively large investments and cooperation, including introducing artificial intelligence technology to do some automated things; the second change is the Internet business It is more distinctive and the information is more transparent. Users can quickly see what new businesses and channels are emerging in the market. This requires the team to remain agile and efficient at all times, including connecting with upstream and downstream businesses; the third change is that new businesses are beginning to appear in the industry . Marketing forms, such as marketing through live broadcasts, or operating new platforms such as Douyin, etc., require enterprises to open up internal operational processes and data, which also requires timely innovation, agility and efficiency for technical teams.
To sum up, companies need to seize the opportunity and make quick decisions to respond to these changes. For example, break the original data islands and form a unified and intelligent data middle platform. Based on this middle platform, we can better explore customer characteristics and draw user portraits, so that products can better meet customer needs; cooperate with upstream and downstream institutions and When companies cooperate, they need to have strong R&D capabilities, including models, algorithms, customization capabilities, etc., which must match the R&D capabilities of major Internet companies.
Looking at the internal and external aspects, the main difficulties lie in: first, in terms of talent, not every financial company is matched with a strong R&D team that is keen on technological trends; second, resource investment capabilities, such as whether investment at the product level can keep up. ; The third is data security. It is very important to ensure data security within the company and in cooperation with upstream and downstream enterprises on the premise of adapting to the fast-paced business updates and timely responses of the Internet; the fourth is to build on the old infrastructure. Sensitive upgrades include infrastructure, operation and maintenance, R&D, products, etc.
InfoQ: What specifically does "opening up the upstream and downstream" in the second major change refer to?
Wu Chunlan: Digital transformation is first about thinking, secondly about organization, thirdly about process, and fourthly about platform. In terms of thinking, digital transformation is not a matter of one department. The process involves changes in organization and process, and it is necessary to ensure that everyone has unified thinking. In terms of organization and process, data opening involves cross-department sharing. When thinking is aligned, it is also necessary to ensure that the organization The level can be as streamlined as possible to quickly promote relevant decisions. For example, the launch of a new business may involve multiple departments such as operations, products, and R&D. Can everyone transparently understand the entire execution link and clearly understand the company's decision-making background? Only when every department is involved can the process efficiency be truly improved. And not just getting the job done. On the platform, whether the data can actually be used after being connected and what level of data quality needs to be achieved are all matters that the platform needs to focus on optimizing.
InfoQ: How does CEIBS solve the three major problems of talent, security and infrastructure?
Wu Chunlan: In terms of talents, China-Europe Wealth began preparations around 2014. Most of the employees recruited have backgrounds in the Internet and core financial institutions. These employees not only understand the business form of finance, but also have high technical capabilities and acumen. The entire architecture has been adapted to the characteristics of the Internet era from the beginning; in terms of security, in addition to complying with the requirements of relevant national regulatory standards, China Europe Wealth itself has also done a lot of exploration, such as preventing DDoS attacks, traffic cleaning, intranet monitoring, data security and auditing, etc. , these capabilities have been gradually established through development in the past three or four years, but it is still difficult to achieve complete automation. At the infrastructure level, as mentioned above, the initial architecture has adapted to the characteristics of the Internet era. Over the past many years of evolution, CEIBS Wealth has optimized different modules, including the establishment of a distributed database system, the optimization of private cloud systems, etc.
InfoQ: Can you give us some examples of what China-Europe Wealth has achieved through digital transformation?
Wu Chunlan: Take the investment consulting business as an example. First of all, this business needs to quickly understand customer needs and make quick and agile responses based on data-driven logic, which requires high underlying data capabilities. Secondly, as one of the first five companies in the country, A pilot company for fund investment consulting business, the main advantage of China-Europe Fortune lies in its strong independent research and development capabilities. In the past five years, China-Europe Wealth has upgraded its entire infrastructure. The underlying infrastructure matches the general trend of industry technology evolution, achieving software definition and flexible deployment, and reducing computing and operation and maintenance costs. Currently, the company's business is fully deployed on a private cloud based on K8s, which can well support the development of investment consulting and other businesses.
InfoQ: How do you view the role of artificial intelligence technology in the digital transformation of the wealth fund sector?
Wu Chunlan : Regarding the implementation of artificial intelligence technology, I think it is still difficult to implement it on a large scale. Although many companies are currently taking actions in this area, most of them are attempts, such as intelligent customer service, sensitive word review, etc. In actual business, artificial intelligence plays more of a supporting role rather than replacing the labor of many people.
Specific to the financial field, because this field is highly regulated and requires high professionalism, the existing, general-purpose large models may not be able to meet the needs well. In the future, there may be large models specific to this field, but they are still needed. some time.
Facing the future, how can CEIBS join hands with PingCAP to build a distributed database system?
The old database system before migration was based on MySQL.
InfoQ: What was the status of China Europe Wealth’s internal data system before it cooperated with PingCAP’s TiDB database?
Wu Chunlan: Prior to this, the database system of CEIBS was based on MySQL. With the gradual development of business, the traditional data system has encountered some problems. China-Europe Fortune began to think about whether there are some new tools, platforms, and products that can better meet the current demands.
At the technical level, the team faced three obvious problems at the time: First, the DDL operation of large tables, which is generally implemented through the gh-ost tool, is very time-consuming and will generate a large number of binlogs, affecting downstream synchronization. If you encounter a large table with sharding logic, the entire DDL process will take several days. The second is database sharding and table sharding. The data volume of a single table grows very fast, and table sharding is often required. However, development resources are limited, and there is not so much manpower to devote to the work of sub-tables; third, single-node writing, MySQL's traditional one-master-multi-slave architecture, the master node is responsible for application writing. When there are liquidation or batch running tasks, a write bottleneck will occur on the master node.
Distributed database selection and migration
InfoQ: When it comes to distributed database selection, what are the main factors that China-Europe Fortune focuses on?
Wu Chunlan: When it comes to database selection, CEIBS focuses on the high availability, decentralization, high performance of the entire architecture without single points of failure, and the ability to reduce operation and maintenance costs. Taking single-point performance as an example, although the MySQL era can solve the problem by adding machines, elastic expansion cannot be achieved overall. After researching the database selection of some Internet companies and understanding the existing database products on the market, the team finally came into contact with TiDB with a "give it a try" mentality.
After the selection was confirmed, the R&D team tested the stability, availability, and scalability of TiDB for about half a year. The entire platform was placed on TiDB, including the core business, to comprehensively experience its adaptation to the scenario. In fact, the database is a very complex, huge and core project, and it needs to keep pace with the times. The concepts of separation of storage and computing proposed by TiDB at that time are particularly suitable for the scenario capabilities, and after multi-party communication, its architecture is sufficient to withstand the continuous growth of data volume in the coming years.
InfoQ: What are the main steps for CEIF's database migration?
Wu Chunlan: China Europe Wealth will begin research and testing in 2021, and will begin deployment and launch in 2022. This year, we will conduct in-depth testing and complete 30% of business migration, including portfolio investment advisory systems, marketing systems, product systems, user systems and The trading system hopes to be able to run all businesses on TiDB in the future.
Looking back at the whole process, CEIBS's method is quite scientific. First, enterprises need to fully understand the current situation, clearly position needs and match appropriate products; second, the team needs to fully confirm the upgraded overall database architecture and have a clear plan for future evolution; third, cultivate talents, CEIBS Fortune has had close exchanges with the TiDB team for more than half a year, and has learned from its community, and has a full understanding of its technical capabilities, R&D capabilities, existing markets, foreseeable synergies, positioning, and technology evolution direction; fourthly, Ready for filing, that is, two-wheel drive. At first, the business ran simultaneously on TiDB and the old MySQL system. After operating in this mode for more than half a year, the entire technical architecture was well adapted (of course, TiDB itself is compatible with the MySQL protocol). After the business ran well, the formal During the migration, the teams from both parties worked together to improve the compatibility of the old and new architectures and the adaptation of downstream systems. During the migration process, the downstream will not be aware of the upstream's architectural changes. The team has made full preparations and strictly followed the plan.
The production TiDB cluster configuration is as shown above. In order to cope with complex business scenarios, over-configuration is chosen at the hardware level. In terms of architecture, the computing layer uses 5 servers, of which 3 are deployed in a mixed deployment of TiDB-server and PD, and the other 2 are used to receive complex SQL requests (resource isolation). Each TiKV server has three disks attached to it, and each disk serves as an independent TiKV node. All TiKV have a total of 3*3=9 nodes. The cluster architecture can be seen in the figure below.
Overall evaluation after migration
InfoQ: What is your overall evaluation of database system replacement?
Wu Chunlan: The first is agility, which does not require too much effort in resource allocation and can promote innovative business faster; the second is the simplification of the company structure, and the unified database architecture reduces the difficulty of operation and maintenance and upgrades; the third is The links of some computing tasks under the HTAP architecture are shortened, and risks are relatively more controllable; finally, it is conducive to future business innovation and growth.
Specific to the technical level, TiCDC simplifies data synchronization . TiCDC can synchronize data in TiDB to MySQL and Kafka (canal-json format), which greatly reduces the transformation work of data synchronization; the observability, supporting dashboard and grafana are very good use. When encountering problems or performance bottlenecks during the testing phase, you can quickly locate the problem and increase the efficiency of testing; server hardware failure , server hardware failure in the cluster causes downtime, without affecting any business; the subsequent downtime process for parts replacement is also very smooth ; Tiflash optimizes fuzzy queries . If the business needs fuzzy queries, TiFlash converts row storage data into column storage, and uses MMP to accelerate queries.
InfoQ: From an operations perspective, how have the costs changed before and after the migration?
Wu Chunlan : Overall, a lot of costs have been saved at the operation and maintenance level. For example, the original system required splitting a large number of clusters to operate dozens of applications. Now only one TiDB cluster is needed to solve the problem. The cost of such operations and computing resources (servers, etc.) is invisible, so the entire migration process It has already met the requirements of cost reduction and efficiency improvement. Of course, many businesses may be able to afford these costs, but operating efficiencies also vary. After the replacement, operational efficiency and architectural agility have been greatly improved, which is crucial in the current business scenario.
So, why didn’t companies talk about these things in the past few years? In the absence of the Internet and full competition, these issues may not be the most critical and can be solved by human power. However, in the current market environment, efficiency plays a decisive role in many cases, which forces many companies to make adjustments to their old data systems. Moreover, companies do not need to worry about the underlying selection and adaptation issues. All resources are available. With pooling, companies can put all their energy into the business itself to obtain ultimate growth.
InfoQ: What feedback do R&D students have on using TiDB ?
Wu Chunlan: From a research and development perspective, first of all, we chose TiDB after having a clear understanding of ourselves; secondly, as mentioned above, the reduction in difficulty and cost of operation and maintenance can be felt; thirdly, the sub-database and sub-table advocated by major manufacturers The technology is definitely mature, but for small businesses, the workload it brings is huge. With limited R&D resources, the cost has to be considered; then, the business needs to be timely and flexible, and the expansion of TiDB capabilities make this possible; finally, TiDB's HTAP fusion architecture solves many problems. In the past, large-scale data computing tasks consumed a lot of resources and ran very slowly. TiDB can still perform such tasks when resources are isolated. Intelligent routing and resource isolation can ensure that multiple services can be placed in a cluster, and each service is configured with a designated RU to ensure that services will not interfere with each other. When encountering sudden traffic, the explosion radius can also be controlled to help accurately judge resource consumption, and the performance is very good, which is very important for business development.
Future plan
InfoQ: What are the future migration plans?
Wu Chunlan: The overall plan is to complete 70%-80% of the business migration this year, and the preliminary preparations have been basically completed. If the progress is faster, it is expected that 90% of the business migration will be completed by the end of this year to the beginning of next year, basically covering all core businesses of the entire Internet. It is hoped that the entire underlying platform capabilities will be fully prepared before new market opportunities arrive. I believe that the future has broad prospects.
At the technical level, we will try to replace Haproxy or F5 with TiProxy in the future, which can ensure lossless upgrade of the cluster and provide high-level functions such as current limiting and circuit breaker. In the future, we can capture all SQL, realize traffic replay, and improve test efficiency; functional integration will The management and control of Dashboard, TiUniManager, DM-web, and even TiCDC are concentrated on one platform. The platform can also provide auxiliary functions such as backup management and alarm adjustment. Inspection functions often rely on people to analyze Dashboard and Grafna's Performanceoverview to judge the cluster. Condition. The inspection function can save labor costs and rely on AI to provide accurate cluster operation reports with relevant optimization suggestions.
About CEIBS
China Europe Wealth was established on August 14, 2015. It is the sales subsidiary of China Europe Fund Holdings and holds a fund sales business license approved by the China Securities Regulatory Commission. Its APP basically covers all fund varieties in the industry, and provides investors with professional and convenient investment tools and services such as fund trading, big data base selection, smart fixed investment, financial planner consultation, etc., and is committed to providing investors and partners with a one-stop Internet Wealth Management Solutions. China Europe Fund is a shareholder of China Europe Wealth, and China Europe Wealth implements business isolation from shareholders.