The Elephant Turns - How platform architecture embraces business innovation

[Yuan Chuang Conference Preview] 1024 Programmers’ Day (two days before), meet at the Open Source China office, let’s talk about AI! >>>

introduction

This is an architectural practice and inspirational lesson. If you are responsible for the architect of a very large and complex platform (such as e-commerce, payment, logistics) , and are faced with various technical liabilities (such as architectural complexity, team collaboration complexity), and at the same time, the business is faced with the transition from platform services to scenarios. The transformation of cultural innovation . Then this article may be of benefit to you:

1) How to reshape a 10-year-old funding platform structure?

2) How to build a shared platform architecture to drive the transformation of R&D collaboration models and improve overall R&D efficiency?

3) How to build a flexible innovation growth structure on the platform and reduce the opportunity cost of innovation?

1. The origin of the capital innovation platform

1. What is a capital business platform?

A fund transaction processing platform that supports various business scenarios , such as a transfer, a red envelope sending behavior (send, receive, withdraw), etc., seemingly simple actions are actually very complex behind the scenes. First, business complexity. Except for acquiring transactions, all others can be classified as capital transactions, so they carry a very complex business model. Second, from the perspective of the transaction itself, it is responsible for the matching of transaction elements, capital flow processing, and final completion of the contract. It is the core system in the payment system that connects the previous and the next.

2. Transformation from platform to scenario-based innovation

The treasury business platform has experienced multiple stages of evolution: from the Jushi application at the beginning to service-oriented and platform-oriented during the rapid development of the industry. In recent years, the capital business has been rapidly transforming from tool-based products to scenario-based services. How to support innovation and trial-and-error "both stably and quickly" is a huge challenge to architectural design.

2. Problems and Contradictions

1. Appearance contradiction

The contradiction between the demand for rapid trial and error in innovative business and the R&D delivery cycle is high. The threshold for capital innovation is high. The R&D delivery cycle generally starts at one month. Blindly adding people can no longer solve the overall delivery efficiency problem. The application architecture of the funding platform has historically been Along with the development of products and business lines, a situation of tripartite confrontation and hundreds of schools of thought contending (individuals, businesses, and groups) has been formed. With large-scale innovation, business boundaries are gradually broken, and capabilities are repeatedly built and the quality is uneven. Although the team once expanded on a large scale, due to platform debt and complexity, it was unable to stimulate the motivation for innovation at the business layer, resulting in being light-headed and unable to move forward.

2. Problem definition

2.1. Chimney-style capital platform structure

The chimney structure of capital platforms with high marginal costs and duplication of capabilities has achieved flexible development and local optimization in historical periods. However, with the development and integration of businesses, the cost of repeated construction is far greater than the advantages brought by flexibility. For example, tob's corporate payment and toc's small wallet, the bottom layer is based on the innovation of shared accounts, and all accounts The asset model and deposit and withdrawal process all need to be built, and the huge investment in research and development is obviously not in line with the intuition of the business side.

2.2. Architecture for Internet growth

Alipay is good at being a payment platform, but when faced with the construction of Internet-oriented scenarios and large-scale business growth, it lacks relevant architectural experience, and its R&D efficiency is lagging behind, and it cannot keep up with the pace of innovation and trial and error.

3. Overall design

1. Architectural Vision

In the past year, we launched a funding innovation platform project group. It defines the current and future unified platform architecture goals for the large capital domain for capital business innovation. In terms of overall design, we have a vision to design the next generation of innovation-oriented funding platform architecture to make the funding platform more stable and make business innovation more focused on the delivery of innovative logic. The figure below is the direction of the first version of the design.

4. Key Architecture Design (1) Reshaping the Capital Platform Structure

The capital platform (funding transaction processing) is the root node of all capital business innovations and supports the growth of all upper-level business capillaries. However, in past practice, due to its "three highs" characteristics (high technical entry threshold, high implementation complexity, and high stability requirements), the delivery cycle is too long, which is the first impression most frontline businesses have of the "payment team" , seriously restricting the expansion of innovative business. Based on the governance of existing structural liabilities (multiple chimneys), as well as the judgment of future scenario-based innovation and the demand for the delivery efficiency of financial capabilities, we have made an overall reconstruction of the capital domain structure. Overall, it is divided into three stages: domain model reconstruction, platform design, and capability product design.

1. Domain model reconstruction

1.1 Summarize domain boundaries through use case analysis

Domains must have clear boundaries, that is, find an appropriate dividing line (what to do and what not to do) to decouple the complex relationships between domains. The most critical thing is to identify "conceptual classes" - nouns (fields), adjectives (capabilities), and verbs (relationships) - from each business use case. The second step is induction and clustering, dismantling it into different areas according to the purpose of independence and reuse. Defining the domain boundaries between services takes the red envelope business as an example for analysis (see the figure below): red envelopes are passed among friends through various gameplays. Funds, driven by red envelopes, are transfers of funds between payers and recipients. After formatting the requirements, we can easily identify two types of independent fields and capabilities. According to the reusability, we can abstract and sink the fund processing capabilities to facilitate reuse in other scenarios.

Define the domain boundaries within the service . The essence of the capital field is to convert the fund transfer needs of upstream weak transaction business scenarios into capital orders, and use the orders as the carrier to carry out business activities around people, accounts, business entities, and business assets as participants. Payment and asset settlement services. Funding business domain modeling L0: From a complete business perspective, abstract peripheral dependencies and coordination capabilities of services (cashier, payment, charging, permission restriction, security, etc.) into reusable domain components , and integrate orders in the component dimension The contract parameters understood in the domain, the command parameters from the upstream source, and the call parameters to the downstream system support various business activities;

Funding Core Area Modeling L1: Split the previous fund order model into fund orders and fund flow models to make explicit the differences between fund order flows related to business behavior and fund flows related to fund settlement; at the same time, abstract participants and businesses The asset model finally provides the ability to expand capital participants, expand business assets, flexibly orchestrate capital order flows, and flexibly orchestrate capital flows to solve the complexity of capital processing and improve future scalability.

1.2 Reasoning about the unified trading model through deduction

Can trading models be reused across industries? In addition to induction from the past, a deductive process is also required, that is, through the business life cycle and business elements, we can put forward hypotheses and reason from a more macro business perspective. For example, we assume that various order transaction types are essentially The vehicle for matching participants, assets and payments. This vehicle can be combined by different transaction models. Let’s go back to the red envelope business use case we just talked about. The red envelope business represents a multi-stage order flow transaction model. This transaction model can also be reused in other business scenarios, such as DingTalk transfers (requires confirmation from the payee) payment). In addition, we serve many B and C scenarios, including B2B ( single transaction, collection ), B2C (single transaction, batch, multi-stage), C2C (single transaction, batch, multi-stage), B2B2B It can be said to be one of the most complex transaction models in the payment system. One of the core pain points of the old code is the unreasonable design of the order field model. The old system is relatively rigid in the field model design of the above different transaction scenarios: such as batch, two-stage, and single. In the system code design, the corresponding model and code design are both It is a chimney design that cannot digest the demands of business innovation for new transaction models. The order model needs to bear more downstream coordination capabilities. The chimney-shaped design of the domain model brings more complexity to the "coordination" work, making the coordination capability itself unable to be reused. Reshape the order domain model: extract commonalities and take advantage of the polymorphism of the Java language. We abstract the common parts of the domain model into a basic order model. On this basis, we use transaction models such as parent-child orders and batch orders as derivation or implementation of the basic order model. . In this way, the downstream coordination domain model no longer understands the specific transaction model, so that the impact of changes and additions to the transaction model on the system can be effectively controlled.

2. Platform design - shared business platform

The platform design aims to achieve maximum asset reuse and can be flexibly expanded through configuration. Generally, teams based on platform-based design can complete complete demand delivery based on interface contracts, such as e-commerce product platforms and contract fulfillment platforms; payment acquiring platforms, payment platforms, accounting platforms, etc. So what is a "shared business platform"? Due to the particularity of the capital business, there are multiple business units, multiple technical teams, and in some cases there will be overlap (the B business technical team takes over the needs of the A business team). Obviously, the self-closed loop delivery of the platform can easily form a delivery Hotspot. The business technology team is close to the business and can better arrange priorities. Based on this consideration, the concept of "shared business platform" was proposed at the beginning of the design of the capital business platform. Similar to a shopping mall complex, the platform team is responsible for the planning and construction of shopping mall infrastructure (such as water, electricity, coal, and regional planning). What business each store does, how it is decorated, and how it is marketed can be determined by the business as long as it complies with the regulations of the mall. The team takes direct leadership.

2.1 Key design to achieve platformization

Component design

Inspiration from the Ford automobile assembly line Regarding how to upgrade from part-level reuse to module and fragment-level reuse, one of our classmates proposed that we had learned that the entire vehicle architecture also has a similar reuse design. We shared and learned with a learning attitude. After observing the evolution of the vehicle architecture over the past 100 years, we unexpectedly came up with key problem-solving ideas and further confirmed the correctness of our architectural design ideas.

1) Abstract module - compare and find commonalities among differences. Abstract the greatest common denominator that can be reused by all car models into modules.

2) Achieve business customization through module assembly - the differences within the same module appear as different functions of several parts, but are essentially multiple parts to solve various needs, such as safety, size, energy, emissions, etc. design differences. For example, each module must understand the constraints of "security level", and each module of the vehicle has different modes of performance under different security levels. To summarize the idea of platform-based vehicle building: in order to meet different personalized needs and improve assembly line production efficiency, the entire vehicle architecture is decomposed into a number of independent and interrelated modules according to certain principles to achieve maximum universality of module parts and have the ability to pass The ability to adjust and combine different modules to produce models of different positions and levels. How to realize the adaptation and assembly of components. We were surprised to find that another completely different industry has so similar ideas for solving architectural problems. Through the above explanation, we can summarize the solution after thinking. Idea:

Parts are abstracted into modules, a higher dimensional level of abstraction.
Abstract the constraints that each module must understand into a unified interface. Each module does not depend on each other, but relies on various indicators to constrain interfaces and connection interfaces, that is, dependency inversion.

Therefore, we have upgraded the reusability granularity from the original atomic Java method to the module-level reusable granularity of components, component patterns, process stages, and processes.

The reuse granularity will not be too large. If the reuse granularity is too large, such as business processes, it will result in the inability to digest more transaction scenario abstractions and differences, and ultimately lead to the gradual increase of business processes.
The reuse granularity will not be too small. If the reuse granularity is too small, such as Java methods, it will not only be impossible to manage, but also the reusability will not be strong.

Extensible design

The components have been abstracted and reused, but we still need to meet the customized requirements in different scenarios. It is also the fund transfer from A to B, but in different scenarios, the payment rules are different, such as the refund cycle. , quota, payment channels, security risk control, etc.

Let’s look at the solutions that car sales manufacturers have when dealing with customer customization. First of all, although many vehicle manufacturers use a common production platform, due to different market positioning requirements, their products will be classified into several models of sedans. Several models of SUV and several models of MPV. These different models themselves are different products built on the same platform architecture with different "product-specific constraints". Moreover, when we enter a car 4S store, the sales lady will often let you choose several "packages" for a certain model, such as the Night Edition, Sports Package, etc. Through the above platform structured thinking analysis, it is not difficult to find that these so-called "packages" are actually some "specific market demand constraints" for these car models. For example, under the constraints of the "sports package", the wheel hub parameters of the car will It will become larger, the seats will be equipped with sports waist protection kits, and the color parameters of the car will become cool blue, etc. And when you ask whether seat heating functions can be added to the sports package, most manufacturers may support these more personalized customizations.

(Business architecture layering)

By analogy, we can also provide customers with standard products, additional packages , and personalized customization based on the platform architecture. We can build a product layer, and based on the scalable and configurable capabilities of the platform layer, we can select specific product features and package them into standard products. At the same time, for business customization, optional "business capability" packages are provided, and personalized business configuration and business code expansion can also be done based on different business identities applied for . For many platforms in the industry, Alibaba e-commerce's business capability design essentially forms different levels of business reuse architecture through layered delivery and layered governance, which coincides with our architectural design ideas.

2.2 Shared development and shared runtime

Scenario expansion and platform service isolation realize shared R&D model

After architectural decision-making, we abandoned the past application architecture of two applications (product layer prod) and (core layer core), merged them into one large platform application, and evolved to a business platform application architecture. From the front-end product layer decision-making extension to the scenario-based decision-making extension within the platform, the scenario-based extension is completely separated from the platform in the code in the form of an expansion package, and a "business containerized delivery" model is proposed, using the serverless application architecture model to solve the problem. The development state and deployment state of the coupled scenario expansion package. This method of decoupling business and platform through Serverless Ark module package is very suitable for shared R&D teams that must deliver business on one platform because they cannot split applications. It can achieve both platform and business on the same platform. The code state and running state are completely isolated, and the platform-side capabilities are pure and pollution-free; it also maximizes the reuse efficiency of platform capabilities.

3. Platform capability product design

The software world and the real world should be a process of continuous fitting - through model reconstruction and platform design, the assembly of financial capabilities can be said to be more flexible, but we realize that only technical flexibility is not enough. It is hoped that the internal capabilities of the technology platform can be fully described and externalized into product capabilities (product processes, product functional parameters). Only in this way can the platform continue to stay fresh and maintain iteration.

3.1 Fitting of products and technical capabilities

We all claim that we are in business, but these core business concepts cannot be found in our code at all. This leads to extremely low efficiency in business and technical communication collaboration. What originally required one week of work basically took one month to implement. , and the real code may be solved in just a few lines. This kind of problem is often very common in the payment business. As shown in the figure below, in the past, our mode of receiving requirements was this general pipeline. The reuse of capabilities relied entirely on the architect to abstract while receiving the requirements. What abstract capabilities are there, PD often relies on Word of mouth. We often talk about domain-driven design - more emphasis on business domain architecture. The domain abstractor and the business should be closely connected, but our actual situation may not be like this. Therefore, there are difficulties in function reuse, difficulty in product inheritance, and weak product operation capabilities. The essential reason is the inconsistency between "product capabilities" and "technical abstraction."

(The description of products and technical capabilities relies on experience)

3.2 Explicit expression of product capabilities

So, is there a way to make product capabilities and technical abstractions more closely aligned, and even make product capabilities into "nocode of product functions" that product managers can write, and the Java code written by R&D Can some kind of connection be made? We think it is possible, we just need to do:

1) Deliver technical abstract codes as standardized components and component extension points.

2) Provide PD with a product function workbench, allowing product functions to be defined in a codeless manner. PD creates a new capital product on the workbench and abstracts the product function. We abstract the product function and let PD design a form that can express this function. This form can also be used as an operational position for the product later. We call it operation. view.

3) Establish a connection relationship between the product functions defined by PD and the abstract code of the technology (merger, association, cascade, AND or, condition, etc.)

(Technical specifications converted into product specifications)

5. Key Architecture Design (2) - How to Make Innovation Run Faster

In the previous part, we focused on the platformization and assembly line production of financial capabilities. The focus in this part is how we can improve the speed of end-to-end R&D delivery of business and help the large-scale growth of capital scenario innovation. If capital transaction capabilities are the core engine, then capital scenario innovation is to connect various parts and components to build a complete vehicle and quickly push it to the market for sale to achieve growth. The key here is to be "lightweight" and "agile". Back to payment, through the experience of launching multiple products in the past, we have also summarized the product life cycle of innovative business, which is roughly divided into three stages: scenario construction, operational growth, and insight iteration , as shown in the figure below.

It is not difficult to find that scene construction generally requires three major types of capabilities:

1) Scenario model-oriented construction of CRUD (new) : For example, making an account relationship and aggregation, bill dynamics and comments, all of which have domain characteristics, and I have searched the entire site and cannot find a reusable platform system. This Sometimes we need CRUD capabilities to deliver services in new fields.

2) Service orchestration (reuse) in the middle and Taiwan fields : For example, it involves account (shared account), fund transactions (recharge, transfer, cash withdrawal), social networking, asset core, cashier payment, security and other fields. We have obvious capabilities in these fields. There is no need to repeat the construction. We only need to coordinate Ant's huge middle-end system. As shown in the figure below, in our link, fundapplication relies on the capabilities of many other domains. In the end, we orchestrated and opened the mobilegw interface, and agreed with the front-end on the interface protocol standard.

3) Finally, pages and processes are built through front-end components. Summary: The domain essence of capital scenario innovation = construction of scenario domain CRUD (new creation) + orchestration and aggregation of existing domain capabilities

1. Lightweight construction and growth system

Some of the scene construction are lightweight pages and processes. Here are a few common cases of our innovative business in CY21, such as the account opening scene of Ant Hehua (Xiaohebao) product, the scene of transferring living expenses, C/B New balance mini program position, etc.

The product development of these fixed positions and marketing positions is basically divided into front-end mini-program interactive pages (front-end) + content services (server-side) + simple functional interactive services (server-side). In addition, product and operation students may Provide some refined operational requirements and post-operation data analysis requirements for these positions. One-stop construction and growth platform We started to build a low-code construction platform in CY21: a one-stop operation platform with multiple capabilities such as refined operation capabilities, rapid construction, data intelligence, and multi-terminal openness to improve R&D and operation efficiency. 1) Rapid construction: Provide a one-stop solution for rapid low-code construction for field/position/product pages, improving the research and development efficiency of front-end and back-end students. ——Essentially, in this area where the model is relatively fixed, the traditional manual service-based aggregation model is upgraded to a template-based method that integrates front-end and back-end, and is upgraded to a product. 2) Refined operation: one-stop page/module-based refined operation capability for business self-service, which greatly improves the efficiency of business operations. ——Templification can definitely evolve to the nocode method, and nocode will definitely bring about changes in production relations. This part of the work will change from technical personnel to business self-service, which can be said to be a win-win situation. 3) Data intelligence: Complete full-link data standards and standard investment data effect products, through unified burying and off-line data insight systems, inject operational strategies and business indicators in the operation process to meet the needs of operation students in the operation process. Data insight needs to improve the transformation of business growth. ——After unified templates, data can be easily unified, which is also a business dividend brought by unified and templated data.

We use domain abstraction and lowcode in the field of operation page construction to achieve integrated front-end and back-end templates to improve efficiency. We take one-stop construction as an example to describe how we use lowcode to improve efficiency. At the construction level, we abstracted the content and strategy domain models, unified the orchestration and aggregation of our downstream dependent capabilities based on different domain perspectives, and finally cooperated with front-end students in the investment module to formulate protocol standards and front-end component precipitation standards: 1) Through the content interface and strategy interface, all the capabilities of Ant’s major content platforms, strategy and experiment platforms are organized into one-stop basic capabilities, and converted into a product development system with configurable delivery, allowing content, Strategy-related needs do not require R&D. 2) The investment module abstracts an operation page as a whole, dismantles the structure of the operation page as a whole, and works with the front-end team to precipitate a number of horizontally reusable front-end components and templates. Aggregate the content, data, and operation strategy services that the back-end services need to provide to the front-end into a set of mgw standard interfaces. Reduce the research and development costs of page front-end construction and back-end services by more than double.

2. Lightweight application R&D model

The one-stop construction of the Serverless innovative architecture solves the construction and operation of lightweight scenarios. Although it covers a wide range, it has its own limitations and can only be orchestrated and constructed based on simple components and mature capabilities. Complex scenarios still require standard back-end services to support the output of scenario service capabilities (including the construction of scenario models and service orchestration). During the evolution of scenario-based innovation, in order to decouple funds processing, we specially built a fund scenario-based application (fundapp). In terms of R&D collaboration, we also started with a few classmates, several business teams, and dozens of people. R&D students all work on an innovative application and use multiple bundles to distinguish different businesses. The public bundle accumulates general capabilities and R&D tools, as shown in the figure below.

This kind of architecture design has gradually become a monolith application when encountering explosive innovation. It faces several main problems.

1) Low R&D and operation efficiency: As an entrance system, the gateway interface cannot be developed and tested locally, and any code changes must be published to the joint debugging environment. Publishing once takes 10 minutes, and you have to endure the problems caused by unstable environment.

2) Business iterations are preemptive , collaboration costs are high, and the pain points of train-style release iterations are obvious—one person is late, and everyone is delayed.

3) Low isolation efficiency during research and development: As a single system undertakes more and more businesses and various dependencies are introduced, it takes 15 minutes for the system to start up. As innovation scenarios increase, things will get worse.

4) No isolation during runtime: At present, we do not distinguish different computer rooms for different businesses. Businesses eat from one big pot, and it is difficult to guarantee resources and stability based on products.

2.1 Selection of light application architecture

To solve complex problems, the idea of divide and conquer is generally used, but according to what dimensions should we divide and conquer? It’s not just about business architecture, it’s also about the design of application architecture. If microservices are roughly divided according to scenarios and teams, obviously with the explosion of future scenarios, operation and maintenance costs and basic capability construction costs will still be a big expense. , innovation and trial and error still can’t come up quickly. At the beginning of the top-level design of the innovation architecture, we hoped to use a technology to "allow the business to focus more on the research and development of innovative logic and pay as little attention as possible to the operation and maintenance of the infrastructure." The infrastructure here is not only traditional infrastructure and operation and maintenance (such as monitoring, upgrades, expansion, etc.), but also includes basic service capabilities (general business services, R&D framework, etc.). The flexible development of innovative logic refers to the requirements for rapid development of scenario models, capability orchestration, and mutual isolation between scenarios. This is actually the primary form of Serverless. Specifically, we want to reduce the granularity of the scenario from "microservices" (complete applications) to a smaller "module" level, and then dynamically deploy it to a large innovative base application. . The characteristics of the module may be: independent research and development, independent hot release, and independent logic operation. However, calls between modules should preferably be inter-process, intra-process, or even without the need for heavier serialization modes such as Hessian. In CY21, Ant has a modular framework such as sofa-ark, which provides a R&D framework for separate R&D and deployment of multiple modules within the same JVM. At the same time, the supporting operation and maintenance system and product system for this framework have also begun to take shape. After POC technical verification and architectural decision-making, we are very sure about the evolution direction of this architecture and decided to join the start-up team as a seed user and incubate SOFAServerless as a model business .

2.2 How to isolate and reuse

In the SOFA Serverless system, we regard each innovative business as an independent business module, and the business modules can independently provide external services. At the same time, you can also sink all the general productivity tools mentioned above into the innovation base, including rapid business development tools and frameworks, business templates and scaffolding templates, business module development protocols, etc. During the evolution process, when the business modules found that there were some reusable business capabilities that needed to be accumulated, we also introduced the division of public modules (later evolved into public scenarios), which allowed business modules to call each other without losing performance. SOFA Serverless represents not only the advancement in the field of deployment and operation, but also the advancement in the field of business architecture design. Through this new architecture system, it is possible for our innovative efficiency-improving formula to be fully implemented.

6. Architecture design method

1. Macroscopic design method of architecture

The purpose of architecture is to continuously reduce the complexity of the system and meet the future adaptability of the business through advance design. There are general principles (commonly known as routines) for solving complex problems. In the field of software development, a relatively rich set of design principles and design patterns have also been accumulated, which are the basic concepts that guide us in designing excellent architectures.

2. Architecture is a trade-off between pros and cons

Architectural design is a matter of trade-offs and trade-offs under the condition of meeting objective constraints, such as resource constraints, team technical level, efficiency requirements at different stages of the business, etc. We know that platformization can improve the reusability of capabilities, but it limits the flexibility of innovation. For example, in the process of designing the capital innovation platform, we also fully considered the trade-offs in abstraction granularity and divide and conquer: through the "large-grained" abstraction of the capital platform, we solved the complexity of various transaction models and the problems of capital security. In the light application part that is more innovative, we adopt a "smaller-granularity" tool-dimensional abstraction to minimize the intrusion of scenarios into general capabilities and make business innovation more independent and flexible. At the divide-and-conquer level, the serverless modular architecture we designed is actually a balance between single applications and microservices, taking into account the reusability of the base and the flexibility of the modules. Of course, the prerequisite for making a choice is to first have a full understanding of the pros and cons of architectural design methods, so that you can make better choices under objective constraints, as shown in the figure below.

In short, there is no silver bullet for architectural design once and for all. Under the guidance of general methods, we must grasp the core contradiction at the moment, evolve during design, and redesign during evolution.

Author｜Lu Jian

Click to try the cloud product for free now to start your practical journey on the cloud!

Original link

This article is original content from Alibaba Cloud and may not be reproduced without permission.