Author | Hongyuan Jun

guide

This article takes the evolution direction of Baidu's vertical offline computing system as the main line, and describes in detail the problems encountered in the development of the search vertical offline computing system and the corresponding solutions. In the process of architecture evolution, the tenet of "there is no best architecture, only the most suitable architecture" has been adhered to, and suitable solutions have been given for the problems encountered in different stages. Especially for the upgrade of ultra-large-scale system architecture in the past 10 years, on the one hand, it is necessary to consider the versatility and adaptability of the system itself to meet the needs of multiple business parties; Improve performance, stability, intelligence and other aspects. I hope readers can gain some inspiration in the process of understanding the evolution of the system.

The full text is 9127 words, and the expected reading time is 23 minutes.

01 Related background introduction

In the past, the search results obtained by users through "Baidu click" were the results crawled from the Internet, also known as "natural results". With the increasing abundance of network information, natural results cannot effectively meet user needs. In order to solve the problem that natural results cannot meet the search needs, a solution for search results of various vertical categories is proposed. On the one hand, it brings better content to users, allowing users to experience the convenience of instant search. On the other hand, It can also help high-quality content producers increase traffic.

With business development, in addition to changes in standard and general business processing requirements, more and more businesses have custom code update requirements. Through custom data processing, on the one hand, the students in charge of the product can customize the original incoming data according to business needs, and convert the original data into the final search result data. On the other hand, some general default functional mechanisms limit the development of a small number of vertical categories. The business hopes to introduce more policy model logic signal processing and scoring mechanisms to improve the effect of sorting and recalling.

In this context, the business has increasingly strong demands on the framework engine for data processing and computing, and its functions have gradually become an indispensable part of the entire offline processing process of vertical search.

02 Computing system evolution process

The offline data processing of Baidu search system has gone through the following stages in terms of the development stage of the timeline:

a. Original offline processing system: This stage is mainly to realize the construction of business processing entry from 0 to 1 . On the whole, a complete framework system has not been formed and the development cost is high, and all business logic is mixed in a public service, and different data invokes different policy logic through different configurations.

b. Business offline processing architecture: At this stage, a complete set of business service processing framework is initially formed , with a unified service framework and development stage, realizing the cloud native and service isolation model.

c. Serverless Architecture: The access efficiency of the business is further improved at this stage . The business is shifted from management service to management business processing function . The business can quickly test and go online by registering the function. At the same time, it supports the automatic scaling of the container instance, so that in the business The use efficiency has been greatly improved while the corresponding resource cost has been sharply reduced.

d. Data intelligence architecture: On the basis of the original service deployment, it realizes data management at the same time, further upgrades from function management to demand management , realizes multilingual service framework support, and further reduces costs and improves efficiency.

03 Computing system core design core ideas & core implementation

The detailed characteristics and core implementation of each stage are described in detail below.

3.1 The original offline processing system

This framework was applied before 2018, when the overall development of the vertical category was in its infancy. The main requirement of the business at this stage is that the original business data can be directly online. The main energy of the architecture focuses on the construction of general business capabilities. With the gradual deepening of the business, it is found that a small number of businesses have gradually evolved the need for custom processing.

According to the business needs at that time, the business data processing system was finally derived from the original data processing module . The current data processing method is to use a unified Handler processing method. Of course, this processing is not mandatory, and most businesses still do not need custom processing. There is no data isolation between businesses at all . The core meaning of this architecture is to provide custom processing capabilities for businesses from scratch . As shown in the figure below, the computing system only includes the computing engine part at this time, and other systems are not built at all. Some details of the system will be described in detail below.

3.1.1 Service Features

There are two main characteristics of business in this period:

For most businesses, the most important core appeal is the construction of general capabilities.
For individual businesses, there is a small amount of simple custom processing needs.

So the core of this point in time is mainly to solve the problem of business processing entrance.

3.1.2 Core Design

The offline architecture module of the initial version is very simple, as shown in the blue part in the figure below. With the development of business, in order to meet the needs of business custom processing, the red data channel is introduced, in which the unified data processing module mainly handles business custom processing logic.

As shown in the figure above, the gray part is the data system accessed by the outer layer of the business. At the beginning, there were only XML, POST and data queue modes. The blue part is the original data frame code when the business is not customized. The red part is the framework module introduced for business customization requirements:

User data proxy : It is mainly to adapt to the business data access method, and the data protocol is evened out. It is mainly used for RPC of different user protocols to convert data at the database construction layer.

General needs of users : mainly for unified processing of common needs of users, and functional processing according to different configurations of different businesses: such as conversion processing of business pictures and videos, business user version management and user core mechanisms, etc.

Unified data processing module : mainly to unify the processing and processing of user data, mainly to provide an entry for data processing for all connected business data.

3.2 Business Processing Architecture

As mentioned above, the core value of the 3.1 architecture is the construction of custom processing capabilities from 0 to 1. However, with the rapid development of business, more and more businesses require custom processing capabilities. The demand for business custom processing has rapidly expanded by 10 times from the original single digits. Therefore, in order to meet the needs of the business for ease of use and stability in custom development, the original 3.1 centralized processing mode is upgraded to a service framework mode . Generally speaking, the computing system construction at the current stage basically has two layers: service layer and business layer:

Business layer: Business can basically be realized through the platform, basic service management.

Service layer: Refactor the computing engine so that the new service framework supports efficient business development.

Some details of the system will be described in detail below.

3.2.1 Service Features

Since dozens of business customization requirements of the previous generation system in 3.1 were developed in the same module and run online in a unified way, the main problems encountered were as follows:

Efficiency : The business is developed in the same module, which often encounters conflicts during the process of business development and online, and the problem of online queuing, resulting in a relatively long overall effective cycle.

Stability : Due to the poor isolation of this kind of business traffic mixing mode, problems with a single business often affect all businesses.

Therefore, the architecture builds a complete service framework, and the business can build its own business code based on the service framework to solve the above problems in terms of efficiency and stability:

Efficiency : The process of service development and launch is completely isolated, there is no business blockage during the process of service development and launch, and the overall service launch cycle is shortened from days to hours.

Stability : Since each business service execution logic has a completely independent app, the stability of the business is greatly improved, and there will be no mutual influence between businesses.

3.2.3 Core Design

Business processing framework : build a data processing framework based on which businesses can implement their own business processing functions.

Task platform : Each task registration, upgrade, and management can be managed through the task platform.

Unified gateway : a unified entrance for all data, unified reception of upstream data, and forwarding and distribution of data.

Business APP : Each business has its own independent service app. Each business development is based on an independent branch development of the framework. The online is for each business to go online separately, and the service traffic of business processing is also completely isolated.

3.3 Serverless Architecture

As stated in 3.2 above, the problem of system isolation has been solved, and the rapid development of business has grown from dozens to hundreds of business applications. When the business develops to a blowout state, the form of the service framework can no longer meet the business development needs.

The serverless architecture is shown in the figure below. At this time, the overall layout of the computing system is relatively complete. Compared with the previous generation computing architecture, it not only optimizes the entire process included in the business layer, but also provides complete tool functions; the service layer fully considers the design of the architecture. and multiplexing, while increasing the control layer to increase the function of intelligent scheduling to dynamically schedule resources for system traffic.

Some details of the system will be described in detail below.

3.3.1 Business Features & Problem Solving

Business learning cost : A large number of new business accesses need to be developed, resulting in the learning cost of the framework itself taking a few days, and the time for new business access and development is basically at the weekly level.

How to save resources : A large number of services also bring rapid expansion of the app. The hundreds of connected machines have been used up, and the resource bottleneck has been reached. The current amount of resources cannot support the rapid expansion of services.

3.3.3 Core Design

Application layer : Aiming at the basic service architecture, various cloud services are open to the business, from business access and development to subsequent debugging and testing, to monitoring after the service goes online.

Service layer : This part is the basis of the overall architecture, and the application layer is developed based on it. This part is the cornerstone of the entire system and belongs to the core framework of the overall system.

Scheduling layer : Through the expansion and contraction according to the traffic pattern, it can ensure that the follow-up services can be automatically operated.

The following focuses on the main content of the service layer, which mainly includes two parts for business:

Extremely abstract business framework : It is the foundation of the core framework, providing a new development paradigm and laying a good foundation for subsequent intelligent scheduling.

Highly reusable basic services : Encapsulate powerful and rich back-end service capabilities, support business reuse at low cost, reduce development costs and improve stability. At the same time, the system also provides powerful orchestration capabilities, and supports business development from simple to complex at low cost.

Extremely abstract business framework

As the carrier of the FaaS layer and business code, the business framework is the code framework of the entire business logic. The framework itself maintains data flow semantics, and calls business function codes for directed acyclic graph (DAG) data flow. The business framework is real-time computing for directed acyclic graph data streams. Based on the company's basic data stream framework, it supports complete streaming computing semantics. The business framework is constructed in combination with the functions required by business scenarios:

The left side of the figure above is the actual execution code of the user. For the interface definition of the function: both input parameters and return values are of Dict type. The code logic of the business is directly implemented in the function. For the data that needs to be changed, as shown in the figure above, the business adds a timestamp field under the root directory, and then passes the updated result to the downstream.

The business side is developed using a scripting language with low development costs (such as Python), and the basic service framework is implemented using C++. Combined with data compression, batch processing, data pre-distribution and other mechanisms, the business can simplify the development of the service framework and optimize the service performance. The balance between service performance and development cost is achieved through optimization strategies at the architecture level.

Highly Reusable Basic Services

The back-end services that the business depends on include ten service capabilities such as multimedia persistent services, data storage services, and policy calculations. The support goals of all service architectures are service access through simple configuration and a small amount of code.

General capabilities of the architecture : including index processing (inverted, forward, vector index), data review (political sensitive data/pornographic data identification & filtering), multi-channel distribution, data database building and other capabilities.

Joint research and development capabilities with business : low-quality data filtering capabilities (implementing data cleaning/normalization/data deduplication/category splicing), multiple data fusion, data quality scoring calculation (quality scoring/cheating identification/material scoring).

Based on the company's strong basic capabilities : multimedia processing services (external links to internal links/OCR/watermark calculation/repeated image calculation/subject recognition/video dumping, etc.), natural language processing services, data precipitation services.

The supported basic service (BaaS service) has two main characteristics: simple and stable & fully integrated with other high-quality capabilities in the company . Users can directly reuse capabilities through SDK calls, operator reuse, and data stream reuse . There is no need for service deployment and management. Service usability and stability are handled by the search center. Any service backend can be closed in one place, avoiding frequent communication and processing of business with multiple service teams, and greatly reducing the cost of business use. The initial business access usually only requires a few simple functions (for example, modifying information in some fields), and most businesses can be developed directly using the platform-based development templates we provide. However, as the business is gradually deepened and gradually used, the business will gradually transition to complexity. For example, a certain business in Taiwan, in the search center, builds a data system with deep customization through the combination of dozens of capabilities.

3.4 Data Intelligence Architecture

As stated in 3.3 above, the problem of service access efficiency has been solved to a large extent, and resource utilization efficiency has been greatly saved according to the expansion and contraction of traffic. The number of business apps has grown to thousands. , the business puts forward higher requirements for efficiency, cost, and service quality. So far, a four-layer architecture has been constructed from the business layer, logic layer, service layer, and control layer to realize the complete transformation of the offline computing system from an instructional computing system to a declarative computing system.

3.4.1 Service Features

Most of the iterative development of efficiency business is aimed at a few fields, but current business students still need to understand the full topology of the service to develop.
With the development and iteration of business complex and deep plowing, business developers have multi-language development to improve the efficiency of service development and execution.
How does the business achieve rapid access to new categories and rapid iteration without comprehensive understanding.
The calculation engine has higher execution efficiency, and uses fewer resources to calculate and run more service resources.
Whether business problems can be quickly located & discovered and automatically handled.

3.4.2 Core idea

From the perspective of design ideas, the current system architecture is an extension of the previous generation architecture:

Data management: In addition to the original service management, comprehensive data management is introduced to lay the foundation for column calculation.
Orchestration capability: Extend the function capability of the original business manual orchestration to the ability of demand declarative automatic orchestration.
Service processing: The extremely efficient service processing capability supports the efficient development efficiency of various business developers, shifts the overall calculation from row calculation to column calculation, and improves the overall calculation reuse.
Control ability: Expand the original automatic scaling ability into intelligent control ability , so as to achieve automatic identification, automatic distribution, automatic analysis and automatic processing of problems.

The core design concept of the overall architecture is roughly multi-layer abstraction and layered reuse .

3.4.3 Core Design

The core design at the current stage has presented the abstraction capabilities of a complete four-layer architecture:

Application layer : related services that are directly visible to the business, including various applications in each stage of the full cycle of the business from data access to service operation.

Logical layer : It can also be called the orchestration layer, which is responsible for transforming the original business demand expression into real online services. The business selects its own function set through the Codeless platform and submits the corresponding data mapping relationship. The provided function & data binding relationship is transformed into the custom function and mapping relationship of the business.

Service layer : The core of the computing system. The actual business computing runs on this layer, which mainly includes two parts: the computing engine and the service architecture. Undertake the orchestration results of the logic layer upward to run the service, and provide the basic signal downward as the input of the control layer.

Control layer : includes two parts: intelligent scheduling and intelligent control: intelligent control is mainly to ensure service stability through intelligent control by automatically receiving business index data, while intelligent scheduling is not only automatic scaling according to data traffic, but also traffic according to business service relationships Reuse, reduce business double counting.

Below we will explain the design of some core systems (such as the blue part in the above figure).

Requirements Expression Logic

The core of the expression of demand logic is how to transform the original functional requirements of users into expressions of operators, configurations, and relationships of real online services. The most primitive user input consists of two parts:

Business data : Business data needs to be divided into the smallest unit of function, and subsequent operators perform configuration transformation and columnar calculation of service binding. What I said here is a bit of a mouthful, in fact, it is essentially just to register the original Proto or data schema of the business data.

Function set : The function set can be the function set of the default template provided by the system directly used by the user, or it can be a set customized by the user through the platform. It is worth noting that each function has its specified incoming parameters, such as pictures For calculation, you need to pass in the image URL, and for vector calculation, you need to pass in raw data & vector parameters, etc. After the function is completed, there will be corresponding output results that can be specified.

After obtaining the user's original input, the architecture converts the original user requirements into online service deployment information through two-layer logical mapping :

Demand expression service : the original demand of users is transformed into a combination of functional templates . In this stage, the original operator and data will be bound according to the user configuration, and the abstract requirements of the original user will be instantiated into a set of operators with data.

Demand orchestration service : through the expression and combination of the dependencies of business operators themselves and the blood relationship of data dependencies, operators with data are constructed into N directed acyclic graphs.

The overall demand expression can be expressed in a Codeless way. By describing the function set✖️data set description method, coupled with the transformation of two-layer logical mapping, the original user declarative demand can be directly transformed into online service relationship and system configuration , and the service The topological relationship of the operator can be deduced automatically through the expression of business data relationships, and more than 95% of the functions can be fully automatically configured and realized. Of course, there are individual services that cannot be transformed and expressed at low cost, and manual interfaces are also provided.

Calculation engine implementation

The core components of the computing engine layer are roughly divided into two parts, graph computing engine & multilingual operator execution engine.

Graph Computing Engine : The expression relationship of the business topology that is mainly controlled is expressed through a directed acyclic graph.

Operator execution engine : It mainly controls the execution of real business functions of the business. Different languages have different function implementations. For example, Python, GoLang, and C/C++ all have their own execution engines.

The following is a detailed description of the two parts

Graph computing engine implementation : the core function of the implementation is to implement a framework that supports the expression of directed acyclic graphs. It is implemented using C++. In the part of the figure below, the blue part is the framework implementation of directed acyclic graphs. Of course, our framework The underlying implementation supports the expression of various data:
Local queue expression : The lower left corner of the figure is a simple topology expression. The topology relationship is connected in series through the local lock-free queue. A business operator is mounted under each queue. After the business operator finishes processing the data, Operator topology configuration management is distributed to the downstream local queue. If it is the last operator, all data results are processed and output to the remote queue Producer. Here on the left are common functions inside the framework:

Thread management : Each business operator defaults to a multi-threaded distribution mode (if the business operator only supports single thread, you can set the number of threads to 1).

Distribution mode : The data consumption distribution mode defaults to round-robin distribution, fixed KEY distribution (to achieve order preservation) and distribution according to consumption capacity.

Data compression : This supports common data compression methods by default, is an optional throughput optimization method, and supports common compression algorithms such as LZ4, Gzip, and Snappy.

Remote queue expression : Each big blue box represents a specific instance, and remote queues are used to interact between instances. If there is only one business operator in each instance, this interaction method is similar to the expression method of remote queues.
Mixed expression : More usage is mixed. A giant topology will be split into multiple sub-topologies. Each sub-topology is expressed using a local queue, and the sub-topologies are expressed using a remote queue.
Multilingual engine execution layer : The overall framework of the language execution layer has different implementation modes according to different languages, and is roughly divided into three parts: compiled type, interpreted type, and native C++.

Interpretation : The most typical implementation is Python, which was also one of the favorite development languages used by students in the past. On the top of the picture above, the gray part on the left is the C++ development part, the yellow part on the far right is the business code, and the middle part is the C++-to-Python interaction engine call. The essence of this method is that the C++ development engine uses PyBind to implement the specified interface service interpreter, operates according to the specified data serialization and deserialization methods, and then converts it into a Python Dict in real time when the function is called.

Compiled type : typical usage of Golang. Similar to the implementation of Python, the gray part on the left is the part developed by C++, the yellow part on the far right is the Golang business code, and the middle part is the interaction engine from C++ to Golang. The implementation of Golang is a bit more complicated than that of Python. The essence is to use native CGO as the user interface. In order to unify the user interface layer, it is actually divided into two parts: the left half is directly interacting with C++, and the direct implementation of C++ is responsible for converting native C++ into The function pointer of the underlying C type makes the call. The right half uses Golang to implement and is responsible for deserializing natively serialized functions into business structures, and then making actual calls.

Native C++ : Many people here may find it strange that C++ is also compiled. There is already a compiled implementation mode. Due to consideration of versatility, serialization and deserialization operations are bound to be performed during the data transfer process. In the implementation process of native C++, we completely discard all serialization and deserialization operations during the implementation process, and the data is queued locally. The transfer in is completely the business data pointer (not serialized data), and each business operator can directly use the reflection mechanism based on the original data pointer in the process of processing data (there are many simple map implementation methods). Get the data item of the corresponding column, the whole process is lock-free and super efficient. The core optimization idea here is the data chain processing process. Each operator on the zipper shares the same input and output. Each operator is actually a function with the same interface, so that the combination of function pointers can be adjusted at will to form different processing chains. For example: a request is sorted by model A, and a request is sorted by model B. They can share the operator of the previous sequence, and only differ in the last operator. On the basis of chain processing, operators can also be derived classes that inherit the same base class interface, or lambda expressions. Combined with some programming techniques such as the factory model, the adjustment of the processing chain can be configured, dynamic, and scripted. It is measured that a single computer (single thread/operator) can reach hundreds of thousands of processing power for executing business single operator, ordinary pure data items with branches, and multi-node topology computing.

Through the implementation of the new computing engine, it is fully compatible with the usage of the previous generation of computing systems, simplifying the functions that are not frequently used, and optimizing the complex topology execution mode. Through the decoupling of the architecture layer and the business layer, it supports multi-language and efficient execution methods. , the average calculation efficiency of the data frame for the relocation of the old business is increased by 5 to 10 times , and the business development efficiency is further improved due to the targeted construction of the business framework.

Realization of intelligent control

The automatic problem analysis engine is the brain of the entire intelligent control system. It receives the raw data provided by the observation upstream, conducts automatic analysis and decision-making, and processes it through the self-healing capability provided by the system. The core idea of the automated problem analysis engine: As long as RD students can find the problems and solutions that have occurred in history, they can be converted into system rules and post-function sorting. Then the next time you encounter a problem, you don't need human intervention. The core analysis process of the rule engine is 2-stage:

Phase 1 : Configuration of the traditional configurable rule engine (yellow part in the upper right corner of the figure above), configure the logical relationship (AND or NOT) of multiple collection index items, here is mainly for the basic analysis function of the problem, to determine whether the rule trigger.

Phase 2 : Based on the results of this basic analysis, the post-function execution analysis is performed. This is mainly an analysis supplement for complex problems. Finally, the execution engine executes the function according to the returned result.

The execution results of the problem analysis engine are as follows:

Prerequisite : The developer needs to configure the processing logic rules (and the data items that the rules depend on, required) & callback function (optional).
Data parser : The data parser is mainly responsible for the original extraction of data, which is divided into the following three steps:

a. Configuration analysis: logic execution is analyzed according to the data information configured by the developer;

b. Data extraction: According to the parsed configuration, it can be obtained through the data interface, and the required information can be extracted from different media according to the configured information from the unified interface;

c. Data normalization: Normalize the raw data of different media into a unified data format for use by the rule manager.

Rule manager : The rule manager is mainly responsible for the core logic analysis work, which is divided into the following steps:

a. Rule analysis: According to the rule logic configured by the developer, the original configuration information is interpreted into the original rule tree.

b. Execute calculations: perform calculations respectively according to the data results provided by the data parser and the configured function rules. The most important thing in the calculation process is the basic analyzer, which provides five basic capabilities and dozens of common logic calculations to assist rule configuration.

c. Rule logic operation: perform logical operations according to the rule tree parsed by the upper layer and the calculation results of each data item, and determine whether to perform advanced data analyzer according to the execution results. If the judgment result is true, then according to the configured The post function is processed.

Advanced data analyzer : There are two modes as shown in the figure. For simple basic analysis that can judge the result, the default processing function is directly used for data extension; for simple logic rules that cannot be accurately expressed, developers can customize the post Analysis function, the function will pass the original data and the calculation result of the basic calculation as parameters, and the developer only needs to describe the analysis logic clearly through the processed data.
Action executor : It is the real execution engine of the analyzer, which is dynamically adjusted according to the parameters contained in the result of the rule operation.

Through the construction of the intelligent control system , tens of thousands of abnormal problems are analyzed and processed at the monthly level, and the proportion of automatic recovery accounts for more than 95% of the total. prevent ordinary problems from becoming serious).

04 Conclusion and Outlook

The whole article takes the development of offline computing system as the main line, and explains the problems encountered and solutions throughout the full text. In particular, the core of the current new-generation architecture is a declarative design centered on data + functions, combined with an efficient computing engine and intelligent design, to further improve the overall offline computing system. In fact, although many problems in the past have been solved in the design, there are still imperfections. Of course, the current effect is far from reaching the final ideal state, and some system functions need to be continuously polished and upgraded.

——END——

recommended reading

MMKV application optimization practice of Dujia Editing App

Baidu engineers analyze the decoding strategy

Baidu engineers analyze strong chemistry

Talking about the Design and Development of Unified Authority Management Service

Baidu APP iOS terminal package size 50M optimization practice (5) HEIC picture and useless class optimization practice