From data to large model application, on November 25th, Hangzhou Yuanchuang Conference was held to share development tips.

The process of designing a system is the process of building a building. The quality of the architectural design directly determines the quality of the building.

When we design the system architecture, we always encounter a series of problems, such as how to start the architecture of a large system and where to start designing? Should the system be divided into multiple modules? How should the modules be divided to be more reasonable? Or maybe you feel that the requirements put forward by the product are very unreasonable and completely affect our normal architectural design! Can we muddle along and not pay attention to non-functional requirements?

These problems made us at a loss when we first started architecture design. However, as we completed system architecture design one after another, we found that architecture design has rules and regulations to follow. As long as we learn these rules and routines, and use them in the work process With continuous accumulation and precipitation, a complete architecture design methodology will be formed. When facing new large-scale system architecture design, it will be carried out step by step and rhythmically, and finally the overall architecture design will be completed.

Principles of architectural design

Architectural design needs to follow some principles:

1. Architecture design requires method system

Architectural design is not a "single method" that is used directly for architectural design, but a "method system" composed of multiple unique methods, and this system will continue to evolve with the development of new technologies.

2. Architectural design is driven by questioning

Architectural design is a question-driven process. On the basis of "demand-driven", we need to constantly question the intermediate results of our architectural design, and further introduce more "quality attributes" and more "functional scenarios" through "questioning" .

3. Multi-view under multi-stage

Architectural design, multi-stage or multi-view? Architectural design is first of all "multi-stage". We divide the architectural design into multiple stages, and only in each stage will the "view" dimension be considered.

Three stages of architectural design

Stage 1. Preparatory stage

The goal of the preparatory stage: comprehensively understand the requirements, grasp the characteristics of the requirements, and determine the driving force of the architecture design.

In the preparatory stage, we need to comprehensively sort out and understand the requirements, and not miss any detail of the requirements. At the same time, we analyze the various quality attributes and system constraints generated by the requirements, and take these constraints into account when designing the architecture so that no major architectural attributes are missed.

Phase 2. Conceptual architecture

Conceptual architecture must consider all aspects of requirements including functionality , quality , and constraints .

Phase 3: Refining the architecture

In the architecture refinement stage, we designed five views from five different perspectives to complete a comprehensive design of the entire system.

A through-line of architectural design

Consideration of non-functional requirements : Non-functional requirements cannot be solved overnight, because during the design process, new requirements will continue to be discovered. Even if the design is completed, constraints that affect non-functional requirements will appear during the development stage, so in the Throughout this phase, attention should be paid to non-functional requirements.

Preparatory architecture stage analysis

The most important goal of preparing the architecture is to establish an overall view of demand, grasp the characteristics of demand, and determine the driving force for architecture design. Through detailed analysis of requirements, we have a macro perception of requirements, while also taking into account the constraints imposed on system design by system quality requirements and constraints.

Requirements structuring

Demand is structured, not scattered demand points. Only by structuring the analyzed demand can the entire demand be perceived macroscopically. You can use the ADMEMS two-dimensional matrix to sort out the factors affecting the architecture.

For example, the following matrix analysis divides the requirements into multiple dimensions, and analyzes them horizontally from the three aspects of "general function", "quality", and "constraints". The broad function refers to the basic functions that need to be met, and the characteristics of products or business personnel. Ask directly. The quality dimension refers to technical design and maintenance such as high concurrency, high availability, and scalability that need to be considered when designing the system to ensure that the system can meet basic needs while also responding to subsequent system evolution and extreme scenarios (for example: surge in user volume, flash sales) The satisfaction of waiting. Constraints are some restrictions during system design, such as launch date, launch environment, developer skill level, etc. Vertically divided into three maintenance levels: "business-level requirements", "user-level requirements" and "development-level requirements". "Business-level requirements" refer to the basic requirements put forward by products or business personnel, and "user-level requirements" are derived from the system From the perspective of users, potential needs such as user computer operation level and user usage habits are discovered, while "development-level needs" are discovered from the perspective of R&D personnel, such as scalability, testability, technical environment and other different dimensions. needs.

By structuring the requirements, we can comprehensively analyze the overall requirements and understand the requirements as a whole. At the same time, we can also discover the system constraints from different angles and start designing at the very beginning of the system design to prevent the omission of major constraints. Architectural design failure.

Analyze the impact of constraints

Several aspects of constraint analysis:

1. Binding requirements from products or operations personnel

Non-functional requirements of the system, such as: online time, budget, construction period requirements, etc.

Restrictions related to business areas, such as business rules or business restrictions, relevant laws, patents, etc.

2. Binding requirements from users

Users of the system will also have restrictive requirements, such as the user's computer level, age group, usage preference, country, etc.

For example, if the user's computer skills are generally weak, the interaction method should not be too complicated when developing, and at the same time, the robustness of the system should be taken into consideration to prevent the system from being hung up by the user.

The external environment when users use the product may also produce constraints. For example, whether the access environment is an intranet or an external network, this determines the different network permissions provided by the system for accessing links. If the signal strength of the access environment is low, the performance requirements of the system will be higher.

3. Binding requirements from development or operation and maintenance personnel

The technical level and degree of integration of the development team also restrict the development of the system. If the developers are all senior R&D personnel and have an in-depth understanding of the current technology stack, the development progress will be faster. If it is a new team and needs to If you need to learn the technology stack before you can intervene in development, additional considerations need to be made in terms of construction period or system risk.

4. Current technical environment of the industry

The maturity of middleware in the current technical environment, programming language and popularity, advantages and disadvantages, etc., will all impose constraints on architectural design.

Classification of constraints:

1. Direct restraint

For example: the system runs on the linux platform.

2. Convert constraints into functional requirements

For such constraints, it can be directly converted into functional requirements

For example: The supplier has its own set of city information tables -> Functional requirements elicited: City conversion is required

For example: the supplier's server performance is poor, the maximum tps is 10 -> the resulting functional requirements: a current limiting request is required

3. Convert constraints into quality attribute requirements

For example: the system user’s computer skills are not high

Converted into quality attributes: ease of use (otherwise it wouldn’t be used), robustness (the system is paralyzed)

Determine critical qualities

The key qualities of the system require trade-offs. It is necessary to confirm which aspects the business personnel should pay more attention to or determine which ones are necessary and which ones can be appropriately ignored based on meeting the needs.

We need to first determine which quality attributes the architecture focuses on supporting, and then make trade-offs between conflicting quality attributes. For example, when the quality attribute of performance is met, maintainability and testability will be reduced due to the introduction of new solutions or components; when scalability is improved, it will have an impact on the performance and security of the system, etc. What we need to do is make trade-offs among the various key qualities.

Identify key features

Identify 4 aspects of key functionality

1. Core functions

2. Must-do functions

3. High-risk functions

4. Unique functions

Features that do not exist in other common systems

Note the derived requirements:

When transferring from requirements to design, due to the complexity of the solution formulation process, a large number of derived requirements will be generated, and the derived requirements are several times the original requirements.

Example:

Original requirement: regularly pull supplier data.

Derived requirements:

1. Due to the large number of suppliers, it is necessary to introduce distributed scheduled tasks and concurrent pulling of the cluster

2. Due to the large amount of supplier data, separate database and table design is required.

3. Need to search quickly, introduce storage engine components, etc.

We must consider these derived requirements. Although the business requirements are not reflected, the key influencing factors of architectural design are missing.

Comparison of architectural drivers:

Business requirements drive architecture:

Significant demand-driven architecture:

It can be seen from this that through an architecture driven by major requirements, more critical parts can be taken into consideration, the designed architecture can better meet the requirements, and the probability of successful architecture design will be higher.

Conceptual architecture stage analysis

In the conceptual architecture stage, the system is properly decomposed without getting bogged down in details.

The process of conceptual architecture is to first conduct a preliminary design based on key functions, and then perform high-level segmentation of the designed system. Next, consider non-functional requirements (key qualities and constraints), and then modify your preliminary design, and the cycle continues. In the process of questioning and optimization, the architectural design is improved.

initial design

The goal of preliminary design is to discover responsibilities without going into detailed design. Based on key functions, preliminary design is carried out, and flow chart design is carried out based on main process, key process, golden process, etc. to discover responsibilities.

high-level segmentation

Divide complex systems into multiple secondary systems. Or directly divided into specific subsystems.

Two ways of high-level segmentation:

1. System segmentation

Considerations for segmentation include system functions, deployment environment, language, system scale, etc.

For example, a large system is divided into order, product, supply chain and other systems.

2. Segmentation within the system

Carry out internal segmentation of the system based on its responsibilities, calling relationships, versatility, etc.

The most common one is layering. For example, a system is divided into gateway layer, service layer, search module, man terminal, etc.

layered angle

1. Logical layering

Logical layering attaches great importance to the division of responsibilities. Responsibilities are often directly related to the relationship between the upper layer and the lower layer. The upper layer and the lower layer can be distributed on different machines or on the same machine.

2. Physical layering

Software units distributed on different machines.

3. Universal layering

Those with different versatility are divided into different layers. Generally, the greater the versatility, the lower the level is.

Consider non-functional requirements

The specific method is: use the goal-scenario-decision table , see the figure below:

Architectural design is driven by doubts. For example, if you question the availability of the system and consider that the system may be down, you will introduce a cluster deployment design. If you consider that the downstream interface may time out or have exceptions, you will introduce a design for interface degradation, etc.

5 elements to consider for your scene

1. The source of influence, whether it comes from inside the system or outside the system

2. How it affects

3. Affected objects

4. What are the problems or values?

5. What is the environment?

Scenario trade-offs:

Value, cost, development difficulty, probability of occurrence. For some scenarios, after comprehensive weighing and thinking, it may not be supported. Not all scenarios must be supported, otherwise there may be over-design.

Analysis of the detailed architecture stage

logical view

The logical view is the division of responsibilities of different parts of the system . According to different responsibilities, the system can be divided into fine-grained splits into multiple subsystems.

layered refinement

According to the needs of system design, the layers of the system can be refined. For example, the presentation layer -> business layer -> data layer can be refined into: presentation layer -> control layer -> interface layer -> interface implementation layer -> data layer.

The introduction of partitions

The concept of partitioning is related to business processes. The basis of partitioning is: responsibility . For example, the settlement process can be used as a partition, and the order process can be used as a partition. Dividing the system into multiple partitions can not only support parallel development, but also divide the system into multiple sub-domains, which is conducive to the convergence of business concepts and business processes.

Mechanism extraction

Mechanism refers to the public parts of the system that can be abstracted, such as public tools, public components, public processes, etc. Extracting these public parts is crucial to architectural design.

Principles for dividing subsystems:

1. Units with different responsibilities are divided into different subsystems

2. Units with different versatility are divided into different subsystems

3. Units that require different development skills are divided into different subsystems to take into account the workload, and further divide the system that is too large.

development view

The task of developing the architectural view is to map "logical responsibilities" to "program units", such as: "source programs" to be written independently, reusable libraries, frameworks, etc.; at the same time, selection of development technology, such as: development language, Development tools, etc., and then also need to establish the relationship between program units, project division, directory structure, compilation dependencies, etc.

Run view

The work content of running architecture design is to determine which control flows to introduce: processes, threads, etc.; determine the tasks of each control flow, and also deal with related issues, such as the creation, destruction, communication mechanism, etc. of control flows, between control flows The synchronization relationship, whether there is resource contention, whether locking is required, etc. also need to be considered.

physical view

3 tasks for physical architecture design

1. Hardware selection and physical topology

2. Software to hardware mapping relationship

3. Optimization of the plan

Key points of thinking: "Overhead" and "contention" are the core. Contention should be avoided and overhead should be reduced.

Data view

Data view is the data storage design of the system. Based on the analysis of the system, one or more data strategies are determined. The six common data distribution strategies are as follows:

1. Independent Schema

Different system applications use different data schemas, and the data is completely independent. Generally, different systems with clear boundaries can use this method.

2. Concentrate

Different system applications use the same database. Generally, applications with associated attributes can use this method. For example, if a system is divided into a server side and a management side, but both belong to the same system, the same database can be used.

3. Partition

horizontal partition

Horizontal partitioning is our common table partitioning scheme. When a schema cannot meet our data volume requirements, it can be divided into multiple partitions, each partition storing a part of the data.

vertical partition

Vertical partitioning is another dimension of the partitioning strategy. When a single database cannot carry a huge amount of data, we can also perform vertical partitioning based on the type of data.

4. Copy

Multiple databases store the same data, and ensure data synchronization between different databases according to the established update strategy. We commonly use the separation of reading and writing libraries, which is this solution. The main library provides writing capabilities, and the slave library provides reading capabilities. Among them, the slave library The data is synchronized based on the main database data.

5. Subset

According to the requirements of some special scenarios, it is necessary to save part of the original data. For example, application1 saves all orders, and application2 only needs some of the successfully issued orders for subsequent analysis operations. You can use the subset strategy for data view design.

6. Reorganization

Use multiple different applications as data sources and heterogeneous to other applications for data analysis or subsequent process use.

Summarize

Three stages of architectural design: preliminary architecture stage; conceptual architecture stage; detailed architecture stage

Four elements of architectural design: structuring requirements; analyzing the impact of constraints; determining critical qualities; determining key functions

Three steps of conceptual architecture: preliminary design based on key functions; system high-level segmentation; analysis of non-functional requirements

Five views of the refined architecture: logical view; development view; operational view; physical view; data view

A through link: consideration of non-functional requirements

References

1. "First-line Architecture Design Guide"

Author: JD Retail Feng Xiaotao

Source: JD Cloud Developer Community Please indicate the source when reprinting

How to do a good job in architectural design? Is there any rules to follow in architectural design? | JD Cloud Technical Team