General Architecture Design Rules for Complex Business Systems

1. What is a complex system

We often refer to complex systems, so what exactly is a complex system. Let's look at Wikipedia's definition: a complex system (English: complex system), also known as a compound system, refers to a system composed of many components that may interact with each other. Two points are emphasized:

• made up of dots
• There are various associations between points

The size and complexity of the two points directly determine the complexity of the system. For example: Take our e-commerce system as an example. It is divided into many parts , such as commodities , inventory, procurement, orders, logistics, and finance . B-side business entry, management and other systems. There are innumerable connections between various parts and systems, which can be described as a complex system. Of course, it is far more than that. As the complexity of business continues to increase, the complexity of the entire system will also become more and more complex. 

Second, what is the structure

We often talk about "architecture" in life, so what exactly is "architecture":

The definition in Robert C.Martin's "Clean Architecture": Software architecture refers to the shape given to the software by people who design software. This shape refers to how the system is divided into components (Components), how each component is arranged (Arrangement), and components How to communicate between (Communication, communication)

Wikipedia's definition: an abstract description of the overall structure and components of software, used to guide the design of all aspects of large software systems IEEE's definition: architecture = structure of constituent units + relationship of constituent units + principles and guidelines

Generally speaking, it will include several contents:

• Whole: Emphasize the composition of parts, emphasizing the joint force
• Rules: Emphasize that there is an association between parts, there are rules and constraints
• Communication: Emphasize that there is communication and interaction between parts

In this way, our human society itself is a social structure, with various responsibilities, division of labor, and circles. As far as our software system is concerned, DDD is a structure, MVC is also a structure, and big data design also has a big data structure. Therefore, architecture is everywhere, and a good architecture can regulate and guide specific issues and fields.

3. The nature of architecture 

We know that the word architecture originated from the construction industry. The original English word is: Architecture. The explanation on Wikipedia is the process and product of planning, designing and building buildings. Then let's use the construction industry to understand. Building a house is very familiar to everyone. Then we build a small flat, a two-story high-rise, a 5-story small high-rise, a 10-story, and a few hundred-story skyscraper. The process, factors, Risk is quite different. Building skyscrapers requires higher costs, more uncertainties in the process, and greater challenges and risks, such as how to choose a site, what kind of structure to choose, how to bear load, how to control lighting, how to optimize, how to heat, how to Water supply, drainage, how to ventilate, how to avoid shocks and so on. The more we consider these things, the better the future quality and controllability of the house will be.

Therefore, the architecture is essentially a guiding constraint to agree on the relationship between the whole and parts, and between parts and parts, so as to make the whole more stable and reliable.

4. Architecture classification 

The example we gave above can be called architectural architecture. In fact, there are many types of architecture:

For example: business architecture, application architecture, technical architecture, data architecture, etc. The classification of a single architecture will have different views from different dimensions, and the complexity will also vary considerably.

For example: the enterprise-level architecture can highlight the company's overall strategy, business involvement, distribution, and efforts. And a single business line also has its own business structure, highlighting its own business goals, strategies, etc.

The same is true for application architecture and technical architecture, and there will be architectures with different levels of vision. Let's briefly explain our common architectural separation from the internal perspective of business lines. 

1. Business structure

When it comes to business structure, it is designed at the top level. The definition and division of business will even affect the establishment and relationship of the entire company's overall organizational structure. The business architecture is biased towards the division of business domains, model design, language conversion for the overall business, and internalization into the common language of the domain.

2. Application Architecture 

Reflect the structural relationship within the application. How to design an application, including how to divide modules, how to implement functions, how to support technology, how to display data, how to define processes, how to implement logic, how to store data, etc., are all within the scope of application architecture. The MVC, layered architecture, CQRS, DDD traditional onion ring architecture, and DDD hexagonal architecture that we often talk about can all be attributed to the category of application architecture.

3. Technical Architecture 

Technical architecture is not necessarily limited to a single application, especially in the current era of microservice architecture, how services interact, how services are governed, how data is stored, how caches are built, etc., are all within the scope of technical architecture. The technical architecture provides a technical foundation for the application and business architecture to enable better business development, more robust iteration, and development. 

5. What factors need to be considered in the structure 

1. Functional requirements

No matter what the architecture is, our first consideration must be to meet our actual business requirements. A structure without requirements is equivalent to a castle in the air, which is unrealistic and impractical. This is not really a schema. Generally speaking, functional requirements will directly determine the business architecture and have little impact on the application and technical architecture. Our architecture must be able to correctly and completely support the functional requirements.

2. Non-functional requirements 

The first priority is to meet the functional requirements of the architecture. At the same time, we need to consider stable and reliable support functions, that is, we also need to meet some non-functional requirements, such as performance, reliability, scalability, compatibility, and so on. 

3. Reliability 

In order to better serve the functions, we need to ensure that the architecture can run stably and efficiently. There will be no occasional service crashes or unavailability.

4. Availability

Similarly, the service must always be available externally. Even if there is a problem with a single service instance, we can still provide external services normally.

5. Expansibility

Functional requirements are not constant, especially in today's agile era, requirements are not raised at once. We need to have a comprehensive positioning and control of the overall capabilities of the system and services. This requires our architecture to be easily extended to support new requirements.

6. Governance ability 

A good architecture must be convenient for operation, management and monitoring. Even from the micro level to project management, the code must be easy to maintain, expand, and collaborate. 

7. Response performance 

In general, functional requirements have certain expectations for performance. This business requires us to do a lot of work on the architecture, such as: read-write separation, caching, asynchronous intervention, etc., to meet the responsiveness of the overall architecture.

6. How to analyze complex systems 

Some students have misunderstandings. When they think of a system like this, they feel that there will be great complexity, and they will consider quitting. But what you think is difficult is not necessarily difficult. We all know an idiom: "Those who are difficult can't, those who can meet are not difficult" , often due to different experiences, people's ideas and thinking about the same problem will be different. This is why we emphasize the importance of experts when designing the system. Especially the DDD domain-driven design method, which is gradually mentioned and widely used, advocates the importance of domain experts. Only in this way can we identify the complexity and fundamental pain points of real problems, and then be able to objectively and reasonably derive reliable and appropriate solutions. Obviously, there are two very important links in complex system design: requirements analysis and architecture design.

During the requirements analysis process, we need to confirm what problems the requirements are to solve and what roles they are facing. Now the popular analysis method is the DDD domain-driven analysis method. There are probably several steps in analyzing business requirements using DDD patterns:

• confirm role
• Confirm role function
• Confirm problem subdomain
• Acknowledgment Models, Events, Attribution
• Confirm Bounded Context

7. Design principles for complex systems

• Identify core issues. For the acceptance of requirements, some people will directly enter the development and design stage, especially for small partners who enter and exit the workplace. In fact, when encountering a demand, we need to think more about why we need to do this demand. Understanding this is very helpful for us to carry out business and other related architecture design, and then steer the entire demand, so that we will not easily go astray.

• To simplify complex problems, it is necessary to disassemble complex problems into small modules and break them one by one. The responsibilities of each module will be relatively single, and the future scalability and maintainability will be relatively independent and simple.

• Make sure to use a common language for communication, especially in domain-oriented design, everyone must have the same understanding of the domain model.

• Clarify the positioning, relationship, interaction, etc. of the system and model.

• Possess future planning capabilities, including systems, technologies, solutions, capacity, etc., so that the system can provide value services better and more stably in the long run.

• Follow various design patterns, best practices, avoid starting from 0, including: SOLID design principles, CAP theory, BASE theory.

Eight, the architectural characteristics of complex systems

1. Emphasis on functional disassembly, modular design, and atomic design

Complex systems must be carefully divided into functions, modules, and domains. Each module should have a clear, single responsibility. In this way, when we analyze the problem, we can focus on the problem in a certain range without causing too much impact, which is convenient for the maintenance and expansion of the overall system.

2. The ability to expand vertically and horizontally is crucial

When we do small functions, we may not think too much about it. But when it comes to complex systems, we must consider a lot, including future function bearing, traffic bearing, data scale, response requirements, etc., all of which require us to reserve sufficient vertical or horizontal expansion capabilities. These cannot be achieved overnight, but the necessary expansion needs to be left according to the plan to make the system have long-term value.

3. Architecture first

For complex systems, it is no longer something that one or several flow charts can solve. We need to clarify the domain division and domain boundaries through the domain architecture, the functional modules and functional boundaries through the system architecture, and the responsibilities, boundaries, structural divisions, and dependencies of each application through the application architecture. Through the technical architecture, we clarify the technology stack we use and the application boundaries in the overall system. Through the data architecture, we can clarify our data storage method, structure, data usage method, etc.

These architectures must be clear, clear, and focus on the long-term value of the system.

4. Divide and conquer

For complex systems, splitting is inevitable, and large problems are resolved into small problems. According to the division of domains, modules, and functions, we attribute problems to different boundaries and break them one by one. If small problems can be solved properly, then through reasonable dependence and combination, large problems can be effectively solved and the purpose of the entire system construction can be achieved.

9. Typical Complex Problem Solving Architecture 

With the continuous progress of society and the development of information components, we need more information methods to solve systematic problems. Earlier we used more data-driven models, that is, we would first think about what kind of structure will be used to store related data, what kind of dependencies between models, how to organize data, how to put Data and peripheral interaction, these ideas are also typical MVC architecture.

The MVC architecture forces us to develop for the view. We know that the change of the view is the most uncontrollable. The more user-friendly things are, the more susceptible to user subjective influence. We know that there are complex dependencies that must exist in complex systems. Dependencies cannot exist in the view part, and they will definitely appear as interface dependencies. For complex systems, we have to force us to change our thinking and force us to design for interfaces. Combined with the complexity of the business system, if you want the system to have long-term value in the future, you have to split the large system, describe it in a unified business language, and split the unidentifiable problems into identifiable problem domains. Solve, this is the method of domain-driven design that is becoming more and more popular now.

1. Domain Driven Design 

Domain-driven design forces us to no longer be driven by data, but to be driven by domain. Encountered problems, we divide and disassemble in the advanced field. Which problem domain does this problem belong to, or which problem domains need to be disassembled, and then the final problem is solved through the combination and dependence of domains.

Domain-driven, as early as 2004 Eric Evans in the book "Domain-Driven Design: Tackling Complexity in the Heart of Software" (Domain-Driven Design: The way to deal with the complexity of the core of software) on the two aspects of strategy and tactics Detailed elaboration.

At present, for the design of complex systems, the domain-driven model is conducive to the sustainable development of the system.

2. Microservice architecture 

In fact, the microservice architecture is a variant of the earlier SOA (service-oriented architecture). In fact, this word was widely circulated in the industry since an article "Are Microservices the Future" published by Martin Fowler in 2014. The microservice architecture emphasizes decentralized management and maintains the autonomy and independence of services as much as possible. Emphasis is placed on the integration and acquisition of capabilities through different small services. In this way, we can selectively expand the service vertically and horizontally, and at the same time avoid the bloat of a single system and the stacking and coupling of functions. 

3. Cloud Native Architecture 

When it comes to the original, everyone is familiar with it. For example, when we say that IOS and Android have native interfaces, it means that the interfaces are already supported by them. When it comes to cloud native, for services, we emphasize more that services are inherently capable of deploying and providing services on the cloud. This ability makes the service have the inherent ability of decentralization and the ability of horizontal expansion. This is also the ability that microservices emphasizes. 

4. DevOps Architecture 

Before DevOps, we have been talking about agile, and the industry also has tactical solutions. Such as Extreme Programming, Scrum, etc. If agile is more about solving the coordination and efficiency among requirements, products, R&D, and testing, then DevOps is more about solving the coordination between R&D and operation and maintenance. DevOps has developed in full swing in recent years, which is closely related to the development of domain-driven, microservice architecture, cloud architecture technology, and virtualization technology (especially the development of Docker). To be precise, it is a joint force of subtle combinations of various technologies. With the development of DevOps, O&M no longer cares about application deployment and other issues. These things can be handled by R&D. O&M is more about providing R&D with automated construction, integration, deployment, monitoring, etc. basic ability.

5. Big data architecture

Today is a digital era, and all walks of life are busy with digital transformation. For complex business systems, the value of data is particularly prominent, so naturally there must be demands for the processing of massive data and the mining of value. Then the massive data storage, extraction, transmission, cleaning, calculation, mining and other capabilities need to be designed through the pattern of big data architecture. 

10. Summary

Nowadays, the key to system design has become distributed, cloud-based, micro-service-based, and big-data-based. The essence of the architecture remains unchanged, but due to the development of society, our needs, the problems that need to be dealt with, and the dependencies are becoming more and more complex. We need to use a developmental perspective and always follow the forefront of technology to promote, optimize, and iterate the architecture design of the system. .

The architectural design of a complex system cannot be achieved overnight, and the right one is the right one.