【Technical Architecture】Evolution of Technical Architecture

foreword

1. Explanation of terms (common concepts)

1.1 Application / System

A program or a group of programs that cooperate with each other to complete a set of services.

Life example analogy: In order to complete a task, a team composed of one person or a group of people who cooperate with each other is built.

1.2 Module / Component

When the application is more complex, in order to separate the responsibilities, the part with clear responsibilities and strong cohesion is abstracted from the concept, which is easy to understand.

Life example analogy: In order to conquer a certain stronghold in the army, personnel are divided into assault teams, blasting teams, cover teams, communication teams, etc.

1.3 Distributed

Multiple modules in the system are deployed on different servers, that is, the system can be called a distributed system. For example, the Web server and the database work on different servers, or multiple Web servers are deployed on different servers.

Life example analogy: In order to better meet the needs of reality, a work team in the same office site is dispersed to different work sites in multiple cities for remote cooperation to complete the goal. Communication between modules across hosts is basically done with network support.

1.4 Cluster (Cluster)

A specific group/group of components deployed on multiple servers to achieve a specific goal, the whole is called a cluster. For example, multiple MySQLs work on different servers and jointly provide database service targets, which can be called a group of database clusters.

Life example analogy: In order to solve the army's operational goal of conquering a large city with strong defenses, the headquarters will gather a large number of artillery units to form an artillery strike cluster. Distributed vs clustered. Usually, there is no need to strictly distinguish the subtle concepts of the two. If you look closely, distributed emphasizes the physical form, that is, working on different servers and completing tasks through network communication; while clustering cares more about the logical form, that is, whether to complete a specific service goal. .

1.5 Master (Master) / Slave (Slave)

In a cluster, there is usually a program that needs to take on more responsibilities and is called the master; other programs that take on subsidiary responsibilities are called slaves.

For example, in a MySQL cluster, only one of the databases on the server allows data writing (addition/deletion/modification), and all data modifications of other databases must be synchronized from this database, so that database is called the main database. , other databases are called slaves.

1.6 Middleware

A class of software that provides different applications to communicate with each other, that is, a bridge between different technologies, tools, and databases.

Life example analogy: At the beginning of a restaurant, it will go to the market to buy vegetables every day, but as the business volume of the restaurant increases, a purchasing department is established, which is dedicated to purchasing business, which is called the bridge between the kitchen and the vegetable market.

1.7 Containers (Docker)

Docker is an open source application container engine that allows developers to package their applications and dependencies into a portable image, and then publish it to any popular Linux or Windows operating system machine, and can also implement virtualization.

It can be understood as a container, which contains the goods of each user and is packaged as a whole.

1.8 Container Orchestration (K8S):

kubernetes, referred to as K8s, is an abbreviation formed by replacing the 8 characters "ubernete" in the middle of the name with 8. Is an open source, used to manage containerized applications on multiple hosts in the cloud platform, the goal of Kubernetes is to make the deployment of containerized applications simple and efficient.

It can be understood as a cargo ship, the size of the container is installed, and the cargo condition is reasonable to organize the container to complete the overall cargo handling.

2. Evaluation indicators

2.1 Availability

Investigate the probability/expectation that the system can provide services normally within a unit time period. For example: Annualized system availability = system normal service time / total time in a year.

There is an indicator implied here, that is, how to evaluate whether the system cannot provide normal, we will not go into it. The 4 9s we usually say mean that the system can provide 99.99% availability, 5 9s means 99.999% availability, and so on. We usually only briefly express the pursuit of our system with the non-quantitative goal of High Availability HA.

2.2 Response Time (Response Time RT)

Refers to the time it takes for the user to complete the input to the system to give the user response. For example, the response time of ordering food delivery = the time when the food delivery is received - the time when the order is completed. Usually what we need to measure is the longest response time, average response time and median response time. In principle, the smaller the better, but in many cases, due to implementation limitations, it is necessary to judge throughput (Throughput) vs concurrent (Concurrent) according to the actual situation: Throughput examines the number of requests that the system can successfully process within a unit time period . Concurrency refers to the highest number of requests supported by the system at the same time.

For example, on a 2-lane expressway, 20 vehicles can pass in one minute, then the concurrency is 2, and the throughput in one minute is 20. In practice, the concurrency is often not directly obtained, and in many cases, the throughput of a very short period of time (such as 1 second) is used instead. We usually use the non-quantitative goal of high concurrency (High Concurrnet) to briefly express the pursuit of the system.

3. Stand-alone architecture

In the early stage, we need to use our capable technical team to quickly put the business system into the market for testing, and can quickly respond to changing requirements. But fortunately, there were very few user visits in the early stage, and we did not put forward high requirements on our performance and security. Moreover, the system architecture is simple and does not require a professional operation and maintenance team, so it is appropriate to choose a stand-alone architecture.

• related software

  • Web server software: Tomcat, Netty, Nginx, Apache, etc.
  • Database software: MySQL, Oracle, PostgreSQL, SQL Server, etc.

• advantage:

  • Simplicity: Standalone architecture is the simplest application architecture because it does not involve any network communication and distribution issues.
  • Reliability: When a stand-alone architecture application fails, it will only affect an independent instance and will not affect other instances.
  • Scalability: Single-machine architecture applications are usually very scalable, because performance and capacity can be improved by simply adding more hardware resources.
  • Security: Standalone architecture applications are generally more secure than distributed architectures because they do not suffer from distributed denial of service (DDoS) attacks.

• shortcoming:

  • Flexibility: Stand-alone architecture applications are generally more difficult to scale and modify than distributed architectures because they require code changes and updates on all instances.
  • Load balancing: Standalone architecture applications are often difficult to achieve load balancing because they do not have the distributed load balancing mechanism in distributed architectures.
  • Fault recovery: When a stand-alone architecture application fails, only one instance is affected, but they cannot achieve fault tolerance and fault recovery in a distributed architecture.
  • Performance: The performance of stand-alone architecture applications is usually limited by hardware resources, since they can only utilize the resources of one server or computer.

4. Application data separation architecture

As the system went live, we found success without surprise. A group of loyal users appeared in the market, which gradually increased the number of visits to the system and gradually approached the limit of hardware resources. At the same time, the team also accumulated a lot of experience in business processes during this period. Facing the current performance pressure, we need to take precautions to carry out system reconstruction and architecture challenges to improve the carrying capacity of the system. However, due to the tight budget, we chose to separate the application and data, which can improve the carrying capacity of the system at a minimum cost.

The main difference from the previous architecture is that database services are independently deployed on other servers in the same data center, and application services access data through the network.

• advantage:

  • Flexibility: The application-data separation architecture can easily expand the capacity and performance of applications and data storage, and improve system availability.
  • Reliability: The application data separation architecture can ensure the reliability of the system through load balancing and fault tolerance mechanisms.
  • Scalability: The application data separation architecture can increase the processing power and storage capacity of the system by adding server nodes, and improve the scalability of the system.
  • Security: Application data separation architecture can provide better security, because applications and data storage are deployed separately, reducing the risk of data leakage.

• shortcoming:

  • Complexity: The application data separation architecture introduces more components and network topology, which increases the complexity of the system and requires more management and maintenance work.
  • Cost: The application of data separation architecture requires more hardware resources, including servers, storage devices, network devices, etc., which increases the cost of the system.
  • Synchronization issues: The application of data separation architecture needs to consider the issue of data synchronization to ensure data consistency and integrity.
  • Architecture selection: The application data separation architecture needs to be selected according to specific application scenarios and requirements, and the dependencies and interaction methods between applications and data storage need to be considered.

5. Application service cluster architecture

Our system has been welcomed by users, and there have been explosions, and a single application server can no longer meet the demand. Our stand-alone application server first encountered a bottleneck. There were two solutions before our technical team. Everyone had a heated discussion on the advantages and disadvantages of the solutions:

• Vertical expansion / vertical expansion Scale Up. Handle more traffic by purchasing application servers with better performance and higher prices. The advantage of this solution is that there is no need to make any adjustments to the system software; but the disadvantage is also obvious: the relationship between hardware performance and price growth is non-linear, which means that choosing hardware with twice the performance may cost more than four times the price , Secondly, there is an obvious upper limit for hardware performance improvement.
• Horizontal expansion / horizontal expansion Scale Out. By adjusting the software architecture, adding application layer hardware, and sharing user traffic to different application layer servers, the system's carrying capacity is improved. The advantage of this solution is that the cost is relatively low, and the upper limit of improvement is also large. But the disadvantage is that it brings more complexity to the system and requires a more experienced technical team. After the team's study, research and discussion, the horizontal expansion solution was finally chosen to solve this problem, but this required the introduction of a new component - load balancing: in order to solve the problem of which application server the user traffic is distributed to, a Specialized system components do traffic distribution. In practice, load balancing not only refers to working at the application layer, but may even be in other network layers. At the same time, there are many traffic scheduling algorithms. Here are a few common ones:
  • Round-Robin polling algorithm. That is, the requests are allocated to different application servers in sequence very fairly.
  • Weight-Round-Robin round robin algorithm. Give different weights to different servers (such as different performance), and those who can do more work.
  • Consistent hashing algorithm. The hash value is obtained by calculating the characteristic value of the user (such as IP address), and the distribution is made according to the hash result. The advantage is to ensure that requests from the same user are always assigned to the specified server. That is, the special account manager service we usually encounter.

• related software

  • Load balancing software: Nginx, HAProxy, LVS, F5, etc.

• advantage:

  • High availability: The application service cluster can ensure that when an application server fails, other servers can automatically take over its work to ensure the continuous availability of the system.
  • Load balancing: The application service cluster can distribute requests to multiple application servers through the load balancer to balance the load and improve the processing capacity of the system.
  • Fault tolerance: Application service clusters can detect faulty application servers and automatically remove them from the cluster to ensure system reliability and stability.
  • Performance: Application service clusters can utilize computing and storage resources of multiple servers to provide higher processing power and larger processing capacity.

• shortcoming:

  • High initial investment cost: The application service cluster architecture requires investment in purchasing more servers and equipment. Compared with a single server, the initial investment cost is higher.
  • High management and maintenance costs: Application service clusters require more management and maintenance work, including configuration, debugging, and monitoring. Compared with a single server, the management and maintenance costs are higher.
  • Troubleshooting is difficult: Faults in the application service cluster may cause the entire system to be paralyzed, so troubleshooting is difficult and requires more technical support and troubleshooting.
  • Difficulty in management and maintenance: Application service clusters require more technical and experience support, and are difficult to manage and maintain, requiring professional technicians to operate and manage them.

6. Read-write separation / master-slave separation architecture

The application service cluster architecture mentions that after we distribute user requests to different application servers through load balancing, they can be processed in parallel, and as the business grows, the number of servers can be dynamically expanded to relieve pressure.

But in the current architecture, no matter how many servers are expanded, these requests will eventually read and write data from the database. After a certain level, the pressure of data is called the bottleneck point of the system's carrying capacity.

Can we scale database servers like application servers?
The answer is no, because the database service has its particularity: if the data is distributed to each server, the consistency of the data will not be guaranteed.

The so-called data consistency here refers to: for the same system, no matter when and where, we should see a consistent data. Imagine the amount of the account managed by the bank. If after receiving a transfer, the data in one database is modified, but the other database is not modified, the deposit amount obtained by the user will be wrong. The solution we adopted was to keep a primary database as the write database and the other databases as slave databases. All the data in the slave library comes from the data of the master library. After synchronization, the slave library can maintain the same data as the master library. Then, in order to share the pressure on the database, we can hand over all the write data requests to the master library for processing, but the read requests are distributed to each slave library.

In most systems, read and write requests are disproportionate, for example, 100 reads and 1 write, so as long as the read requests are shared by each slave library, the pressure on the database will not be so great. Of course, this process is not free. Data synchronization from the master database to the slave database actually has a time cost, but we will not discuss this issue further for the time being.

• advantage:

  • Load balancing: read-write separation/master-slave separation architecture can distribute read requests and write requests to different servers through the load balancer to balance the load and improve the processing capacity of the system.
  • Read operation performance: The read-write separation/master-slave separation architecture can allocate read operations to dedicated read servers to improve the performance and throughput of read operations.
  • Write operation performance: read-write separation/master-slave separation architecture can assign write operations to dedicated master-slave servers to ensure the performance and consistency of write operations.
  • Scalability: The read-write separation/master-slave separation architecture can increase the processing power and storage capacity of the system by adding server nodes, and improve the scalability of the system.

• shortcoming:

  • Failure recovery: There is a data synchronization delay between the read server and the write master-slave server in the read-write separation/master-slave separation architecture, which may cause failure recovery problems.
  • Data consistency: There are data differences between the master and slave servers in the read-write separation/master-slave separation architecture, which may cause data inconsistency.
  • Maintenance cost: The read-write separation/master-slave separation architecture requires more servers and equipment. Compared with a single server, the initial investment cost is higher.
  • Difficulty in management and maintenance: Read-write separation/master-slave separation architecture requires more technical and experience support, management and maintenance are difficult, and professional technicians are required for operation and management.

7. Introduce cache - hot and cold separation architecture

As the number of visits continues to increase, it is found that some data in the business are read more frequently than other data. We call this part of the data hot data, and the corresponding cold data. For hot data, in order to improve the response time of its reading, you can increase the local cache, and increase the distributed cache externally to cache popular product information or html pages of popular products. Through caching, most requests can be intercepted before reading and writing to the database, greatly reducing the pressure on the database.

The technologies involved include: using memcached as a local cache, using Redis as a distributed cache, and also involves issues such as cache consistency, cache penetration/breakdown, cache avalanche, and invalidation of hot data sets.

• advantage:

  • Separation of reading and writing: By storing hot data in the cache, efficient processing of read operations can be achieved, the load on the database can be reduced, and the concurrent processing capability of the system can be improved.
  • Hot data cache: storing hot data in the cache can reduce access to the database, improve system response speed, and improve user experience.
  • Cold data storage: storing cold data on disk can save memory resources, reduce system load, and improve system stability.
  • Cost reduction: By using caching, hardware costs can be reduced because the need for high-performance hardware can be reduced, reducing the total cost of ownership of the system.

• shortcoming:

  • Data consistency: Since the data on the cache and the disk are not synchronized in real time, the issue of data consistency needs to be considered to ensure data consistency and integrity.
  • Cache penetration: When the hotspot data does not hit the cache, it needs to be queried on the disk, which will lead to a waste of cache and a decrease in system performance.
  • Cache avalanche: When there is a lot of hot data in the cache, it may cause an avalanche effect of the cache and reduce the concurrent processing capability of the system.
  • Invalid update: When the hotspot data is updated, the data in the cache needs to be updated, which may cause the cache to become invalid and the data needs to be reloaded.

• related software:

  • Caching software such as Memcached and Redis

8. Vertical sub-library

As the amount of business data increases, storing a large amount of data in the same library has become somewhat incapable, so the data can be stored separately according to the business. For example, for comment data, it can be hashed according to the product ID and routed to the corresponding table for storage; for payment records, tables can be created by hour, and each hour table can be further split into small tables, and user ID or record number can be used to route data .

As long as the amount of table data for real-time operations is small enough and requests can be evenly distributed to small tables on multiple servers, the database can improve performance through horizontal expansion. The aforementioned Mycat also supports access control when large tables are split into small tables.

This approach significantly increases the difficulty of database operation and maintenance, and has higher requirements for DBAs. When the database is designed with this structure, it can already be called a distributed database, but this is only a logical database as a whole. Different components in the database are implemented by different components, such as the management and request of sub-database and sub-table. Distribution is implemented by Mycat, SQL parsing is implemented by a stand-alone database, read-write separation may be implemented by gateways and message queues, the summary of query results may be implemented by the database interface layer, etc. This architecture is actually MPP (large-scale A class of implementations of the Parallel Processing) architecture.

• advantage:

  • Improve performance: Vertical sub-databases can separate hot data from common data, store hot data in memory, and greatly improve the processing power and throughput of the system.
  • Improve scalability: vertical sub-library can increase the processing power and storage capacity of the system by adding server nodes, and improve the scalability of the system.
  • Improve maintainability: Vertical sub-databases can store different business data separately, which is convenient for maintenance and management separately, and improves system maintainability.

• shortcoming:

  • Data consistency: Vertical sharding stores data of different businesses in different databases, which may cause data inconsistency.
  • Write performance: Vertical sharding will write write operations to the corresponding database, which may cause write performance problems.
  • Security: Vertical sharding stores sensitive data in different databases, which may cause security issues.
  • Difficulty in management and maintenance: Vertical sub-databases need to manage different databases separately, which is difficult to manage and maintain, and requires professional technical personnel to operate and manage.

• related software

  • Greenplum, TiDB, Postgresql XC, HAWQ, etc. Commercial ones such as NTU General’s GBase, Ruifan Technology’s Snowball DB, Huawei’s LibrA, etc.

9. Business Split - Microservices

With the increase of personnel and business development, we assign the business to different development teams for maintenance. Each team independently implements its own microservices, and then isolates direct access to data from each other. Technologies such as Gateway and message bus can be used , to realize the call association between each other. Some businesses such as user management can even be commissioned as public services.

• related software

  • Spring Cloud、 Dubbo

• advantage:

  • Vertical separation: Deploying different business modules or services independently can improve the maintainability and scalability of the system.
  • Rapid iteration: By splitting the business into microservices, rapid iteration and agile development can be realized, and development efficiency can be improved.
  • Independent deployment: The microservice architecture allows different business modules or services to be deployed independently, improving system availability and fault tolerance.
  • Independent upgrade: The microservice architecture allows different business modules or services to be upgraded independently without interrupting the normal operation of other services.

• shortcoming:

  • Data consistency: There may be data dependencies between different services, and data consistency needs to be ensured.
  • Database synchronization: There may be a need for data synchronization between different services, which may cause performance bottlenecks and consistency issues.
  • Distributed transactions: In a distributed environment, processing transactions may become more complicated, requiring the introduction of distributed transaction solutions, such as compensation mechanisms or two-stage transaction protocols.
  • Security issues: In a distributed environment, security issues need to be considered, such as preventing SQL injection and preventing malicious requests.
  • Technical challenges: When implementing a microservice architecture, certain technical capabilities are required, such as distributed system design, microservice framework selection, etc.

10. Introduction of containerization - container arrangement architecture

As the business grows, it is found that the resource utilization rate of the system is not high. Many resources are used to cope with short-term high concurrency, and are usually idle. Dynamic expansion and contraction are required. There is no way to directly offline servers, and development, testing, and production are performed every day In an environment where all sets of environments must be isolated, the workload of operation and maintenance becomes very large. The emergence of containerization technology has brought new ideas to the solution of these problems.

At present, the most popular containerization technology is Docker, and the most popular container management service is Kubernetes (K8S). Applications/services can be packaged as Docker images, and the images can be dynamically distributed and deployed through K8S.

A Docker image can be understood as a minimal operating system that can run your application/service, which contains the running code of the application/service, and the operating environment is set according to actual needs. After the entire "operating system" is packaged into a mirror, it can be distributed to machines that need to deploy related services, and the service can be started by directly starting the Docker mirror, making the deployment and operation and maintenance of the service easier.

Services usually have production and R&D k8s clusters, which are generally not shared, and R&D clusters use namespaces to complete application isolation. Some companies are divided into R&D and test clusters according to R&D purposes, and some companies complete inter-departmental resources through organizational structures. reuse.

• related software

  • Docker、 K8S

• advantage:

  • High cohesion: Business splitting can encapsulate different functions or businesses into small, autonomous microservices, improving the cohesion and coupling of the system.
  • Low coupling: Business splitting can decouple different business logic and functional modules, so that each microservice can be developed and deployed independently, improving the flexibility and scalability of the system.
  • Independent deployment: Microservices can be deployed independently to improve system availability and stability. When a microservice has a problem, only the service can be maintained and repaired without affecting the normal operation of other services.
  • Independent expansion: Microservices can be expanded independently. When processing capacity needs to be increased, nodes or resources can be added separately to improve the processing capacity and throughput of the system.

• shortcoming:

  • Complex orchestration: As the number of microservices increases, the calling relationship and dependencies between services will become more complex, requiring more complex orchestration and scheduling strategies to ensure the normal operation of the system.
  • Excessive communication: The communication between microservices needs to be transmitted by means of message queues, etc., which will increase the complexity and cost of the system, and will also bring some performance problems.
  • Distributed issues: After micro-services, the number of services will increase, and management will be more difficult. A more complete management and monitoring mechanism is needed to ensure the availability and stability of the system.
  • Complexity: Microservices split the business into multiple small services, and each service needs to be developed and maintained independently, which will increase the complexity and difficulty of development, testing, deployment, etc.

11. Summary