Internet Architecture (1) Overall Architecture Design

Overall architectural design

The basic means of architecture are division and combination. First break up the system and then reassemble it.
The process of splitting is to split the system into various subsystems/modules/components. When dismantling, the positioning problem of each component must first be solved, and then the boundaries between each other can be divided to achieve reasonable splitting.
Integration means organically integrating separate components according to the latest final requirements.
The split structure enables developers to focus on business and technology, and achieve development agility. The result of integration makes the system soft and can change according to needs, achieving business agility.

Classification of architecture

Architecture can generally be divided into business architecture, application architecture, and technical architecture.

1. Business architecture: Help developers better understand the system from a conceptual level, such as business processes, business modules, input and output, and business domains.
2. Application architecture: Helps develop landing systems from the logical level, such as data interaction relationships, application forms, and interaction methods, making the entire system logic easier to understand. For example, the well-known SOA belongs to application architecture.
3. Technical architecture: Mainly solves the problems of technology platform selection, such as operating system, middleware, equipment, multiple computer rooms, horizontal expansion, high availability, etc.

It should be noted that the system architecture first serves people. The system must be highly orderly, have reasonable application logic, and have clear business concepts. Nowadays, everyone is discussing more about technical architecture, such as high concurrency design, distributed transaction management, etc., just because this does not require business context and makes communication easier. When designing the specific architecture, we must first pay attention to the business architecture and application architecture.

Architectural evolution of large websites

Start with an e-commerce website

Taking an e-commerce website as an example, a transaction-type website must have user (user registration, user management), product (product display, product management), and transaction (order, payment) functions. The initial architecture should look like this:
The above is a trading website built on a single machine based on Java technology. What should be noted here is that the functional modules interact with each other through method calls within the JVM, while the application and database are accessed through jdbc.

Single machine load warning, data and application separation

As the website opens and the number of visits continues to increase, the load on the server will inevitably continue to increase. At this time, the database and application will be divided from one machine to two machines.
Change: The website has changed from one machine to two. This change has very little impact on us. In the case of a stand-alone machine, the application uses JDBC to connect to the database. Now that the database is separated from the application, you only need to change the database address from the local to the IP address of the database server in the configuration file.

Why is it so divided?
From a computer perspective, from the access of a request to the final return, the performance bottleneck will only be: CPU, file IO, network IO, memory and other factors. If a certain resource is consumed too much, it will usually cause the system to respond slowly, so adding a machine makes the IO and CPU resources of the database exclusive to one machine to increase performance.

Reasons for various resource consumption

• CPU: Mainly context switching, each CPU can only execute one thread at the same time, and CPU scheduling includes preemption and polling. During the switching process, the CPU needs to store the execution status of the current thread and restore the status of the thread to be executed. Scenarios such as IO and lock waiting will trigger context switching. When there are too many context switches, the kernel will occupy more CPU.
• File IO: For example, frequent log writing and slow disk processing speed will cause IO performance problems.
• Network IO: Including memory leaks, memory overflows, and insufficient memory.
  In fact, whether it is application layer tuning or hardware upgrade, it is nothing more than the adjustment of these factors.

Application server complex alarm, how to move the server to the cluster

The pressure on the application server increases. Based on the detection results of the application, optimization can be carried out in areas with high performance pressure. You can consider turning a single machine into a cluster through horizontal expansion and retrograde optimization.
The application server has changed from one to two. There is no direct interaction between the two application servers. They both rely on the database to provide services outside the heap. Then two problems will be thrown at this time:

1. End user choice between two application servers? (This problem can be solved by DNS or load balancing equipment)
2. Session problem?

Horizontal and vertical expansion

For large-scale distributed architectures, we have been pursuing a simple and elegant way to cope with the increase in access and data volume. This method usually means that there is no need to change the software program, and it can be solved only by upgrading the hardware or increasing the number of machines. That is, scaling design under distributed architecture.

vertical telescopic

Support the growth in access and data volume by upgrading or adding hardware to a single machine. The advantages are low technical difficulty and low operation and modification costs. The disadvantage is that the machine performance has bottlenecks and the cost is high.

Horizontal scaling

The method of increasing the number of machines to support the growth of access and data volume is called horizontal scaling. Horizontal scaling theoretically has no bottlenecks, but the disadvantage is that the technical requirements are relatively high.

Introduce load balancing equipment

Service routing is implemented based on load balancing equipment.

Load balancing algorithm

polling method

Distribute the request sequence to the background servers in turn, and treat each server in a balanced manner, regardless of the actual number of connections to the server and the current system load.
Disadvantages: When the server hardware configurations in the cluster are different and the performance is greatly different, they cannot be treated differently.

random method

Through the system random function, one is randomly selected for access based on the size of the list in the background server. As the amount of calls increases, the actual effect is close to evenly distributing traffic to each server, which is the effect of the polling method.
Advantages: Simple to use, no additional configuration and algorithms required.
Disadvantages: The characteristic of random numbers is that balance can only be ensured when the amount of data is large to a certain extent, so if the request volume is limited, the load balancing requirements may not be met.

Source address hash

According to the client IP address requested by the service consumer, a hash value is calculated through a hash function. This hash value is modulo calculated with the size of the server list. The result is the serial number of the server address to be accessed. The source address hash method is used for load balancing. If the server list remains unchanged for the same IP client, it will be mapped to the same backend server for access.

weighted polling

The configuration of different backend servers may be different from the load of the current system, so their ability to withstand pressure is also different. Assign higher weights to machines with high configuration and low load so that they can handle more requests, while machines with low configuration and high load are assigned lower weights to reduce their system load. Requests are allocated in sequence and according to the weight. to the backend.

Minimum number of connections method

The previous centralized request methods can maximize the utilization of the server through reasonable distribution of the number of requests, but in fact the balance of the number of requests does not represent the balance of the load.
The minimum number of connections method dynamically selects a server with the smallest number of current backlog connections to handle the current request based on the current connection status of the back-end server, improves the utilization of the back-end server as much as possible, and reasonably distributes the load to each server. .

session problem

In some scenarios, requests need to have status characteristics, and the session+cookie mechanism is introduced to record the session of each request.
When the session starts, assign a unique session ID (sessionid) to the current session, and then tell the browser this ID through a cookie. Each time a request is made in the future, the browser will bring this session ID to tell the web server that the request belongs to Which session. On the web server, each session has independent storage and is stored in different sessions. If you encounter a situation where cookies are disabled, the general approach is to put this identifier in the URL.

Session sharing in a distributed environment

Session copy

Servers implement session replication or sharing: session data synchronization is added between WEB servers, and synchronization ensures the consistency of session data between different web servers. Generally, application containers support Session Replication mode.
There is a problem

1. Synchronizing session data causes network bandwidth overhead. As long as the Session data changes, the data needs to be synchronized to all other machines. The more machines there are, the greater the data overhead will be.
2. Each web server must save a lot of session data. If the entire cluster has a lot of session data, the content used to save session data on each machine will be very serious.
   This solution relies on the application container to complete Session replication to solve the Session problem. The application itself does not care about this matter. This solution is not suitable for scenarios with many cluster machines.

Session centralized storage

Use mature technologies for session replication, such as gemfire used by 12306, or the common in-memory database redis.
Session data is not saved to the local machine but is stored in a centralized storage place, and modifications to the Session also occur in the centralized storage place. Web servers use sessions to read from centralized storage. This ensures that the session data read by different web servers is the same. The specific way to store Session can be a database.
There is a problem

1. Reading and writing session data involves network operations, which causes delays and instability.
2. If there is a problem with the machine or cluster that centrally stores Sessions, it will affect the application.

Session is maintained on the client

Cookie storage is used, but there are risks on the client side, the data is not secure, and the amount stored is relatively small. To maintain the Session on the client side, the Session must be encrypted.

Database pressure increases, reading and writing are separated

As the business continues to grow, the amount of data and access continue to increase. For situations where there is more reading and less writing, you can consider the separation of reading and writing to optimize data pressure.
This structural change will bring about two problems:

1. How to synchronize data
   We hope to use the reading library to share the reading pressure on the main library, so the first thing that needs to be solved is how to copy to the reading library. Database systems generally provide database replication functions, and we can directly use the database's own mechanism. For example, MySQL's Master+slave replicates data.
2. How the application routes data sources
   to the application. Adding a read library has a certain impact on structural changes, that is, our applications need to choose different database sources according to different situations.

A search engine is actually a reading library

A search engine can be understood as a reading library. Products are stored in the database, and the website needs to provide users with real-time retrieval functions, especially product retrieval. For such read requests, if all libraries are read, there will actually be a bottleneck in performance. The use of search engines can not only greatly improve the retrieval speed, but also reduce the pressure of reading the database. The most important job of a search engine is to build an index based on all the data. As the data being searched changes, the index also needs to change accordingly.

A powerful tool to speed up data reading - cache and distributed storage

In large websites, storage and computing problems are basically solved, so storage is an important support system.

Distributed file systems and NoSQL

For some pictures and large texts, distributed file systems are used to implement file storage: Taobao's TFS, Google's GFS, and open source HDFS.

NoSQL can store some non-relational data.

Data cache

Large websites will use some data cache internally, mainly to share the reading pressure of the database. The Soul Village system is generally used to save and query key-value pairs. Application systems generally put hotspot data into the cache, and the filling of the secondary cache should also be completed by the application system. If the data does not exist, the data is read from the database and placed in the cache.

Page cache

In addition to data caching, we can also cache pages. Data caching can speed up the data reading speed when the application responds to requests, but the page is ultimately displayed to the user, some dynamically generated pages or pages with particularly high traffic. , you can do some caching of pages or content.

Make up for the shortcomings of relational databases and introduce distributed storage

We use relational databases the most, but in some scenarios relational databases are not suitable. Distributed storage systems are introduced, such as redis, mongoDB, cassandra, HBase, etc.
Depending on different scenarios and data structure types, choosing an appropriate distributed storage system can greatly improve performance. Distributed systems provide a high-capacity, high-concurrency, and data-redundant support through a cluster.

After separation of reading and writing, the database encountered bottlenecks again

By separating reading and writing and using distributed cache to replace relational databases in certain scenarios, the pressure on the main database can be reduced and the problem of data storage can be solved. But as the business continues to develop, our main library also encounters calmness.

Dedicated database, vertical data splitting

When the data of different businesses is split from one original database to multiple databases, you need to consider how to handle the original single-machine cross-business transactions.

1. Use distributed transactions.
2. Remove transactions or not pursue strong transaction support.

After vertically splitting the data, the pressure of putting all business data in one database is solved, and more optimizations can be carried out according to different business characteristics.

After vertical splitting, a bottleneck is encountered and the data is split horizontally.

Horizontal splitting of data is to split the data of the same table into two databases. The reason for horizontal splitting of data is that the data volume of a certain business data table or more reaches the bottleneck of a single database. At this time, You can split the table into two or more databases.

The difference between horizontal database splitting and vertical data splitting is that vertical splitting splits different tables into different databases, while horizontal splitting splits the same table into different databases.

The difference between horizontal data splitting and read-write separation is that read-write separation solves the problem of high reading pressure.

The impact of horizontal splitting:

1. SQL routing problem, Yuyao determines which database the current request is sent to based on a condition.
2. When processing primary keys, auto-increment ID cannot be used, and a global ID is required.
3. Since the data of one business is split into different databases, some queries need to be obtained across two databases. If the amount of data is large and paging is required, it will be more difficult to process.

Application challenges

As the business develops, more and more functions will be used, and the applications will become larger and larger. We need to consider how to prevent the application from becoming larger, which requires splitting the application from one application to multiple applications.

Servitization: Divide the application into three layers. The topmost web system is used to complete different business functions; the middle is some service centers, and different service centers provide different business services; the bottom is the business database.

There are several important changes compared to before:

1. Access to business functions is not only method calls within a single machine, but also introduces remote service calls.
2. Shared code is no longer scattered among different applications, the implementation is placed in various service centers.
3. The interaction of the database is placed in the service center, allowing the web application to pay more attention to the interaction with the browser without having to pay too much attention to the business logic.

What is distributed architecture

Definition of distributed architecture

A distributed system is one where components located on a network of computers communicate and coordinate a target system solely by passing messages. There are two important factors:

1. Components are distributed over the network.
2. Components communicate and coordinate actions only through messages.

The meaning of distributed architecture

1. Upgrading a single machine to improve performance is increasingly less cost-effective.
2. There is a bottleneck in single-machine processing.
3. Requirements for stability and availability.