Existence is reasonable. Every new technology has its reason for its existence. When learning a new technology, I personally feel that we should first understand its ins and outs and historical origin, because the business determines the technology, and the technology should be carried out correspondingly according to different businesses. Selection, rather than blindly following the solutions of big companies, technology for technology's sake. From a small website accessed by a few people to a large system accessed by hundreds of millions of users, the technology and architecture used must also be changed accordingly. The following is based on my reading experience in the book "Technical Architecture of Large Websites - Core Principles and Case Analysis" Introduce the evolution of large-scale website architecture.
Characteristics of large-scale website software systems
There are 4 points that need special attention:
- High concurrency and large traffic. It needs to face high concurrent users and large traffic access.
- High availability. The system provides services without interruption without downtime.
- Massive Data. Need to use a large number of servers to store and manage data.
- Bad security environment. High attention should be paid to hacker attacks.
The evolution and development process of large-scale website architecture
The following introduces the corresponding technical architectures according to the 10 stages of business scale from small to large:
1. Initial stage
To develop a small website, only one server is needed. On the server, you only need to deploy the application server, database server, and then deploy the application to access. The architecture diagram is as follows:
2. Separation of application services and data services
When the business scale of the application system is large, considering the performance of the application and the storage space of the database, it is necessary to consider the separation of the application service and the data service. The server that provides the service needs to be equipped with better CPU, and the server that provides the database service needs to be equipped with better hard disk and memory. The book also mentions a file server. In fact, file servers are rare in enterprise applications. If the user uploads a lot of files, a server with a larger hard disk needs to be opened. At this point, the architecture diagram is as follows:
3. Use cache
The business system often has some data that needs to be accessed frequently, but is not frequently modified, such as all subordinate units of a group company, and the chart summary data on the home page. For such data, caching technology should be used to reduce interaction with the database and improve the performance. Access speed, so that the pressure on the database server to process read requests can be effectively relieved.
There are two types of caches used by websites, one is the local cache on the application server, which is limited by memory and the amount of cached data is limited, and the other is the remote distributed cache, which can solve the problem of memory capacity limitation through hardware configuration. At this point, the architecture diagram is as follows:
4. Use application server clusters
When the business system needs to accept a larger number of concurrent accesses, even the best configuration of one application server may not solve the problem, while the cluster technology is to share the access and storage of the original application server by adding multiple application servers. Storage capacity, which requires a server with load balancing adjustment to be responsible for judging and distributing user access requests to a certain application server. The architecture diagram is as follows:
5. Database read and write separation
When the number of users in the business system grows to a certain scale, even if the cache technology is used, it is necessary to consider the impact of a large number of read and write operations on the performance of the database server. At this time, the common practice is to divide the database into two sets, one as the main library, responsible for write operations, and one as the slave library, responsible for read operations, and the slave library needs to be copied from the master library, and the configuration requirements of the master library must be higher. Through read-write separation, the load pressure of the database server can be effectively improved. The architecture diagram is as follows:
6. Use a reverse proxy and CDN
When users in different regions visit the website, sometimes there will be delay in website access. At this time, reverse proxy and CDN technology need to be used. The basic principle is caching technology. When users request website services, the CDN server will judge the read In the nearest network provider room, the reverse proxy caches the resources requested by the user. The architecture diagram is as follows:
7. Use a distributed database system
In a business system, when the demand cannot be met even after the read-write separation of the database, it is necessary to consider dividing the database into a business database, and deploying the databases of different businesses on different physical servers. The architecture diagram is as follows:
8. Use nosql and search engines
When the business system has special requirements for storage and retrieval of some data, two more servers can be considered at this time, namely the nosql server and the search engine server, which can effectively reduce the access pressure of the database server and manage multiple data. source of trouble. The architecture diagram is as follows:
9. Business split
In order to cope with increasingly complex business scenarios, large-scale systems divide the entire website into different application systems by means of divide and conquer. In terms of technology, each application system can be independently deployed and maintained. Application systems can access each other through hyperlinks, and can also create a new message queue server for data distribution. The architecture diagram is as follows:
10. Distributed Services
For large-scale application websites, after the distributed database and business split are performed, each application system needs to establish connections with all distributed databases. When there are many application systems, only the database may lead to insufficient database connection resources. . The solution at this time is to extract the public services in all the application systems, establish connections with the required distributed databases, provide business services, and carry out independent deployment. At this time, the application system can reduce the number of direct and distributed databases. connect. The architecture of large websites has evolved here, and basically most of the technical problems have been solved. At this point, the architecture diagram is as follows:
