This article is transferred from [ Today's headlines ]
This article uses Taobao as an example to introduce the evolution process of the server architecture from 100 concurrent to tens of millions of concurrency. It also lists the relevant technologies encountered in each evolution stage, so that everyone has a overall understanding of the evolution of the architecture At the end, the article summarizes some principles of architectural design.
Before introducing the architecture, in order to avoid that some readers do not understand some concepts in the architecture design, here are a few of the most basic concepts:
① Distributed: Multiple modules in the system are deployed on different servers, which can be called a distributed system. For example, Tomcat and database are deployed on different servers, or two Tomcats with the same function are deployed on different servers.
② High availability: When some nodes in the system fail, and other nodes can take over it to continue to provide services, it can be considered that the system has high availability.
③Cluster: A specific field of software is deployed on multiple servers and provides a type of service as a whole, this whole is called a cluster.
For example, the Master and Slave in Zookeeper are deployed on multiple servers to form a whole to provide centralized configuration services.
In a common cluster, the client is often able to connect to any node to obtain service, and when a node in the cluster goes offline, other nodes are often able to automatically replace it to continue to provide services, which shows that the cluster has high availability.
④ Load balancing: When the request is sent to the system, the request is evenly distributed to multiple nodes in some ways, so that each node in the system can handle the request load evenly, then the system can be considered to be load balanced.
⑤ Forward proxy and reverse proxy: When the system wants to access the external network, the request is forwarded through a proxy server in a unified manner. From the external network's perspective, it is the access initiated by the proxy server. At this time, the proxy server implements a forward proxy.
When an external request enters the system, the proxy server forwards the request to a server in the system. For the external request, only the proxy server interacts with it. At this time, the proxy server implements a reverse proxy.
In simple terms, forward proxy is the process that the proxy server replaces the internal system to access the external network, and reverse proxy is the process that the external server forwards to the internal server through the proxy server when requesting access to the system.
Architecture evolution
Stand-alone architecture
Take Taobao as an example: At the beginning of the website, the number of applications and the number of users are small, and you can deploy Tomcat and the database on the same server.
When the browser initiates a request to www.taobao.com , it first converts the domain name to the actual IP address 10.102.4.1 through the DNS server (domain name system), and the browser instead accesses the Tomcat corresponding to the IP.
As the number of users grows, resources compete between Tomcat and the database, and the stand-alone performance is insufficient to support the business.
First evolution: Tomcat is deployed separately from the database
Tomcat and database monopolize server resources respectively, significantly improving their respective performance. As the number of users grows, concurrent read and write databases become a bottleneck.
Second evolution: introduction of local cache and distributed cache
Add a local cache on the same server as Tomcat or the same JVM, and add a distributed cache externally to cache popular product information or HTML pages of popular products.
The cache can intercept most requests before reading and writing to the database, greatly reducing database pressure.
The technologies involved include the use of Memcached as a local cache and Redis as a distributed cache, as well as issues such as cache consistency, cache penetration / breakdown, cache avalanches, and invalidation of hotspot data sets.
The cache resists most access requests. As the number of users grows, the concurrency pressure mainly falls on the stand-alone Tomcat, and the response gradually slows down.
The third evolution: the introduction of reverse proxy to achieve load balancing
Deploy Tomcat on multiple servers separately, and use reverse proxy software (Nginx) to distribute the request evenly to each Tomcat.
It is assumed here that Tomcat supports a maximum of 100 concurrency and Nginx supports a maximum of 50,000 concurrency. In theory, Nginx can distribute requests to 500 Tomcats, which can resist 50000 concurrency.
The technologies involved include: Nginx and HAProxy, both of which are reverse proxy software working at the seventh layer of the network. They mainly support the HTTP protocol, and also involve session sharing and file upload and download issues.
Reverse proxy greatly increases the amount of concurrency that the application server can support, but the increase in the amount of concurrency also means that more requests penetrate the database, and the stand-alone database eventually becomes a bottleneck.
The fourth evolution: database read-write separation
Divide the database into a reading library and a writing library. There can be multiple reading libraries. Synchronize the data of the writing library to the reading library through the synchronization mechanism. For the scenario where the latest written data needs to be queried, write one more copy in the cache. Cache to get the latest data.
The technologies involved include: Mycat, which is a database middleware, which can be used to organize the separate read and write of the database and the sub-database and sub-table. The client accesses the underlying database through it, and also involves data synchronization and data consistency issues. .
Businesses are gradually increasing, and the gap in access between different businesses is large. Different businesses directly compete for databases and affect each other's performance.
Fifth evolution: the database is divided into business bases
Saving data of different businesses in different databases reduces the resource competition between businesses. For businesses with a large number of visits, more servers can be deployed to support them.
At the same time, cross-business tables cannot be directly correlated and need to be resolved through other channels, but this is not the focus of this article. Interested parties can search for solutions by themselves. As the number of users grows, the standalone write library will gradually reach the performance bottleneck.
Sixth evolution: split large tables into smaller tables
For example, for the review data, you can hash according to the product ID and route it to the corresponding table for storage; for payment records, you can create a table by hour, and each hour table continues to be split into small tables, using the user ID or record number to route the data .
As long as the amount of table data for real-time operations is small enough and requests can be distributed to small tables on multiple servers evenly, the database can improve performance by horizontal expansion. Among them, Mycat mentioned above also supports access control when the large table is split into small tables.
This approach significantly increases the difficulty of database operation and maintenance, and has higher requirements for DBAs. When the database is designed to this structure, it can already be called a distributed database.
But this is just a logical database as a whole. Different components in the database are implemented by different components separately.
For example, the management and request distribution of the sub-database and sub-table are implemented by Mycat, the analysis of SQL is implemented by a single-machine database, the separation of reading and writing may be implemented by the gateway and the message queue, and the summary of the query results may be implemented by the database interface layer, etc. This architecture is actually a type of implementation of MPP (mass parallel processing) architecture.
At present, there are many MPP databases in both open source and commercial applications. Among the open source, Greenplum, TiDB, Postgresql XC, HAWQ, etc., such as GBase from Nanda General, Snowball DB from Ruifan Technology, LibrA from Huawei, etc.
Different MPP databases have different emphasis. For example, TiDB focuses more on distributed OLTP scenarios, and Greenplum focuses more on distributed OLAP scenarios.
These MPP databases basically provide SQL standard support capabilities like Postgresql, Oracle, and MySQL. They can parse a query into a distributed execution plan and distribute it to each machine for parallel execution. Finally, the database itself aggregates the data and returns it.
It also provides capabilities such as rights management, database and table division, transactions, and data copies, and most of them can support clusters of more than 100 nodes, greatly reducing the cost of database operation and maintenance, and enabling the database to achieve horizontal expansion.
Both the database and Tomcat can be horizontally expanded, and the supportable concurrency is greatly improved. With the growth of the number of users, the Nginx of the single machine will eventually become a bottleneck.
Seventh evolution: use LVS or F5 to load balance multiple Nginx
Because the bottleneck is in Nginx, it is impossible to achieve load balancing of multiple Nginx through two layers of Nginx.
The LVS and F5 in the figure are load balancing solutions that work at the fourth layer of the network. LVS is software that runs in the kernel state of the operating system and can forward TCP requests or higher-level network protocols.
Therefore, the supported protocols are more abundant, and the performance is much higher than Nginx. It can be assumed that a single LVS can support hundreds of thousands of concurrent request forwarding; F5 is a load balancing hardware, similar to the capabilities provided by LVS, and the performance is more than LVS High, but expensive.
Because LVS is a stand-alone version of the software, if the server where LVS is located goes down, the entire back-end system will be inaccessible, so a standby node is required.
You can use Keepalived software to simulate the virtual IP, and then bind the virtual IP to multiple LVS servers. When the browser accesses the virtual IP, it will be redirected to the real LVS server by the router.
When the main LVS server goes down, Keepalived software will automatically update the routing table in the router and redirect the virtual IP to another normal LVS server, thus achieving the effect of high availability of the LVS server.
It should be noted here that the drawing from the Nginx layer to the Tomcat layer in the above figure does not mean that all Nginx forward requests to all Tomcat.
In actual use, it may be that several Nginx are connected to a part of Tomcat, and these Nginx are realized through Keepalived for high availability, and the other Nginx is connected to another Tomcat, so that the number of accessible Tomcat can be doubled.
Since LVS is also stand-alone, as the number of concurrency grows to hundreds of thousands, the LVS server will eventually reach a bottleneck. At this time, the number of users reaches 10 million or even hundreds of millions. The users are distributed in different regions, and the distance from the server room is different, resulting in The delay of access will be significantly different.
The eighth evolution: load balancing of machine rooms through DNS polling
A domain name can be configured in the DNS server corresponding to multiple IP addresses, and each IP address corresponds to a virtual IP in a different computer room.
When a user visits www.taobao.com , the DNS server uses a polling strategy or other strategies to select an IP for the user to access.
In this way, the load balance of the computer room can be achieved. At this point, the system can achieve the horizontal expansion of the computer room level. The concurrency of tens of millions to 100 million can be solved by increasing the computer room. .
With the richness of data and the development of business, the demand for retrieval and analysis is becoming more and more abundant. Relying on the database alone cannot solve such a rich demand.
The ninth evolution: introduction of technologies such as NoSQL databases and search engines
When the data in the database reaches a certain size, the database is not suitable for complex queries, and often can only meet the common query scenarios.
For statistical report scenarios, the results may not be able to be ran out when the amount of data is large, and it will cause other queries to slow down when running complex queries. For scenarios such as full-text retrieval and variable data structures, the database is inherently inapplicable.
Therefore, it is necessary to introduce suitable solutions for specific scenarios. For mass file storage, it can be solved through the distributed file system HDFS.
For Key Value type data, it can be solved through solutions such as HBase and Redis. For full-text retrieval scenarios, it can be solved by search engines such as ElasticSearch. For multi-dimensional analysis scenarios, it can be solved by Kylin or Druid.
Of course, introducing more components will increase the complexity of the system at the same time. The data saved by different components needs to be synchronized, consistency needs to be considered, and more operations and maintenance methods are needed to manage these components.
Introducing more components to solve the rich demand, the business dimension can be greatly expanded, and then comes an application that contains too much business code, and iterative business upgrade becomes difficult.
Tenth evolution: big application split into small applications
The application code is divided according to the business sector to make the responsibilities of individual applications clearer, and each can be upgraded and iterated independently.
At this time, some common configurations may be involved between applications, which can be solved through the distributed configuration center Zookeeper.
There are common modules between different applications, and the separate management of the applications will result in multiple copies of the same code, resulting in the upgrade of all application codes when the public functions are upgraded.
The eleventh evolution: the reuse of multiple functions into microservices
If user management, order, payment, authentication and other functions exist in multiple applications, then the code of these functions can be extracted separately to form a separate service for management.
Such a service is a so-called microservice, and applications and services access public services through HTTP, TCP, or RPC requests and other methods. Each individual service can be managed by a separate team.
In addition, services such as service governance, current limiting, fusing, and downgrade can be implemented through Dubbo, Spring Cloud and other frameworks to improve the stability and availability of services.
Different services have different interface access methods, and application codes need to adapt to multiple access methods to use services.
In addition, applications access services, and services may also access each other. The call chain will become very complicated and the logic becomes chaotic.
Twelfth evolution: introduction of enterprise service bus ESB to shield access differences of service interfaces
Unified access protocol conversion through ESB, applications uniformly access back-end services through ESB, and services and services also call each other through ESB, thereby reducing the degree of system coupling.
This single application is split into multiple applications, public services are extracted separately for management, and the enterprise message bus is used to solve the problem of coupling between services. This is the so-called SOA (service-oriented) architecture. This architecture is related to microservices. The architecture is easy to confuse because the form of expression is very similar.
Personally understand, the microservice architecture refers more to the idea of extracting the public services in the system for separate operation and maintenance management, and the SOA architecture refers to an architectural idea of splitting services and making service interface access unified. SOA architecture The idea of microservices is included.
With the continuous development of business, applications and services will continue to increase, the deployment of applications and services becomes complicated, and the deployment of multiple services on the same server must also solve the problem of conflicting operating environments.
In addition, for scenarios that require dynamic expansion and contraction, such as the big promotion, if you need to horizontally expand the performance of the service, you need to prepare the operating environment on the newly added service, deploy the service, etc., and operation and maintenance will become very difficult.
Thirteenth evolution: introduction of containerization technology to achieve isolation of operating environment and dynamic service management
Currently, the most popular containerization technology is Docker, and the most popular container management service is Kubernetes (K8S). Applications / services can be packaged as Docker images, and images are dynamically distributed and deployed through K8S.
The Docker image can be understood as a minimal operating system that can run your application / service, where the application / service running code is placed, and the operating environment is set according to actual needs.
After the entire "operating system" is packaged as an image, it can be distributed to the machines that need to deploy related services, and directly starting the Docker image can bring the service up, making the deployment and operation and maintenance of the service simple.
Before the big promotion, you can divide the server on the existing machine cluster to start the Docker image to enhance the performance of the service. After the big promotion, you can close the mirror without affecting other services on the machine (before, the service was running The system configuration needs to be modified on the newly added machine to adapt to the service, which will lead to the destruction of the operating environment required by other services on the machine).
After using containerization technology, the problem of dynamic expansion and contraction of services can be solved, but the machine still needs to be managed by the company itself. When it is not a big promotion, it still needs to idle a lot of machine resources to deal with the big promotion. The cost is extremely high and the resource utilization rate is low.
Fourteenth Evolution: Carrying System with Cloud Platform
The system can be deployed on the public cloud, using the massive machine resources of the public cloud to solve the problem of dynamic hardware resources.
In the time period of the big promotion, apply for more resources temporarily in the cloud platform, combine Docker and K8S to quickly deploy services, release resources after the big promotion, truly pay on demand, and the resource utilization rate is greatly improved. Greatly reduces the operation and maintenance costs.
The so-called cloud platform is to abstract a large amount of machine resources into a whole resource through unified resource management. On the top, you can dynamically apply for hardware resources (such as CPU, memory, network, etc.) on demand, and provide general operations system.
Provide commonly used technical components (such as Hadoop technology stack, MPP database, etc.) for users to use, and even provide developed applications, users can solve the needs (such as audio and video transcoding services, email Services, personal blogs, etc.).
The following concepts will be involved in the cloud platform:
- IaaS: Infrastructure as a service. Corresponding to the above-mentioned machine resources are unified into the whole resource, the level of hardware resources can be applied dynamically.
- PaaS: Platform as a service. Corresponding to the above provides common technical components to facilitate system development and maintenance.
- SaaS: Software as a service. Corresponding to the above-mentioned provision of developed applications or services, payment is made according to function or performance requirements.
So far, from the high-concurrency access problem mentioned above, to the service architecture and system implementation level, there are respective solutions.
But at the same time, it should also be realized that in the above introduction, actual problems such as cross-computer room data synchronization and distributed transaction implementation are intentionally ignored. These problems will have the opportunity to be discussed separately later.
Architecture design summary
① Does the adjustment of the structure have to follow the above evolution path?
No, the above-mentioned architecture evolution sequence is only for individual improvement on a certain side. In the actual scenario, there may be several problems to be solved at the same time, or the bottleneck may be reached by another aspect at this time. It should be solved according to actual problems.
For example, in the case of government websites, the concurrency may not be large, but the business may be very rich. High concurrency is not a problem to be solved. At this time, the solution that may be required first is a rich demand.
② To what extent should the architecture be designed for the system to be implemented?
For a single-implemented system with clear performance indicators, it is sufficient to design the architecture to support the performance indicators of the system, but leave an interface to extend the architecture so that it is not needed.
For evolving systems, such as e-commerce platforms, they should be designed to meet the requirements of the next stage of user volume and performance indicators, and the architecture should be iteratively upgraded according to business growth to support higher concurrency and richer services .
③What is the difference between server-side architecture and big data architecture?
The so-called "big data" is actually a general term for solutions for scenarios such as massive data collection, cleaning, conversion, data storage, data analysis, and data services. Each scenario contains multiple optional technologies.
For example, data collection includes Flume, Sqoop, Kettle, etc., data storage includes distributed file systems HDFS, FastDFS, NoSQL databases HBase, MongoDB, etc., and data analysis includes Spark technology stack, machine learning algorithms, etc.
In general, big data architecture is an architecture that integrates various big data components according to business needs, and generally provides distributed storage, distributed computing, multi-dimensional analysis, data warehouse, machine learning algorithms and other capabilities.
The server-side architecture refers more to the architecture at the application organization level, and the underlying capabilities are often provided by the big data architecture.
④ Are there any principles of architectural design?
The design principles are as follows:
- N + 1 design. Every component in the system should be free of single points of failure.
- Roll back design. Ensure that the system is forward compatible, and there should be a way to roll back the version when the system is upgraded.
- Disable the design. It should provide a configuration to control whether specific functions are available, and can quickly go offline when the system fails.
- Monitoring design. In the design stage, we must consider the means of monitoring.
- Multi-live data center design. If the system requires extremely high availability, you should consider implementing multiple activities in multiple data centers. The system is still available when at least one computer room is powered off.
- Use mature technology. Newly developed or open source technologies often have a lot of hidden bugs. If there is a problem, no commercial support may be a disaster.
- Resource isolation design. A single service should avoid all resources.
- The architecture should be able to scale horizontally. Only when the system can expand horizontally can bottleneck problems be effectively avoided.
- Non-core is purchased. If non-core functions require a large amount of R & D resources to be resolved, consider purchasing mature products.
- Use commercial hardware. Commercial hardware can effectively reduce the probability of hardware failure.
- Iterate quickly. The system should quickly develop small function modules, go online as soon as possible for verification, and find problems early to greatly reduce the risk of system delivery.
- Stateless design. The service interface should be made stateless, and the current interface access does not depend on the state of the interface last accessed.
Author: China Oushi
Introduction: Has many years of development experience in the field of big data, has an understanding of commonly used big data technologies, and has some experience in architecture design, high concurrency, and distribution. I like to learn new technologies and are happy to share. Welcome to follow my SegmentFault blog: huashiou.
Author: opera fans into the sub-
link: https: //www.jianshu.com/p/537b3ee7229d
Source: Jane books
are copyrighted by the author. For commercial reproduction, please contact the author for authorization, and for non-commercial reproduction, please indicate the source.