Typical Web App Architecture
reverse proxy service
| The following is an example of a typical high-load web application: | The above figure shows a typical high-performance web application with three-tier architecture. This mature architecture has been widely deployed in many large web applications including Google, Yahoo, Facebook, Twitter, and Wikipedia for many years. |
| The outermost reverse proxy server in the three-tier architecture is responsible for accepting user access requests. In practical applications, the proxy server usually completes at least some of the tasks in the following list: |
Connection management : maintain the connection pools of the client and the application server respectively, manage and close long connections that have timed out.
Attack detection and security isolation : Since the reverse proxy service does not need to complete any dynamic page generation tasks, all requests related to business logic are forwarded to the back-end application server for processing. Therefore, the reverse proxy service is hardly affected by application design or backend data vulnerabilities. The security and reliability of a reverse proxy usually depends only on the product itself. Deploying a reverse proxy server at the front-end of application services can effectively establish a reliable security isolation and attack detection mechanism between back-end applications and remote users.
If required, higher security assurance can be achieved by adding additional network isolation devices such as hardware firewalls at border locations such as external networks, reverse proxies, back-end applications, and databases.
Load balancing : Usually, strategies such as Round Robin or the least number of connections are used to complete load balancing based on client requests; technologies such as SSI can also be used to split a client request into several parallel computing parts and submit them to multiple application servers respectively.
Distributed cache acceleration : Reverse proxy groups can be deployed on network boundaries geographically close to hotspots. Accelerates web applications by providing buffering services close to customers. This actually constitutes the CDN network.
Static file server : When a static file request is received, the file is directly returned without submitting the request to the back-end application server.
Dynamic response caching : Cache dynamically generated responses that will not change within a period of time to avoid repeated queries and calculations performed frequently by the back-end application server.
Data Compression Transport : Enable GZIP/ZLIB compression algorithm for returned data to save bandwidth.
Data Encryption Protection (SSL Offloading) : Enable SSL/TLS encryption protection for communication with clients.
Fault Tolerance : Track the health of backend application servers and avoid dispatching requests to failed servers.
User authentication : Complete the work of user login and session establishment.
URL alias : establish a unified url alias information externally to shield the real location.
Application Mashup : Mash different web applications together through SSI and URL mapping techniques.
Protocol conversion : Provides protocol conversion services for backend applications using protocols such as SCGI and FastCGI.
At present, the more famous reverse proxy services include: Apache httpd+mod_proxy / IIS+ARR / Squid / Apache Traffic Server / Nginx / Cherokee / Lighttpd / HAProxy and Varnish and so on. |
application service
The application service layer is located between the back-end general service layer such as the database and the reverse proxy layer. It receives client access requests forwarded by the reverse proxy service upward, and accesses the structured storage and data query services provided by the database layer downward. .
The application layer implements all the business logic of the Web application, and usually completes a large number of computing and data dynamic generation tasks. Each node in the application layer is not necessarily completely equal, and may also be split into different service clusters based on SOA, μSOA and other architectures.
The above figure shows a typical high-concurrency, high-performance application layer node working model. Each Web application node (represented by the box marked "App" in Figure 5) usually works on its own server (physical server or VPS), and multiple application nodes can effectively work in parallel to facilitate to achieve horizontal expansion.
In the example shown in the figure above, the web application node consists of three important parts: the IO callback thread pool, the web request queue, and the background worker thread pool. The servo process is as follows:
When a web request arrives, the underlying operating system completes (or is IO ready) through IOCP, epoll, kqueue, event ports, real time signal (posix aio), /dev/poll, pollset and other types of IO closely related to specific platforms. The callback mechanism notifies the AIO (Asynchronous IO) callback thread to process the web request that has arrived.
After a worker thread in the AIO callback pool receives an incoming web request, it first tries to preprocess the request. During preprocessing, a local cache is used to avoid costly database queries. If the local cache hits, the result in the cache is directly returned to the client (still in the form of asynchronous IO), and the request is ended.
If the specified web request requires that the queried data cannot be hit by the local cache, or the web request requires a database write operation, the request will be appended to the specified queue by the AIO callback thread, waiting for an idle one in the background worker thread pool The thread processes it further.
Each thread in the background worker thread pool maintains two persistent connections: one is connected to the underlying database service, and the other is connected to the distributed cache (memcached) network. By allowing each worker thread to maintain its own long connection, the background worker thread pool implements the database and distributed cache connection pool mechanism. Keep-Alive greatly improves application processing efficiency and network utilization by reusing the same network connection for different requests.
Background worker threads wait for new requests to arrive on the Web request queue. After taking a new request from the queue, the background worker thread first tries to use the distributed cache service to hit the query operation in the request. If the network cache misses or the request requires further processing such as database writing, the database operation is performed directly. to complete this web request.
When a web request is processed, the background worker thread will return the processing result as a web response to the specified client in the form of asynchronous IO.
The above steps outline how a typical Web application node works. It is worth noting that due to the differences in design ideas and specific functions, there may be great differences in working modes or architectures between different Web applications.
It should be noted that, unlike the phase-triggered notification mechanisms such as epoll/kqueue/event ports, for edge-triggered AIO completion event notification mechanisms such as Windows IOCP and POSIX AIO Realtime Signal, in order to avoid the underlying IO completion queue (or real-time The memory buffer is locked in the non-paged memory pool for a long time due to overlong signal queue) or overflow. The AIO callback method in the above system is actually composed of two independent thread pools and an AIO completion event queue: a The thread pool is specifically responsible for continuously waiting for events arriving in the system AIO completion queue and submitting them to an internal AIO completion queue (this queue works in user mode, has a user-controlled elastic size, and does not lock memory ); at the same time another thread pool waits on this internal AIO completion queue and processes AIO completion events that keep arriving on the queue. This design reduces the workload of the operating system, avoids message loss, memory leaks, and memory exhaustion that may occur in extreme cases, and also helps the operating system to better use and manage the non-paged memory pool.
As a typical case: most of Google's web applications, including search engine, Gmail mail service, are implemented using C/C++. Thanks to the efficiency and power of the C/C++ language, Google not only provides the best web application experience for Internet users around the world, but also completes a web search on its millions of distributed servers around the world. The excellent performance of only 0.0003 kW h. For further discussion on Google Web application architecture and hardware scale, you can add this group to get it: Exchange learning group: 744642380 There will be some video recordings recorded by senior architects: Spring, MyBatis, Netty source code analysis, high concurrency, high performance , The principles of distributed and microservice architecture, and JVM performance optimization have become a must-have knowledge system for architects. You can also receive free learning resources, which are currently benefiting a lot
Database and memcached service
Database services provide relational or structured data storage and query support for upper-layer Web applications. Depending on the specific use case, web applications can use plug-in mechanisms such as database connectors to provide access support to different database services. Under this architecture, users can flexibly select or change different database products that are most suitable for the current situation of the enterprise. For example: users can use an embedded engine such as SQLite to complete rapid deployment and functional verification in the prototype stage; switch to a cheap MySql database solution in the initial stage of the application; wait until the business demand continues to rise and the database load continues to increase before switching to Migrate from more expensive and complex solutions like Clustrix, MongoDB, Cassandra, MySql Cluster, ORACLE, etc.
The Memcached service is a distributed data object caching service based entirely on memory and <Key, Value> pairs, with incredible query efficiency and an elegant large-scale distributed architecture without inter-server communication. For high-load web applications, Memcached is often used as an important database access acceleration service, so it is not a required component. Users can wait until the database service in the real environment has a performance bottleneck before deploying it. It is worth emphasizing that although memcached is not a required component, its years of deployment in large-scale web applications such as YouTube, Wikipedia, Amazon.com, SourceForge, Facebook, Twitter, etc. can prove that memcached can not only be used in high-load environments Long-term stable work, and can dramatically improve the overall efficiency of data query. For further discussion of memcached, you can add this group to get it: Exchange learning group: 744642380 There will be some video recordings recorded by senior architects: Spring, MyBatis, Netty source code analysis, high concurrency, high performance, distributed, microservices The principles of architecture and JVM performance optimization have become a necessary knowledge system for architects. You can also receive free learning resources, which are currently benefiting a lot
Of course, we should also note that the distributed cache system represented by memcached is essentially a compromise solution to improve the average access efficiency at the expense of consistency - the cache service increases some records in the database Distributed copy. For multiple distributed copies of the same data, strong consistency guarantees cannot be achieved unless a consensus algorithm such as Paxos and Raft is used.
Paradoxically, the memory cache itself is used to improve efficiency, which makes it impractical to use the above-mentioned expensive distributed strong consistency algorithm: the current distributed strong consistency algorithm requires every access request (Whether reading and writing) requires simultaneous access to the majority of replicas including the master and slave nodes of the background database - obviously, this is not as efficient as not using the cache at all.
Also, even distributed consensus algorithms like Paxos and Raft can only guarantee strong consistency at the level of a single record. That is: even if such an algorithm is applied, it cannot provide strong transaction-level consistency guarantees.
In addition to this, distributed caching also increases program design complexity (requires trying to hit or update the cache while accessing the database), and also increases access latency in worst-case scenarios (e.g. RTT waits on misses) delays, as well as delays when nodes go offline, network communication failures, etc.).
At the same time, it can be seen that since 20 years ago, all mainstream database products have already implemented a mature and high hit rate multi-layer (disk block, data page, result set, etc.) caching mechanism. Since the distributed cache has so many defects, and the database product has its own excellent caching mechanism, why can it become an important cornerstone of modern high-load Web App?
The fundamental reason is that for the technical environment ten years ago, the RDBMS (SQL) system, which lacked the ability to scale horizontally at that time, has become a bottleneck that severely restricts the expansion of network applications such as Web App. To this end, NoSQL database products represented by Google BigTable, Facebook Cassandra, and MongoDB, as well as distributed cache products represented by memcached and redis, have appeared one after another, and each played an important role.
Compared with MySQL, ORACLE, DB2, MS SQL Server, PostgreSQL and other "traditional" SQL database products at that time, both NoSQL databases and distributed cache products are essentially at the expense of the strong consistency of the former in exchange for Better horizontal scalability.
It should be seen that this choice was a helpless and painful choice made under the technical conditions at the time, and the system became complicated as a result - where transaction and strong consistency guarantees are required, and the amount of data is small, use no cache Layer traditional RDBMS; there is a certain room for compromise in consistency, and use distributed cache as much as possible to accelerate where there are more reads and fewer writes; use NoSQL on big data with lower requirements for consistency; if the amount of data is large, at the same time The requirements for consistency are also high, so we can only try to solve the problem by sub-database and sub-table of RDMBS. For this reason, we need to develop various middleware to realize complex operations such as data access request distribution and result set aggregation... There are many different scenarios, and their combination and interweaving adds to the complexity again.
Looking back, this is a chaotic era in which the old order was broken and the new order has not yet been established. The old RMDBS lacked the ability to scale horizontally, could not meet the big data processing needs of the new era, and none of them could replace the status of the old system. A pervasive structured data management solution that can meet the needs of most users at the same time.
This is an era of green and yellow, and products such as BigTable, Cassandra, and memcached are the result of "self-help" by manufacturers such as Google, Facebook, and LiveJournal in that era. In this way, products with the goal of "meeting your own business needs at the least cost" are naturally not easy to have good universality.
However, today (2015), we are finally coming out of this predicament. With the gradual maturity of Google F1, MySQL Cluster (NDB), Clustrix, VoltDB, MemSQL, NuoDB and many other NewSQL solutions and the continuous advancement of technology, horizontal scalability is no longer the bottleneck of RDBMS. Today's architects can ensure that the system has sufficient horizontal scalability while achieving distributed transaction-level (XA) strong consistency guarantees:
As shown in the figure above, after NewSQL has a good horizontal expansion capability, distributed cache and NoSQL products are no longer urgently needed in the architecture to make up for this shortcoming, which makes the design and development work return to the original simplicity and clarity again . The Object Storage service provides storage and access support for massive unstructured BLOB data such as audio, video, pictures, and file packages.
Such a concise, clear and simple architecture makes everything seem to have returned to years ago. Object storage services are like FAT, NTFS, Ext3 and other disk file systems, and NewSQL services are like MySQL, SQL Server and other "stand-alone" databases. But everything is different. Business logic, database and file storage have all evolved into highly available clusters that support horizontal scaling, and have made huge leaps in performance, capacity, availability, reliability, scalability, etc.: human beings always Continue to progress in a spiral way - in every change that seems to return, it actually contains the sublimation of essence.
As GlusterFS, Ceph, Lustre and other mountable distributed file systems that support the Native File API become more mature and complete, it is also expected to gradually replace existing object storage services in most occasions. So far, the evolution of the Web App architecture can be regarded as a rebirth - this is not Nirvana. When we can achieve an efficient and highly available multi-virtual one (Single System Image) system in the true sense, Nirvana will be truly come. At that time, we wrote a distributed application and there would be no difference between writing a single-machine version of a multi-threaded application today - the process is naturally distributed and highly available!
Scalability of three-tier architecture
From small to centralized deployment in a single physical server or VPS, to large distributed applications composed of millions of physical servers across the world. The three-tier Web application architecture described earlier exhibits incredible scalability.
Specifically, in the early stages of project verification, application deployment and service operation, the above three-tier service components can be deployed in the same physical server or VPS. At the same time, the deployment difficulty and overall system resource overhead can be further reduced by canceling the memcached service and using embedded database products with low resource overhead and easy deployment.
As the project operation expands and the load continues to increase, when the single-server solution and simple vertical expansion can no longer meet the project operation load, users can achieve horizontal expansion by running each component in a distributed manner on multiple servers. Purpose. For example, a reverse proxy can achieve distributed load balancing through DNS CNAME record rotation or layer 3/4 forwarding (LVS, HAProxy, etc.). Application services can be distributed and load balanced by reverse proxy using strategies based on round-robin or least load first. In addition, the use of shared IP-based server cluster solutions can also achieve load balancing and fault tolerance.
Similarly, memcached and database products also have their own distributed computing, load balancing and fault tolerance solutions. In addition, database access performance bottlenecks can be improved by replacing non-relational (NoSQL) database products, or using master-slave databases and replication. Database query performance can be greatly improved by deploying memcached or similar services.