Preface
Senior distributed system R&D engineers and architects have summarized years of work experience, from the front-end to the back-end, from the network transmission to the load balancing, from the transaction to the database, from the high concurrency to the high availability, from the three dimensions of principle, application and practice. Comprehensively explain the technology stack of the distributed system in two dimensions.
From the perspective of technical principles, high concurrency, and performance optimization, this article analyzes the problems caused by the complex and changeable application scenarios in the distributed environment from multiple technical levels, and gives high-performance optimization solutions and high-availability architectures. Program.
Through this article, readers can improve the overall stability of the application in a distributed environment in actual work.
The article incorporates an introduction to a variety of mainstream technology stacks in the distributed field, allowing readers to fully understand the knowledge of distributed system architecture.
Because the content of the article is too much, the editor only took out some of the knowledge points to introduce to everyone, each section has more detailed content! !
table of Contents
main content
This article constructs and arranges the content according to the order of interaction in the distributed environment, and it is recommended to read them in order of the catalog. The full text consists of 10 chapters, and the details are as follows.
Chapter 1 Introduction to Distributed Architecture; Distributed architecture is the application of distributed computing technology. Currently, more mature technologies include J2EE, CORBA, and .NET (DCOM). This article focuses on J2EE. J2EE is a middleware technology launched by Sun, which aims to simplify and standardize the development and deployment of multi-layer distributed enterprise application systems, and can provide a platform for distributed application software to share resources among various technologies. The implementation of the J2EE standard can significantly improve the portability, security, scalability, load balancing, and reusability of the system. It mainly has the following characteristics.
Has a distributed architecture. The distribution of J2EE components has nothing to do with the server environment, all resources can be accessed through the distributed directory, developers do not need to spend energy on the distribution of components and resources.
Adopt a multi-tier distributed application model. J2EE divides application development into multiple different layers and defines components on each layer. According to their layer, each application is distributed on the same or different servers to form a component-based multi-layer distributed system, including the client layer, the presentation logic layer, the business logic layer, and the enterprise information system layer.
Have standards for application servers. J2EE is the first middleware standard widely recognized and adopted by the industry.
With the continuous development of the mobile Internet, computer systems have transitioned from a single-machine state to a multi-machine cooperative state. Computers exist in a cluster, and huge and complex application services are constructed in accordance with the guidance of distributed theory.
The highlights of this chapter are as follows:
- Distributed architecture development process
- Distributed architecture design philosophy and goals
- Distributed architecture application scenarios
- Difficulties in distributed architecture design
- Distributed architecture solves pain points
Chapter 2 Distributed architecture front-end interaction; from the traditional interaction to the current more popular front-end interaction process, the front-end interaction under the distributed architecture has become more efficient and comprehensive, and too many technical levels have been iterated and Update.
The highlights of this chapter are as follows:
- Front-end interactive development process
- Front-end interaction
- Difficulties in front-end and back-end interaction
- Front-end and back-end interaction optimization
- Case explanation
Chapter 3 Distributed architecture network transmission; Distributed architecture network is formed by the interconnection of nodes distributed in different locations and with multiple terminals. Any node in the network is connected to at least two lines and other nodes. When any one of the lines fails, the communication can be transferred to other lines to complete, so that the network has high reliability and scalability. Network transmission is realized based on reliable lines and network transmission protocols.
The highlights of this chapter are as follows:
- Network transmission protocol
- Network transmission call process
- Network transmission optimization
Chapter 4 Distributed Architecture Nginx; Nginx is a high-performance HTTP and reverse proxy web server. Nginx can be used as an HTTP server for website publishing, or as a reverse proxy for load balancing. Because it occupies less memory and has strong concurrency, it can be widely used in the Internet.
The highlights of this chapter are as follows:
- How Nginx works
- Nginx source code compilation and installation
- Nginx configuration
- Nginx proxy & load balancing
- Nginx cache
- Nginx current limit
- Nginx shield
- Nginx optimization
- Nginx high availability
Chapter 5 Distributed Architecture Varnish; Varnish is a high-performance and open source reverse proxy server and HTTP accelerator, mainly through caching to achieve Web access acceleration. It caches based on memory, supports accurate cache time, and has high performance. Its VCL configuration management is more flexible, supports back-end server load and health checks, and implements load balancing to poll the server internally.
The highlights of this chapter are as follows:
- How Varnish works
- Varnish source code compilation and installation
- Varnish configuration
- Varnish core instructions
- Varnish cache
- Varnish processing strategy
- Varnish health check
- Varnish optimization
- Varnish high availability
Chapter 6 Distributed Architecture Tomcat; Tomcat is a free and open source Web application server, mainly used in small and medium-sized systems. It has Servlet and JSP specifications inside, and is relatively lightweight. It is loved and used by developers. It is currently more popular. In fact, part of the Tomcat application is an extension of the Apache server and runs as a separate process from Apache.
The highlights of this chapter are as follows:
- Tomcat principle
- Tomcat life cycle
- Tomcat source code compilation and installation
- Tomcat directory structure
- Tomcat loading process
- Tomcat security
- Tomcat cluster
- JVM
- Tomcat performance tuning
Chapter 7 Distributed architecture has high concurrency; a large number of requests may arrive at the server at the same time or in a very short time. At this time, each request requires the server to consume resources to process and make corresponding feedback. The number of threads that can run at the same time, the number of network connections, CPU operations, I/O, and memory are limited, so the number of requests that the server can handle at the same time is also limited. The essence of high concurrency is to solve the problem of limited resources.
Assuming that the number of online users in the system is 200,000, it does not mean that there are 200,000 concurrent users of the system. There may be 100,000 users viewing static articles on the homepage at the same time, and no request is sent to the server. Therefore, the high concurrent number is based on the real number of users of the system. , And require the server to consume resources for processing. If the server can only open 100 threads, and it takes 1s for exactly 1 thread to process 1 request, then the server can only process 100 requests within 1s, and the extra requests cannot be processed.
High concurrency involves related commonly used indicators such as throughput (TPS), query rate per second (QPS), response time, and the number of concurrent users.
1) TPS (QPS): The number of requests processed in a unit time, the calculation formula is the number of concurrent / average response time.
2) Response time: The time for the system to respond to the request. Generally, the average response time is taken.
3) Number of concurrent users: The number of users who carry the normal use of system functions at the same time. For example, in an instant chat system, the number of simultaneous online users to a certain extent represents the number of concurrent users of the system.
The highlights of this chapter are as follows:
- High concurrency usage scenarios
- High concurrency difficulty
- High concurrency cache
- High concurrency message queue high concurrency optimization
- High concurrency classic case
Chapter 8 Distributed Architecture Transactions; Mainly introduces the differences between ordinary transactions and distributed transactions, as well as the various processing methods of distributed transactions. This chapter will analyze multiple cases and combined code.
The active side application can asynchronously send messages to the passive side application to improve application processing efficiency. The message middleware can decouple the strong coupling between the active side and the passive side, and at the same time introduce the message middleware MQ. Because MQ has unique high concurrency and high throughput capabilities, it can buffer messages and flexibly adjust and control the efficiency of message processing, which greatly improves the concurrency of both systems.
The highlights of this chapter are as follows:
- Introduction to distributed transactions
- Introduction to Distributed Transactions
- Distributed transaction application scenarios
- Difficulties of Distributed Transaction
- Distributed transaction solution
- Distributed transaction case explanation
Chapter 9 Distributed Architecture MySQL; MySQL is a relational database developed by MySQL AB of Sweden and currently belongs to Oracle. MySQL has been widely used in Web applications due to open source reasons. At the same time, because of its small size, high speed, and low cost of use, especially the feature of open source, it is deeply loved by large, medium and small enterprises. MySQL uses standardized SQL. Language helps users better manage the database system.
The highlights of this chapter are as follows:
- How MySQL Works
- MySQL compile start
- MySQL transaction
- MySQL storage engine
- MySQL SQL operation
- MySQL index
- MySQL backup
- MySQL difficulties
- MySQL performance optimization
- MySQL cluster
Chapter 10 Distributed architecture is highly available; high availability means that the system is specifically designed to reduce downtime and maintain high availability of its services. Availability consists of two dimensions: the average time of computer service availability and the average time of computer service maintainability. The average time of computer service availability refers to how long the system can run normally before a failure occurs. The higher the availability of the system, the longer the mean time between failures. The average time of computer service maintainability refers to the average time it takes to process and resume normal operation after a system failure. The better the maintainability of the system, the shorter the average maintenance time. Therefore, the availability of a computer is defined as the proportion of the time the system maintains uptime. It can be seen that high availability is an important goal of the system.
The highlights of this chapter are as follows:
- High availability overview and difficulties
- Highly available content
- Highly available specific applications
- High availability case explanation
This "Distributed System Architecture: Technical Stack Detailed Explanation and Fast Advancement" document has a total of 478 pages. Because the content is too much, I will not introduce more here. If you need the full version, you can forward it. Follow the editor, scan the code to get it!
Who should read this article
This article is a technical article on distributed system architecture. The suitable reading objects are as follows.
- Readers who are interested in the technology stack used in the distributed, distributed full stack.
- Technical personnel who have a weak foundation and want to deepen their understanding of distribution by learning the concepts and design ideas in the distributed full stack.
- Technical personnel who have a strong foundation and want to deeply understand and flexibly use the design ideas and optimization schemes in the distributed full stack
I hope that you can read this article carefully, integrate all the knowledge points, and strive to be proficient, so that in the future job hunting or your own development process, you can reflect your own great value, and I hope this article can be liked by everyone! !