Little Hub reads:
This is a video recorded by Little Hub. I haven't posted it here yet. If you want to watch it, you can check it out. Hope it helps you.
More than half of the front-end and back-end separation blog project has been recorded, and it's a little bit close. I should be able to meet you tomorrow and the day after tomorrow.
Follow my B station: MarkerHub, learn technology together.
Website development history
It is really a good way to learn java with mind map! Today, we use a map to sort out the technologies and knowledge systems involved in the process of a website's gradual increase in traffic from 0 to 1.
How does a website's architecture evolve when there are more and more users and higher and higher concurrency? Next, we will use nine stages to analyze the technical points involved in the development process of each stage one by one.
Getting started with java
First of all, no matter what, to build a website, you must have basic knowledge of Java. In the Java foundation, the characteristics and usage scenarios of arrays, linked lists, queues, and stacks in the data structure should be familiar. Threading and reflection are also relatively basic things, which are often asked in interviews. In the design pattern, I don’t think we need to study very specifically. Many are easy to forget. Many times our business mainly learns to encapsulate, so the code management is not simple. Common patterns: singleton, factory, agent, observation Users, templates, and strategies can be familiar with first. For web knowledge, the four layers of http/tcp need to be understood. For servlet, filter, and listener are all web foundations, the relationship between session and cookie must be understood.
Framework basis
Spring and mybatis are essential framework foundations for java programmers. Spring's ioc and aop are the core concepts. You must be familiar with the bean container and use aop custom annotations. springmvc is the most widely used mvc framework. Corresponding to the relationship and process of the major components of mvc must be familiar with, as well as configuring multiple data sources.
SpringMVC process
-
1. The user sends a request to the front controller DispatcherServlet.
-
2. DispatcherServlet receives the request to call the HandlerMapping processor mapper.
-
3. The processor mapper finds the specific processor (you can search based on the xml configuration and annotations), generates the processor object and the processor interceptor (generate if there is one) and returns it to the DispatcherServlet.
-
4. DispatcherServlet calls the HandlerAdapter processor adapter.
-
5. HandlerAdapter calls a specific processor (Controller, also called back-end controller) through adaptation.
-
6. The Controller returns to ModelAndView after the execution is complete.
-
7, HandlerAdapter returns the controller execution result ModelAndView to DispatcherServlet.
-
8. DispatcherServlet passes ModelAndView to ViewReslover view resolver.
-
9. After ViewReslover parses, it returns to the specific View.
-
10. DispatcherServlet renders the view according to the View (that is, fills the model data into the view).
-
11. DispatcherServlet responds to users.
Mybatis is now the mainstream data layer framework. Of course, spring data jpa is also on the rise. The source code of mybatis is relatively simple, so you can read it through.
Springboot is even more popular nowadays. Spring gives us a complete set of industry solutions, many of which are based on springboot, such as springcloud. It is especially convenient to integrate. For springboot, we need to understand and be familiar with how to assemble third-party modules, how to customize a module starter, how to auto-start loading, etc.
Phase 1: Single project
The first stage is to build a single project. This stage should also be considered as a Java foundation, many of which are common framework combinations for current enterprises, such as spring+mybatis, interface document swagger2, entity verification hibernate validator, search engine lucene, and so on. Then deploy environment centos7, code management git, and continuous deployment git.
emmm~Students who are unfamiliar should learn more, basic things~
Phase 2: Physically separate application and database
Physically separate the application server from the database, so that the pressure on the cpu, memory, etc. is not so great.
Phase 3: Caching and static
There are some general template static foundations, such as template engines such as freemaker or velocity. Nowadays, many code generation frameworks actually use template static technology. After defining the template, pass the parameters to render, and it will generate what you want. Code.
Here nginx is suitable as our static file server, and the following dynamic separation is to use nginx.
Phase 4: Application load balancing, cluster
Nginx can be used as a reverse proxy server, so it can be used as our load balancing tool. There are also load balancing strategies, such as polling, weights, etc., which need to be understood. Generally speaking, nginx is the mainstream, and other apaches are used less. Nginx also has some online skills, such as blue-green deployment, gray-scale deployment and so on.
-
Blue-green deployment
A deployment method that can ensure that the system is online without interruption in providing services
-
Grayscale release
Refers to a publishing method that can smoothly transition between black and white
Load Balance means to allocate to multiple operation units for execution, such as Web servers, FTP servers, enterprise key application servers, and other mission-critical servers, so as to complete work tasks together.
The first problem that needs to be solved after application server load balancing is session synchronization (session). Commonly used frameworks are spring session, shiro-redis, etc. As for the selection of the cache, ehcache is generally not used, and redis is used more.
Of course, because nginx is used as a reverse proxy tool, all nginx needs to be configured for high availability, which is usually the keepalived+nginx combination.
Stage Five: Separation of Dynamic and Static
The often-said solution for separation of dynamics and statics is generally to deploy static files to file servers such as nginx to reduce the request pressure of the application server. There is also the concept of CDN. You should understand what the process is like. After all, it is often said now cdn, cdn, cdn, etc. If you don’t understand, you won’t be able to put it in your mouth~
Generally, we like to use cloud storage to store static things. We often use cloud storage such as Qiniu Cloud and Cos. Of course, nginx is also the main tool for dynamic and static separation, because some css, js and other files are generally placed under nginx.
Phase 6: Distributed cache, cache cluster
At this stage, the cache is mainly used to the extreme, reusing the advantages of redis's five data structures to adjust our data display. In addition, the designed cache problems such as cache penetration, cache avalanche, cache breakdown and other concepts should be familiar with. There is a solution. And the establishment of redis master-slave cluster, sentinel and other mechanisms to ensure the high availability of redis.
Of course, after using the cache, a more important issue was born, that is, how to ensure that the cache is consistent with the data in the database!
Stage 7: Separation of reading and writing, sub-database sub-table
In the seventh stage, the main focus is to separate reading and writing, and sub-database and table, which revolves around the mutual coordination of different databases.
Read and write separation
There are also many ways to separate read and write. Of course, the most common one is to use the inherited AbstractRoutingDataSource and override the determineTargetDataSource() method. Is there any other way? For everyone to popularize science, it can be launched for spring and mybatis:
spring
1. Distinguish different data sources by scanning packages
@MapperScan annotation scans different packages. All mappers under this package use the same data source, mainly the sqlSessionTemplateRef attribute. You can read this article: https://www.cnblogs.com/kangoroo/p/7133543.html
2. Distinguish through annotations
By inheriting AbstractRoutingDataSource, override the determineTargetDataSource() method to specify the data source. The most commonly used means must be understood! Example: renren-fast open source project
mybatis
1. Intercept the name of the method that initiated the operation
You need to agree to add, delete, modify and check the prefix yourself, and then select the data source based on the prefix!
2. Intercept the SQL that initiated the operation
Obtain sql by intercepting the execution method of mysql, and judge the reading and writing instance through sql: https://github.com/shawntime/shawn-rwdb
Sub-library and sub-table
mycat: Based on Proxy, it rewrites the MySQL protocol, disguising Mycat Server as a MySQL database
shareding-jdbc: An extension based on the JDBC interface that provides lightweight services in the form of a jar package
There are many problems that have been derived and dealt with, and many solutions have been born, such as distributed transactions, distributed locks, etc.
-
Distributed transaction CAP theorem
-
Consistency: The client knows that a series of operations will happen at the same time (effective)
-
Availability (Availability): Every operation must end with an predictable response
-
Partition tolerance: Even if a single component is unavailable, the operation can still be completed
-
Distributed transaction BASE theorem
-
Basically Available
-
Soft state
-
Eventually consistent
Unable to achieve strong consistency, but each application can be based on its own business characteristics
Use appropriate methods to achieve ultimate consistency of the system
-
Distributed lock
-
redis lock
-
redis + lua
-
redisson
-
-
zookeeper lock
-
-
Distributed unique primary key ID
-
UUID
-
redis auto-increment ID
-
Phase 8: Microservice architecture
The current rpc solution mainly has two systems
-
Ali system dubbo+zookeeper
-
spring system springcloud
The integration relationship between commonly used components in springcloud has solved the corresponding problems that need to be familiar with and learn to comprehensively use
-
Registration center Eureka
-
Client load balancing Ribbon
-
Declarative Rest calls Feign
-
Fault-tolerant processing Hystrix
-
Service Gateway Zuul
-
Unified management configuration Config
-
Service tracking Sleuth+zipkin
There are also a series of problems brought about by the service split, such as degradation, timeout retries, and current limiting that need to be resolved in order to achieve high availability. The message queue, cache, asynchronous concurrency and other technologies introduced for high concurrency all need to be understood.
Phase 9: NoSql and distributed search engine
One of the key points here is our distributed search engine. We talked about lucene before, and the bottom layer of es and solr are actually lucene. Therefore, we must be familiar with the underlying implementation principle of lucene (inverted index), as well as concepts such as word segmentation. At the same time, we can use three solutions for the data synchronization problem between es and the database
-
Synchronization based on cannal
-
Synchronization based on message queue
-
Synchronization based on logstash
At the same time, regardless of es or mongo, these nosql data stores will have the concept of fragmentation, and you need to understand the principles of high availability and backup.
Video explanation at station B
At present, I have recorded a video (about 150 minutes in total ), explaining the content involved in these nine stages, and doing some more detailed analysis. Interested students can go to station B to check it out and give me a thumbs up , Barrage, collection three in a row! Thanks for the support
Link: https://www.bilibili.com/video/av80313108
(Use app or pc to open it for HD!)
Concluding remarks
Well, today I have summarized the content roughly. All you need is a concept. There are many points involved in each stage. You can go into in-depth study for a certain point. I am Lu Yiming, thank you for reading, and welcome to come to station B to learn with Barrage!
------------------------------------------- < END >---------------------------------------
Lu Yiming MarkerHub May 26
https://mp.weixin.qq.com/s/M7FV8OnB51bFTuK_5Qa06w