"Internet Architecture" Software Architecture-The Evolution Process of Large-scale Distributed Websites

Projects are from a single application, to a distributed application, to a stream-based base stack, this idea.

Monolithic application

app application, db database, server service are all on the same machine

Cluster application

As the business volume increases, one server needs to be split into three servers.
The server service and app are on the same machine. Two applications and one database.

In the real development process, what kind of problems will happen when one application becomes multiple?

1. Session cluster problem
2. Data consistency problem
3. Data bottleneck (Once the traffic comes up, although the application is clustered, but the database is not clustered, it is still a master database), then you must consider the master-slave database.

N multiple templates have been operating the same database, how the database needs to be loaded, split the business, and different businesses access their own database. Reduce the pressure on the main library.
The characteristic of the Internet is to read more and write less.

If Double 11 is on and the transaction volume is large, in fact, the transaction read-write library will be under great pressure. The scheme adopted is: sub-library and sub-table: vertical sub-library, horizontal sub-table. The dedicated library of the module is a vertical sub-library. The sub-table is based on the key fields orderId and userId to store information in the specified table.

Strategies for the level score table (hash, range, list)

1. It is troublesome to estimate and expand the range and list
2. Hash hotspot data is scattered, evenly distributed, and expansion is also troublesome.

front end

1. If users use userId more
2. Data consistency requirements are relatively high
3. Check my order (userId check)
4. userId, time query condition

rear end

1. Internal data requirements are not too high
2. Complex business
3. There is a query by non-userId, the total number of orders placed on the day, and the number of orders placed.
4. Message middleware and es cluster (canal heterogeneous) are organized into what is needed by the back end.
5. Do not query the database, query through es
6. es Get the synchronization points of the database: delay, real-time, dimension, incremental, and full. In fact, it is to read the binlog log.
7. Big becomes smaller, keep splitting, keep merging
8. In the end, it may be a hole in the shape of a hole in the background.

Involving technical points

Java, sub-database and sub-table, redis cache, search engine, RPC remote call, all large-scale distributions involve concurrent programming.

PS: In the selection of technology, you must understand the business to know his solution.