Java application general architecture historical evolution

When we set up a system, we usually need to consider how to interact with other systems, so we first need to know how various systems interact and what technology is used to implement them.

1. Interaction between different systems and different languages
​​Now different systems and different languages ​​uses WebService, Http request. WebService, or "Web Service", abbreviated as WS. Literally, it's actually a "Web-based service". The service is two sides, there is a service demander, there is a service provider. The service provider publishes the service, and the service demander invokes the service published by the service provider. If it is more professional, WS is actually a tool based on the HTTP protocol to realize the communication of heterogeneous systems. That's right! To put it bluntly, WS is still based on the HTTP protocol, that is, the data is transmitted through HTTP. At first, we used CXF to develop SOAP services to implement WS, and later we used REST services to implement WS (this is currently used more, and it is the one I use the most). REST services can also be developed based on CXF, but we generally use springMVC or other MVC frameworks to implement REST services directly.

But in the impression of many people, the word Web service generally refers to various XML-based interactive technologies dominated by IBM ten years ago, and now few people use it except for some companies. In a broad sense, Webservice is a Web service, and everything is a service.

2. Interaction between different systems in the same language The
common different systems in the same language is implemented by RPC (remote procedure call) or RMI (remote method invocation), without providing services to the outside, of course, the above can also be used The interaction between the same languages, but I usually use RPC.

Architecture of different products

3. Architecture evolution of a single product
Generally we only have one product's architecture evolution process. If we need to provide external webServices, we usually use REST services to implement them.

The following paragraph comes from Zhihu

1) Evolution of distributed architecture System architecture evolution process-



initial All resources such as small system applications, databases, and files in the initial stage are commonly called LAMP

features on one server: application All resources such as programs, databases, files, etc. are on one server.

Description: Usually the server operating system uses linux, the application is developed using PHP, and then deployed on Apache, the database uses Mysql, all kinds of free open source software and a cheap server can be used to start the development of the system.

2)



Evolution

. Applications, databases, and files are deployed on separate resources.

Description: The amount of data increases, and the performance and storage space of a single server are insufficient. It is necessary to separate applications and data, and the concurrent processing capability and data storage space have been greatly improved.

3) Evolution of system architecture - using cache to improve performance


Features : A small part of the data accessed in the database is stored in the cache server, which reduces the number of database accesses and reduces the access pressure of the database.

Description: The system access characteristics follow the 28th law, that is, 80% of the business access is concentrated on 20% of the data. The cache is divided into local cache and remote distributed cache. The local cache access speed is faster but the amount of cached data is limited, and there is memory contention with the application.

4) Evolution of system architecture - using application server clusters



After completing the work of sub-database and sub-table, the pressure on the database has been reduced to a relatively low level, and I began to live a happy life of watching the surge in traffic every day. Suddenly, one day, I found that the access to the system began to change again. The trend is slow. At this time, first check the database, the pressure is normal, and then check the webserver, and find that apache blocks a lot of requests, and the application server is relatively fast for each request, it seems that the number of requests is too high, which leads to the need to wait in line , the response speed becomes slower

Features : Multiple servers provide services to the outside at the same time through load balancing, which solves the problem of the processing capacity and storage space limit of a single server.

Description: Using clusters is a common method for systems to solve high concurrency and massive data problems. By adding resources to the cluster, the concurrent processing capability of the system is improved, so that the load pressure of the server is no longer the bottleneck of the entire system.

5) Evolution process of system architecture - separation of database read and write After



enjoying the happiness of the rapid growth of system access for a period of time, I found that the system began to slow down again. What is the situation this time? After searching, I found that the database was written and updated. The resource competition of some database connections in these operations is very intense, which leads to the slowdown of the system

. Features : Multiple servers provide services to the outside through load balancing, which solves the problem of the processing capacity and storage space limit of a single server.

Description: Using clusters is a common method for systems to solve high concurrency and massive data problems. By adding resources to the cluster, the load pressure of the server is no longer the bottleneck of the entire system.

6) Evolution of system architecture - reverse proxy and CDN acceleration



Features : CDN and reverse proxy are used to speed up system access.

Description: In order to cope with the complex network environment and the access of users in different regions, the speed of user access is accelerated through CDN and reverse proxy, and the load pressure of the back-end server is reduced at the same time. The basic principle of CDN and reverse proxy is cache.

7) Evolution of system architecture - distributed file system and distributed database



With the continuous operation of the system, the amount of data begins to increase significantly. At this time, it is found that the query after the database is still a little slow, so according to the idea of ​​​​the database, the work of the table is started.

Features : The database adopts a distributed database, and the file system adopts a distributed database. format file system.

Description: Any powerful single server cannot meet the business needs of the continuous growth of large-scale systems. The separation of database read and write will eventually fail to meet the needs with the development of the business. It needs to be supported by a distributed database and a distributed file system. Distributed database is the last method of system database splitting. It is only used when the scale of single-table data is very large. The more commonly used method of database splitting is business sub-database, which deploys different business databases on different physical servers.

8) Evolution of system architecture - using NoSQL and search engines



Features : The system introduces NoSQL databases and search engines.

Description: As the business becomes more and more complex, the requirements for data storage and retrieval become more and more complex, and the system needs to use some non-relational databases such as NoSQL and sub-database query technologies such as search engines. The application server accesses various data through the unified data access module, which relieves the trouble of managing many data sources by the application program.

9) Evolution of system architecture - business split



Features : The system is split and transformed according to business, and the application servers are deployed separately according to business.

Description: In order to cope with increasingly complex business scenarios, the whole system business is usually divided into different product lines by means of divide and conquer. The relationship between applications is established through hyperlinks, and data can also be distributed through message queues. The same data storage system to form an associated complete system. Vertical splitting: Splitting a large application into multiple small applications. If the new business is relatively independent, it is relatively simple to directly design and deploy it as an independent Web application system. Vertical splitting is relatively simple. The related business can be divested. Horizontal splitting: Split the reused services and deploy them independently as distributed services. New services only need to call these distributed services. Horizontal splitting requires identifying reusable services, designing service interfaces, and standardizing service dependencies.

10) Evolution of system architecture - distributed service



Q: What problems will distributed service applications face?

(1) When there are more and more services, the management of service URL configuration becomes very difficult, and the single point pressure of F5 hardware load balancer is also increasing.
(2) With further development, the dependencies between services become complicated and misunderstood, and it is not even clear which application should be started before which application, and architects cannot fully describe the architectural relationship of the application.
(3) Then, the call volume of the service is increasing, and the capacity problem of the service is exposed. How many machines does this service need? When should the machine be added?
(4) With more services, communication costs have begun to rise. Who should be contacted if a service fails to be adjusted? What are the conventions for the parameters of the service?
(5) A service has multiple business consumers, how to ensure service quality?
(6) With the continuous upgrade of services, some unexpected things always happen, such as memory overflow caused by wrong cache writing, failure is inevitable, every time the core service hangs, affecting a large area, people panic, how to control the failure influence? Can the service be functionally downgraded? Or resource degradation?
This seems to be the content of the core principles and case analysis of the large-scale website technical architecture, but the author summed it up well, so I will reprint it.

4. Product Line Architecture
Another is the business split mentioned above. Now we need to make a product line. We only need a data layer, a general business logic layer, and various application and interface layers in front of it. In the past, we usually do not need to provide services to external systems (systems of external companies). Choose to use EJB to build distributed applications, but now we can use RPC frameworks such as dobbo, thrift, avro, hessian to build distributed applications to achieve interaction between different applications and data sources. In this structural mode, we need to provide services to other companies, and we can write an application to provide rest services to external systems. Generally, most Internet services need to access dozens or even hundreds of internal services, and the communication between them is generally RPC: just like accessing a remote method, input parameters and wait for the return result. This is the easiest way to understand for building complex systems.

As shown in the figure below, the model, file system, and cache are not drawn, so everyone can understand it.

Click to view the original size image

Conclusion :

No matter what kind of architecture we need to do a good job of modularization (as far as possible to achieve module reuse).
Don't overdesign for architecture's sake.
No matter what kind of architecture is to better meet the needs of the business, the architecture should develop with the development of the business.
If the current architecture can meet the current business development, you can consider the next expansion, instead of considering 3 steps, 4 steps or even more at once.