The original text is quoted from: http://kb.cnblogs.com/page/505538/
In the last article , I said that SOA is a very big topic, not only there is no absolutely unified principle, but also the content of many principles themselves are quite vague and broad. Although we can say that SOA ≈ modular development + distributed computing, due to the ambiguity of its principles, it is still difficult for us to say that an application is absolutely SOA-compliant, and we can only identify which are not SOA-compliant.
This article will conduct a detailed analysis of 8 operational service design principles as a reference for SOA practice.
Service Design Principle 1: Optimizing Remote Calls
The remote call here refers specifically to RPC (Remote Procedure Call). Of course, the more object-oriented statement should be remote method invocation or remote service invocation and so on.
Since SO interfaces are usually accessed remotely, and the overhead of network transmission, object serialization/deserialization, etc. is far more than that of local Object access by several orders of magnitude, to speed up the response speed of the system, reduce bandwidth usage and improve throughput, choose High-performance remote invocation is often important.
However, remote invocation methods are often limited by specific business and deployment environments, such as intranet, extranet, homogeneous platform, heterogeneous platform, and so on. Sometimes also consider how well it supports aspects such as distributed transactions, message-level signing/encryption, reliable asynchronous transport, etc. (these aspects are often referred to as SLA: service level agreement), and even developer familiarity and acceptance, etc. Wait.
Therefore, remote invocation methods often need to make choices and trade-offs according to specific situations.
Take Java Remote Service as an example to analyze some of the transmission methods that may be better choices in different scenarios:
- Intranet + same-framework Java client + large concurrency: multiplexed TCP long connection + kryo (binary serialization) (kryo can also be replaced by Protostuff, FST, etc.)
- Intranet + Java clients of different frameworks: TCP + Kryo
- Intranet + Java client + 2PC distributed transaction: RMI/IIOP (TCP + binary)
- Intranet + Java client + reliable asynchronous call: JMS + Kryo (TCP + binary)
- Intranet + clients in different languages: thrift (TCP + binary serialization)
- Extranet + Clients in Different Languages + Enterprise Features: HTTP + WSDL + SOAP (Text)
- External network + take into account browsers, mobile phones and other clients: HTTP + JSON (text)
- External network + clients in different languages + high performance: HTTP + ProtocolBuffer (binary)
In short, in terms of performance, tcp protocol + binary serialization is more suitable for intranet applications. In terms of compatibility and simplicity, http protocol + text serialization is more suitable for external network applications. Of course this is not absolute. In addition, the tcp protocol here does not limit the remote call protocol to only the original tcp located at the fourth layer of the OSI network model, and it can include any non-http protocol above tcp.
Therefore, to answer the questions mentioned above, although WebServices (classic WSDL+SOAP+HTTP) is the technology that best conforms to the aforementioned SOA design principles, it is not equivalent to SOA. I think it only satisfies the bottom line of SOA, not necessarily The best choice for a specific situation. This is like a decathlete who is difficult to compete with the individual champion in each individual event. A more ideal SOA Service should preferably support multiple remote invocation methods and adapt to different scenarios while supporting WebServices. This is also the design principle of distributed service frameworks such as Spring Remoting, SCA, Dubbo, and Finagle.
Remote invocation technology explained: Is HTTP + JSON suitable for SOA?
JSON is simple and easy to read, has excellent versatility, and even supports browser clients well. It is also often used by mobile APPs, and has the potential to replace XML.
However, JSON itself lacks a widely accepted standard schema like XML, and the general remote invocation method of HTTP + JSON also lacks standard IDL (Interface Definition Language) like Thrift, CORBA, WebServices, etc. It cannot form a strong service contract between them, and it cannot do things such as automatic code generation. Therefore, HTTP + JSON may significantly increase the development workload and error probability of complex applications while lowering the learning threshold.
For example, Sina Weibo provides an Open API based on HTTP + JSON, but due to the complex business operations, client class libraries in various languages are encapsulated and implemented on JSON to reduce the workload of users.
In order to solve this problem, there are many different solutions in the industry to add IDL to HTTP + JSON, such as RSDL, JSON-WSP, WADL, WSDL 2.0, etc., but in fact, their acceptance is not ideal.
It is also worth mentioning that the JSON format is as redundant as XML. Even with optimizations such as GZIP compression, the transmission efficiency is usually not as good as that of many binary formats, and compression and decompression will also introduce additional performance overhead.
Remote invocation technology explained: Apache Thrift multilingual service framework
Thrift is a set of cross-language service development framework originally from facebook, supporting almost all mainstream programming languages such as C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, JavaScript, Node.js, Smalltalk, Delphi, etc. Has excellent versatility.
Thrift is widely used by giants such as facebook, twitter and open source communities, and is a very mature technology.
Thrift's service contract is defined by an IDL of the form:
struct User { 1: i32 id, 2: string name, 3: string password } service UserService { void store(1: User user), UserProfile retrieve(1: i32 id) }
Very similar to C language, easy to read and write, much simpler and clearer than WSDL. It is also more convenient than using programming languages such as java. Sometimes you can put all the relevant interface and data structure definitions into the same file, and you don’t need to type a compressed package when you publish it, or you can even paste it directly into the document.
Thrift also provides tools that can automatically generate server and client code corresponding to various languages based on IDL:
[lishen@dangdang thrift]thrift --gen java user.thrift [lishen@dangdang thrift]$ thrift --gen cpp user.thrift [lishen@dangdang thrift]$ thrift --gen php user.thrift [lishen@dangdang thrift]$ thrift --gen csharp user.thrift
I think thrift is a simpler and more efficient technology than WebServices, and is one of the most alternative technologies to WebServices in SOA.
Explanation of Remote Calling Technology: Multiplexed TCP Long Connection
This is a way to pursue extreme high performance and high scalability, which is only briefly introduced here.
The typical ones are twitter's Mux RPC protocol and google's SPDY protocol, in which multiple requests share the same long connection at the same time, that is, a connection alternately transmits byte blocks of different requests. It not only avoids the overhead of repeatedly establishing connections, but also avoids waiting for idle connections to reduce the total number of system connections, and also avoids the head-of-line blocking problem in TCP sequential transmission.
In addition, the default RPC protocol of the well-known open source dubbo framework in China, as well as many small open source RPC frameworks in the industry are also similar.
After the multiplexing mechanism is adopted, both the server and the client are generally required to support additional semantics similar to the session layer (ie, the sixth layer of the OSI network model), so they must rely on the same RPC framework.
Many other RPC mechanisms use short TCP connections. Even if some RPCs use long connections, a connection can only send one request at a time, and then the connection is in an idle state, waiting to receive a response to the request. After the response is complete, the connection can be released or reused.
HTTP 1.1 also supports a long connection based on the pipeline mode, in which multiple HTTP requests can also share a connection, but it requires that the response must also be transmitted and returned in the order of the request, that is, FIFO first in, first out. In a fully multiplexed connection, whichever response is ready first can be transmitted first, without queuing.
Of course, there is no absolute good or bad between short connections, long connections, and multiplexed long connections, which depend on specific business and technical scenarios, and are not detailed here.
Remote invocation technology explained: Java efficient serialization
In recent years, various new Java efficient serialization methods have emerged one after another, constantly refreshing the upper limit of serialization performance, such as Kryo, FST and other open source frameworks. They provide a very efficient implementation of serialization and deserialization of Java objects. Compared with the JDK standard serialization method (that is, the standard serialization based on the Serializable interface, the custom serialization such as the Externalizable interface is not considered for the time being), in typical In scenarios, the serialization time overhead may be reduced by more than 20 times, and the size of the generated binary bytecode may be reduced by more than 4 times.
In addition, the overhead of these efficient Java serialization methods is significantly less than cross-language serialization methods such as thrift's binary serialization, or JSON, etc.
Remote Invocation Technology Explained: RMI/IIOP and Distributed Transactions
RMI/IIOP is a standard remote invocation method in Java EE. IIOP is a CORBA protocol. Only RMI on IIOP supports distributed transactions committed in two phases and provides interoperability with CORBA.
Of course, strict two-phase commit transactions are not efficient, and may seriously affect system scalability and even availability, etc., and are generally only used in very critical businesses.
Remote call technology explanation: Google ProtocolBuffer cross-language serialization
ProtocolBuffer is a cross-language efficient binary serialization method developed by Google. Its serialization performance is similar to thrift. In fact, thrift was originally an imitation of ProtocolBuffer. But the biggest difference between it and thrift is that it does not have its own RPC implementation (because Google has not open sourced the RPC part, but there are a large number of third-party implementations).
Because it is not coupled with the RPC method, ProtocolBuffer can be easily integrated into a large number of existing systems and frameworks. In China, it is also widely used in Open API by Baidu, Taobao, etc., and it is used with HTTP as an efficient cross-platform and cross-organization integration method.
Service Design Principle 2: Eliminate Redundant Data
Also, due to the high overhead of remote invocation of the service, in its input parameters and return results, try to avoid carrying redundant fields that are not required by the current business use case to reduce the overhead of serialization and transmission. At the same time, removing redundant fields can also simplify the interface and avoid unnecessary business confusion for external users.
For example, there is a method in the article service that returns the article list
List<Article> getArticles(...)
If the business requirement is only to list the title of the article, then it is necessary to avoid carrying its contents and other fields in the returned article.
The classic solution here is to introduce the Data Transfer Object (DTO) mode commonly used in OO to customize the data fields to be transferred for specific service use cases. Here's an extra data transfer object that adds an AriticleSummary:
List<ArticleSummary> getArticleSummaries(...)
The extra DTOs are a real hassle, and normal OO programs can often just return their own business model with redundancy.
Service Design Principle 3: Coarse-Grained Contracts
Also, due to the high overhead of remote invocation and the fact that external users of the service cannot understand specific business processes as well as those inside the organization, the contract (interface) of the service usually needs to be coarse-grained, and one of the operations may correspond to A complete business use case or business process, which not only reduces the number of remote calls, but also reduces learning costs and coupling.
Whereas OO interfaces can often be very fine-grained, providing the best flexibility and reusability.
For example, the article service supports batch deletion of articles, which can be provided in the OO interface
deleteArticle(long id)
For users to make loop calls by themselves (not considering optimizations such as back-end SQL), but in the SO interface, it is best to provide
deleteArticles(Set<Long> ids)
For client calls, reducing the possible N remote calls to one.
For example, the use case of placing an order, there is a series of actions
addItem -> addTax -> calculateTotalPrice -> placeOrder
In OO, we can completely allow users to choose flexibly and call these fine-grained reusable methods separately. But in SO, we need to encapsulate them into a coarse-grained method for users to make one-time remote calls, and also hide a lot of complexity of internal business. In addition, the client also changed from relying on 4 methods to relying on 1 method, thus greatly reducing the degree of program coupling.
By the way, if each operation in the above order use case is itself a remote service (usually in the intranet), this coarse-grained encapsulation becomes a classic service composition (service composition) or even service orchestration ( service orchestration). In this case, the coarse-grained service may also improve performance, because for external network customers, multiple remote calls across the network become one cross-network call + multiple intranet calls.
For this kind of coarse-grained service encapsulation and composition, the classic solution is to introduce the Facade pattern commonly used in OO to shield the original object behind a special "facade" interface. At the same time, it is also very likely that we are required to introduce a new data structure of service parameters/return values to combine the original object models of multiple operations, which also uses the aforementioned DTO pattern.
A simple facade example (FooService and BarService are two hypothetical native OO services, and the facade returns their result values combined):
class FooBarFacadeImpl implements FooBarFacade { private FooService fooService; private BarService barService; public FooBarDto getFooBar() { FooBarDto fb = new FooBarDto(); fb.setFoo(fooService.getFoo()); fb.setBar(barService.getBar()); return fb; } }
Of course, sometimes instead of facade and DTO, another local service and domain model can be added in addition to FooService and BarService, which depends on the specific business scenario.
Service Design Principle 4: Universal Contracts
Since the service does not assume the scope of the user, it generally supports clients in different languages and platforms. However, various languages and platforms have great differences in feature richness, which determines that service contracts must take the greatest common divisor of common languages, platforms, and serialization methods to ensure wide service compatibility. As a result, the service contract cannot have advanced features that are only available in some languages, and the parameters and return values must be simpler data types that are widely supported (for example, no object circular references).
If the original OO interface cannot meet the above requirements, we also need the above Facade and DTO to convert the OO interface into a general SO contract.
For example, the original object model
class Foo { private Pattern regex; }
Pattern is a Java-specific precompiled, serializable regular expression (which can improve performance), but without specific framework support, it may not be directly recognized by other languages, so DTO can be added:
class FooDto { private String regex; }
Service Design Principle 5: Isolate Change
Although both OO and SO pursue low coupling, SO requires a higher degree of low coupling due to the wide range of users.
For example, in the aforementioned article service, the article object can be directly returned in OO, and this article object may be used as the core modeling domain model in the OO program, or even as the O/R mapping and so on. In SO, if this article is directly returned, even if there are no redundant fields, complex types and other problems mentioned above, it may make the core object model of external users and internal systems, even O/R mapping mechanism, data table structure, etc. A certain degree of relevance is generated, so that internal refactorings often may affect external users.
Therefore, there is a need for Facade and DTO again here. They are used as intermediaries and buffers to isolate internal and external systems and minimize the impact of internal system changes on the outside.
Service Design Principle 6: Contract First
Service often involves cooperation between different organizations, and according to normal logic, the primary task of cooperation between two organizations is to sign a clear contract first, specifying in detail the content of cooperation between the two parties, the form of cooperation, etc. Form strong constraints and guarantees, and at the same time everyone can work in parallel without waiting for each other. Therefore, in SOA, the best practice method is contract first, that is, contract design is done first, and personnel from different aspects such as business, management and technology can participate together, and the corresponding WSDL or IDL can be defined. The corresponding code of the target language is automatically generated by the tool.
For WSDL, the threshold for making contracts first is slightly higher, and it is difficult to manually compile without good XML tools. But for Thrift IDL or ProtocolBuffer, etc., because they are similar to ordinary programming languages, contract design is relatively easy. In addition, for simple HTTP + JSON (assuming that other description languages are not used), since JSON does not have a standard schema, it is impossible to design a contract with strong binding force. It can only be described by another document or used Example of JSON as input and output.
However, the contract is first, and then the code of the service provider is generated. After all, it brings great inconvenience to the service development work, especially when the contract is modified, the code needs to be rewritten. Therefore, Facade and DTO may also need to be introduced here, that is, Facade and DTO codes are generated by contracts, and they are responsible for adapting and forwarding requests to other internal programs, while internal programs can maintain their own dominance and stability.
Also, contract look-ahead can cause some trouble with the aforementioned multiple remote call support.
Of course, contract first may not be a widely accepted practice, just as "test first" (ie test-driven development) in agile development is usually a best practice, but very few teams actually implement it. Need to constantly explore and summarize. But we can at least argue that Java2WSDL in Echo is not considered SOA best practice.
Service Design Principle 7: Stable and Compatible Contracts
Due to the wide range of users, SO's service contract is similar to the Java standard API. After public release, it must ensure considerable stability and cannot be refactored casually. Even if it is upgraded, it must consider the downward compatibility as much as possible. At the same time, if the contract-first approach is used, frequent changes to the contract in the future will also cause developers to constantly redo the contract-to-target language mapping, which is very troublesome.
This means that SO's quality requirements for contracts may be much higher than those of general OO interfaces. Ideally, special personnel (including business personnel) may even be required to design and evaluate SO contracts (regardless of whether the contract-first approach is used), while the The internal program implementation is handed over to different people, and the two use Facade and DTO as a bridge.
Service Design Principle 8: Contract Packaging
The aforementioned principles are basically aimed at the service provider, while for the service consumer, the corresponding client code generated through the contract can often be used directly. Of course, if the contract itself is a Java interface (such as in Dubbo, Spring Remoting, etc.), you can skip the code generation step.
However, the service's return value (DTO) and service interface (Facade) may be referenced everywhere by the consumer program.
In this way, the consumer-side program is strongly coupled to the service contract. If the service contract is not defined by the consumer-side, the consumer-side is equivalent to giving up part of the dominance of its own program to others.
Once the contract is changed, or the consumer has to choose a completely different service provider (with a different contract), or even change the local program to implement the relevant functions, the modification workload may be very large.
In addition, the client code generated by the contract is often related to a specific transport method (such as webservices stub), which will also bring obstacles to switching remote invocation methods.
Therefore, just like in normal applications, we want to wrap data access logic (DAO or Repository pattern in OO), or wrap basic service access logic (Gateway pattern in OO), in an ideal SOA design, we also You can consider wrapping the remote service access logic. Since there is no proper name, it is temporarily called the Delegate Service mode. It is led by the consumer to define the interface and parameter types, and forward the call to the real service client to generate code, so as to generate code for it. The service contract is completely shielded by the users of , and these users don't even know whether the service is provided remotely or locally.
In addition, even if we use some manual calling mechanisms on the consumer side (such as directly constructing and parsing json, etc., directly sending and receiving JMS messages, etc.), we can also use the delegate service to wrap the corresponding logic.
delegate service example 1:
// ArticlesService is a consumer-defined interface class ArticleServiceDelegate implements ArticlesService { // Assume that it is an automatically generated service client stub class private ArticleFacadeStub stub; public void deleteArticles(List<Long> ids) { stub.deleteArticles(ids); } }
delegate service example 2:
// ArticlesService is a consumer-defined interface class ArticleServiceDelegate implements ArticlesService { public void deleteArticles(List<Long> ids) { // Manually call remote service with JMS and FastJson messageClient.sendMessage(queue, JSON.toJSONString(ids)); } }
From object-oriented to service-oriented, and from service-oriented to object-oriented
Summarizing the above principles, although only a few limited aspects are mentioned, it can be seen that there are still many significant differences between OO and SO in actual design and development, and we do not intend to use the principles of SO. To replace the OO design of the past, but to introduce additional layers, objects and OO design patterns to complement the traditional OO design.
In fact, this kind of calling process is formed:
-
Service provider: OO program <- SOA layer (Facade and DTO) <- remote consumer
-
Service consumer: OO program -> Delegate Service -> SOA layer (Facade and DTO or other dynamic calling mechanisms) -> remote provider
Facade, DTO and Delegate Service are responsible for the intermediate conversion from OO to SO and SO to OO.
Now, the question in the Echo example can be answered: Publishing an OO program as a remote service through a "transparent" configuration method may well accomplish the transition from a local object to a remote object, but it is usually not well done. A real leap from OO to SO.
At the same time, the transparent configuration method usually cannot directly help legacy applications (such as ERP, etc.) to move to SOA.
Of course, in relatively simple and limited applications (such as traditional and partial RPC), transparent remote service publishing brings great convenience.
In addition, all the above discussions on SO focus on the RPC method. In fact, SO also uses the message method for integration. It is also a big topic and will not be discussed in detail here for the time being.
Why can't we give up object orientation?
SO has its specific scenarios, such as remote and indeterminate clients. Therefore, its design principles cannot be borrowed to guide general program development. For example, many OO programs are completely opposite to SO principles, often providing fine-grained interfaces and complex parameter types in pursuit of flexibility in use and powerful functions. sex.
In terms of specific architecture, I think the SOA layer should be a thin layer that wraps and adapts OO applications or other legacy applications to help them be service-oriented. In fact, in normal web development, we also wrap OO applications with a thin presentation layer (or Web UI layer, etc.) to help them face browser users. Therefore, Façade, DTO, etc. will not replace the core Domain Model, Service, etc. in OO applications (the service here is the service in OO, not necessarily SO).
Taken together, an architecture similar to the following is formed:
ideal and reality
It should be pointed out that many of the SO design principles mentioned above are in pursuit of a relatively ideal design to achieve architectural elegance, efficiency, reusability, maintainability, scalability, and so on.
In reality, any theory and principle may require appropriate compromise, because reality is vastly different, and its situation is far more complicated than theory, and it is difficult to have a universal truth.
And it seems that there is no need to pursue perfection and extremes in many aspects. For example, if there is enough capacity to expand the hardware infrastructure, you can consider transmitting some redundant data, choose the simplest transmission method, and make several more remote calls, etc., to reduce Design and development workload.
So is the idealized principle meaningless? For example, Domain-Driven Design is widely considered to be the most ideal OO design method, but very few projects can fully adopt it; Test-driven development is also considered to be the best agile development method, but there are also very few projects. The team can fully adopt it. However, I am afraid that not many people will deny their enormous significance after learning about them.
Idealized principles can better help people understand the nature of certain types of problems, and serve as a good starting point or benchmark to help those who can use them flexibly and make appropriate trade-offs achieve greater results and meet key challenges. This is just like what Confucius said, "If you take it from the top, you will get it; if you take it from the bottom, you will get it from the bottom; if you take it from the bottom, you will get nothing."
In addition, it is worth mentioning that SOA has some idealistic tendencies from its concept itself, such as opening to the "world", not limiting clients and so on. If you are really willing to follow the SOA path, even if you are a local tyrant, being lazy is more important than wasting network bandwidth, but maybe many of your users are local companies, and wasting several times the bandwidth will greatly affect their profit margins.
Extended discussion: Are SOA and agile software development contradictory?
SOA's service contract requires considerable stability. Once it is publicly released (or agreed upon by both parties), there should be no frequent changes. It requires a high degree of anticipation in many aspects. Agile software development, on the other hand, embraces change and continuously refactors. Software design guru Martin Fowler attributes them to the difference between planned design and evolutionary design.
Planning theory (or construction theory) and evolution theory are two trends of thought in modern philosophy, which have far-reaching influence and have derived various theories such as planned economy and market economy, socialism and liberalism.
However, planned design and evolutionary design are not absolutely contradictory, just like planned economy and market economy are not absolutely contradictory, either, this aspect needs to be constantly explored in practice. The design principles and architecture system we discussed earlier are to relatively isolate the SOA layer and OO applications, and divide and conquer. More planned designs are required at the SOA layer, while OO applications can evolve relatively independently, thereby easing SOA and agile development to a certain extent. the contradiction.
Extended discussion: Are SOA and REST the same thing?
In the most essential sense, REST (Representational State Transfer) is actually a resource-oriented architecture (ROA), which is fundamentally different from service-oriented architecture (SOA).
For example, REST is based on the HTTP protocol to perform operations such as adding (HTTP POST), deleting (HTTP DELETE), changing (HTTP PUT), and querying (HTTP GET) to specific resources, similar to the INSERT, DELETE, UPDATE and SELECT operations, so REST is centered on resources (resources can be compared to data). The service in SOA usually does not include such fine-grained operations for resources (data), but coarse-grained operations for business use cases and business processes, so SOA is centered on business logic.
However, in actual use, with the continuous breakthrough of many basic principles of REST, the concept of REST has been greatly generalized, and it often becomes synonymous with many HTTP-based lightweight remote calls (such as the aforementioned HTTP + JSON). ). For example, even the well-known Twitter REST API violates many of the basic principles of original REST.
In this generalized sense, REST can also be said to be a lightweight remote invocation method that helps to implement SOA.
Evolution of SOA Architecture
All the SOA issues discussed above are basically focused on the design and development of the service itself. However, in order for SOA to truly play its greatest role, it still needs to evolve into a larger architecture (that is, transition from micro SOA to macro SOA), which is briefly explained here:
-
The first level is the service architecture : to develop various independent services and meet some of the previous design principles, we have basically focused on this architecture. These independent services are a bit like children's building blocks.
-
The second level is the service composition (composition) architecture : independent services are combined to form new business or new services. Ideally, you can use a way similar to children building blocks, give full play to your imagination, flexibly assemble independent blocks (service) into new forms, and freely replace one of the components. This reflects the highly convenient reusability of SOA and greatly improves the business agility of the enterprise.
-
The third level is the service inventory (list) architecture : unified organization and planning of the reuse and combination of services through a standardized enterprise service inventory (or registry). When there are more and more building blocks, if they are still scattered all over the floor without good sorting and sorting, it is obviously impossible to play.
-
The fourth level is the service-oriented enterprise architecture ...
Summarize
So far, we have only briefly discussed SOA at the micro level, especially some basic design principles and their practices, in order to show the essence of SOA in practice, so as to help SOA better land and enter the daily operation level.
Finally, let’s make an analogy: SOA provides services regardless of the status (regardless of language, platform, organization), and provides services far away (through remote calls), which requires a spirit of serving the people wholeheartedly…