ESB Project Requirements Analysis and Schematic Discussion (Copy and reproduce)

I found a very good article. In order to prevent bloggers from deleting the article in the future, I can't see it, so I copied it completely and saved it.

Original link
http://blog.itpub.net/14789789/viewspace-610864/

-------------------------------- -------------------------------------------------
Below Requirement analysis for




ESB The


requirement analysis phase is an important step in sorting out the relevant functional requirements and non-functional requirements in the project, and it is the key to the success or failure of the entire project. At this stage, we will start from the business needs of the enterprise, sort out the end-to-end cross-system business process; based on the business process, we will conduct service identification according to scientific methodology; starting from the service list, we will sort out the relationship between service consumption and provision; Best practice, define the interface of the service, including the schema description of the service, the type of fields, and the coding rules; according to the consumption-providing relationship of the service, sort out the service mapping and conversion rules and strategies in the ESB.
In general, we need to analyze the requirements of ESB from both functional and non-functional aspects.
For the functional requirements of ESB, we should focus on the following aspects:
1. Sort out the name and number of the systems to be integrated.
2. For each system, it is necessary to understand: whether
the external interface of the system is called outbound (OutBound), called by others (Inbound), or both;
the real-time requirements of the interface, whether it is real-time or batch , or both?
Is the calling method of the interface synchronous or asynchronous, or both?
The operating system platform on which the application system runs.
The programming language of the application system itself? C/C++, Java.....
the existing interfaces of these systems, can they already provide external interfaces, what is the interface method, including what is the communication protocol of the interface, HTTP/MQ/Socket/others? What is the data format of the interface, XML/custom format/other industry standard format? What is the programming language of the interface, Java/C/C++? If the interface itself cannot be provided, what are the requirements or constraints for interface development?
In the case of these application background databases, can the databases be accessed directly?
When each application exchanges data with other applications, the source data format and destination data format, such as converting from text format to XML format?
Transaction characteristics: which processing should adopt two-phase submission; whether multiple messages are required to form a transaction; whether to guarantee the processing order between messages;
the situation of the adapter: for some special systems, whether there is an existing adapter; the adapter is one-way mode of
message communication: Send and Forget, Request/Reply or Pub/Sub?
For the non-functional requirements of ESB, we need to confirm:
1. The scalability and high availability requirements of the ESB platform, including HA and 2.
The performance requirements of the ESB platform, mainly including the frequency of data exchange between systems, the size of the data to be exchanged (the message size will directly affect the efficiency); the requirements for the ESB data throughput and response time at peak times, etc. ;
3. Which transactions should ensure high reliability of data transmission;
4. The manageability requirements of the ESB platform, such as service life cycle management, maintenance and management of the ESB platform; if the enterprise has established SOA management and control specifications, then It is necessary to comply with the constraints of the specifications, such as whether there are specified naming rules, whether the enterprise has enterprise-level data specifications and the specifications of the underlying communication protocols, etc.;
5. Security requirements: whether to use SSL transmission encryption, whether to encrypt/decrypt messages, etc.;
6. Requirements for error handling, logs, and operation monitoring of the platform itself.







ESB
scheme include:
ESB-related IT application environment analysis, defining the interface mode between ESB and related applications;
ESB architecture outline design, and defining architecture principles;
ESB-related product selection, including adapter selection with peripheral systems and ESB product selection;
ESB component model design, decomposing ESB related modules, to meet the SOA separation of concerns and other architectural principles;
ESB operation model design to meet the non-functional requirements of the platform;
ESB platform service flow design, involving routing, conversion and Mapping, etc.;
ESB synchronous, asynchronous or publish/subscribe mode design;
ESB platform access channel and data interface design, including XML/JMS, SOAP/HTTP, EDI/MQ, etc.;
ESB-related adapter design, including technology adapters or Self-developed adapter;
fault tolerance and retry mechanism design of ESB platform, including unified management of logs, etc.;
Figure 1 is an example of high-level architecture design using ESB integration:

Figure 1. ESB reference architecture
ESB
As shown in Figure 1, the ESB architecture design mainly considers aspects such as communication protocol access and conversion, data access and conversion, data processing flow, and service registration and management. Among them, communication protocol access and conversion refers to the support and conversion capabilities of various integrated application systems' communication protocols, such as HTTP, JMS, Socket, FTP, etc.; data access and conversion refers to various integrated applications. Support and conversion capabilities of data formats provided by the system, such as XML, SOAP, custom formats, and proprietary formats (SWIFT, EDI, HL7, etc.) that conform to certain industry standards; data processing processes refer to routing, format conversion, database reading Various processing of data such as writing; unified service registration and storage management refers to the registration, publication, and query of services, as well as the management and control of services during operation, and provides statistical analysis data of service operation status. Component model of ESB Figure
2. ESB component model ESB Figure 2 shows an example of ESB component model, which includes the main components and their functions as follows: 1. Message Broker Runtime component provides message routing, format conversion, message logging, etc. The runtime environment for the operation. The operating environment is provided by IBM Message Broker; 2. The Messaging Broker Instance component is a container for processing MQ-based message service requests. It runs on the Message Broker Runtime as a Broker instance. This instance provides a running container for MQ message business request processor, service log, service location and other functions; 3. The Web Service Broker Instance component is a container for processing Web Service-based business requests. It runs as a Broker instance in Message on Broker Runtime. This instance provides Web Service business request processing, service log, and service location functions.







4. The Event Broker Instance component is the container for processing Pub/Sub events inside the platform. It runs on the Message Broker Runtime as a Broker instance. The container provides the running environment of the Event Handler component, and distributes MQ/JMS-based events to the target queues of different platform components.
5. The Message Handler component handles business requests based on MQ messages, including message parsing, format conversion, service authentication and authentication, service routing, and service logging. The typical process of the MQ message processing by the Message Handler component is as follows:
first, parse the MQ message, analyze the parsed business request, and then use the Authentication and Authorization components to determine whether the requester's business request can be processed later;
through the Service Locating component Perform service positioning and routing for the business request;
convert the MQ-based business request message into a Web Service business request message;
log the entire business request through the Service Logging component;
return the business request processing result to the business initiator, if it fails , which returns an error message.
6. The Web Service Handler component handles business requests based on Web Services. It has similar functions to the Message Handler component, and also includes functions such as message parsing, format conversion, service authentication and authentication, service routing, and service logging. A typical flow for a Web Service Handler component to process a Web Service request is as follows:
First, parse the Web Service request message, analyze the parsed business request, and then use the Authentication and Authorization components to determine whether the requester's business request can be processed later;
use the Service Locating component to service the service request. Locating and routing ;
Log the entire business request through the Service Logging component;
return the business request processing result to the business initiator, and return an error message if it fails.
7. The Event Handler component implements the processing of Pub/Sub.
8. The Service Locating component is responsible for locating specific service providers according to business requests. Service Locating selects suitable services for subsequent calls by querying the service catalog, and the query work can obtain results through real-time service catalog queries.
9. The Service Logging component is responsible for recording the situation in the entire business request processing process. The implementation of this component can be through a file or a database.
10. The Authentication component is responsible for authenticating the service requester and judging whether the service requester can access platform services. The authentication is performed outside the enterprise service bus, and the Authentication component just calls external functions to complete.
11. The Authorization component determines whether the business requester has the right to access a specific service. The verification of the right is performed outside the enterprise service bus, and the Authorization component just calls the external function to complete.
Taking the processing of MQ-based message input as an example, the component interaction diagram of ESB is shown in Figure 3:

Figure 3. ESB component interaction diagram
ESB
Best practices in ESB scheme design
According to some experience in our previous project design and development, we recommend that the ESB scheme design should follow the following best practices:
determine the use of standards: whether to use it and to what extent;
determine the business logic implemented on the ESB : ESB is a service routing and translation center, not an application server, so it does not replace the due server. The cost of complex message parsing and conversion is much higher than that of simple routing operations. Therefore, issues such as routing, format conversion, and service invocation should be mainly considered on ESB, and the processing of the data itself should be handed over to the corresponding application;
determine message format: XML is of course preferred from a standardization perspective, but some specific formats may be used in terms of parsing/processing performance, industry standards, and maximum compatibility with existing applications , such as EDI, SWITF, plain text or custom formats, etc.;
distinguish message header and message body: put the Meta-data of the data, such as security-related information, log level, requester identification, etc. in the message header, and Don't put it in the message body. This makes it easy to change its content and its processing logic. Only handle message headers in ESB to avoid parsing the message body;
refer to ESB-related mature Patterns when designing;
use service registry: if you need to search for service endpoints: it is recommended to search from the service registry, the advantages of this are: service Providers can easily publish new services, and the coupling between service providers and ESBs can be lower, and services can be searched and routed through metadata about the service itself;
focus on performance and high availability considerations;
if necessary Consider transaction integrity below;
Adoption of adapters: The application system realizes two-way interaction with ESB through adapters. Adapters are mainly divided into two types: technology adapters and application adapters. The physical deployment of adapters can be deployed with EIS, with ESB, or separately. Two aspects of communication protocol and message format should be considered in the adapter design;
the design of multiple ESBs: ESB is also a logical component, and multiple ESBs may be required in an enterprise, for example: an internal ESB connects various systems within the enterprise, an external ESB Realize the external connection between the enterprise and its partners; another example: there may be several department-level ESBs and one enterprise-wide ESB within the enterprise;
determine the service version control strategy;
determine the end-to-end QoS criteria;
focus on security;
determine the ESB platform of the IT department The responsible subject, long-term investment.
Security Consideration
of ESB There are two main ways to consider the security of ESB. The first way is to authenticate/authorize the service requester through the Mediation node inside the ESB; the other way is to call an external service to make a service request. As shown in Figure 4, a schematic diagram of these two methods is given in the figure, which can be selected during the specific implementation.

Figure 4. Two security implementations of
ESB
Management and monitoring of ESB services running on the ESB platform
When an enterprise begins its SOA journey, it usually selects a specific project to attempt SOA at the beginning, and then Gradually moving towards enterprise-wide adoption, at this time, most enterprises will face a problem, that is, there are more and more services, and there are many problems in the management of these service catalogs, such as: how to manage all service-related information, Including storage, management, maintenance, access, etc.? How does a service requester decide which service to use? How does the service requester locate the endpoint of the service? How do I get notified when service information changes?
Therefore, we recommend that users consider the construction of a service registry as early as possible. The so-called service registry is an enterprise-wide service information repository, which stores the services and service-related information registered in the enterprise. Its main functions include:
adopting a centralized way to manage service-related information (Interface, Service Location, Service Specification...), providing a "registry center" function for service metadata, allowing users to store, manage and query data containing service descriptions Service metadata components (WSDL, XSD, WS-Policy, SCDL or XML documents);
provide governance functions for services, and realize the management of the entire service life cycle;
provide management of dependencies and containment relationships between services;
provide Categorization and versioning Control (Versioning) and other functions;
provide capabilities such as service discovery and notification.
In addition to the consideration of the service registry, we also need to consider the management of the service. Services not only have specific functions, but also should meet some QoS metrics such as performance, availability, security, etc. The speed of the service response, when it is available, who can be called, how many times it can be called in a certain period of time, and which events should be logged, these are all issues to be considered by service management. Through the organic cooperation between the service registry and the service monitoring platform, we can achieve dynamic access to services according to policies such as service response time and service availability. Let's look at an example:

Figure 5. Service request and response of
ESB before using service monitoring platform

Figure 6. Service request and response of ESB after using service monitoring platform
As
shown in Figure 5 and Figure 6, we can leverage the service The monitoring platform monitors and statistically analyzes services, so that the ESB platform can dynamically bind and invoke high-performance services according to the performance of service providers. The process is as follows:
The service requester requests the "PurchaseOrder" service. The
service registry finds the service provider.
The service registration management platform calls the service provided by Proder1 for the first time.
The performance of Provider1 is monitored by the monitoring
tool . Over time, the performance of Provider1 is getting worse and worse
When the monitoring software monitors this phenomenon, it will stop using the service provided by Provider1.
When service requester requests this service again, the service registration management platform will call Provider2 to request the service provided by Provider2.


ESB development and testing The work to be completed
in the ESB development and testing stage mainly includes:
model-driven rapid development based on eclipse tools;
ESB integration process development;
ESB routing, message processing logic development;
ESB data mapping and transformation development ;
Development and configuration of ESB peripheral adapters;
Unit testing: module-based testing, including adapter testing, routing testing, BO testing, etc.;
Integration testing: ESB integration testing with other service providers and service consumers, focusing on Service interface;
performance test and system test of the ESB platform, that is, the test of the end-to-end business scenarios involved in the entire ESB.
When doing ESB development based on the WebSphere Message Broker platform, best practices in the following areas are usually considered, such as: common error/exception handling, common logging/management mechanisms, sub-process design, transaction integrity and message persistence, Use of ESQL, etc.



concluding remarks
This series of articles combines the characteristics and needs of different industries in the transportation industry and manufacturing enterprises to introduce the scheme design of enterprise ESB in two real environments. Combined with years of project implementation experience and methodology, I share with you the ESB project in the ESB project. Some experience in requirements analysis, program design, etc. in the implementation process, I hope everyone's ESB project will be successful.