BPEL or ESB: which one should you use? Original address http://www.ibm.com/developerworks/cn/websphere/library/techarticles/0803_fasbinder2/0803_fasbinder2.html
When designing SOA solutions, it is not always clear whether to use Web services BPEL processes or ESB mediation flows. This article will cover some considerations to help you decide which one to use.
Overview
In the IBM® SOA Reference Architecture (shown in Figure 1), services are grouped into functional areas and communicate through an Enterprise Service Bus (hereafter referred to as ESB). Ideally, each functional area (such as process services) is a "pure" functional area, providing only one service class for separation of concerns.
Figure 1. SOA reference architecture
In the real world, however, any product set that overlaps usually includes a ribbon. For example, WebSphere Process Server (hereafter referred to as Process Server) is a software component that provides process services in the reference architecture. It evolved from WebSphere MQ Workflow, WebSphere Interchange Server, and WebSphere Business Integration Server Foundation. To make it easier for users to upgrade from the previous generation, it includes functionality equivalent to that in the older product. For example, Interchange Server includes a business object map that exists in Process Server as an interface map.
You can map business objects that flow into or out of a business process. Mapping is also one of the main functions in an ESB. So if you have both a Process Server and an ESB, you need to decide which product should be used to solve a given business problem, because these areas overlap.
There are other use cases where Process Server or ESB might be used. For example, suppose three existing services need to be invoked using two-phase commit. This is called a composite service. In the ESB, you can call these services using mediation flows. Mediation flows are committed or rolled back as transactions. You can use WS-BPEL microflows to invoke these three services while providing transactionality. In this way, both products can be used as a solution. How to decide which product is the right one?
back to top
ESB overview
ESB is an architectural pattern, not a software product. Different software products can constitute an ESB. In some cases, companies use multiple products in different regions, leveraging specific capabilities to meet their unique needs. These different products can be combined to implement the ESB pattern.
In a nutshell, an ESB has four main functions:
Message routing: Sending incoming messages to a destination determined either by hard-coded wired logic or dynamically based on content. Routing is a key feature that enables service virtualization. Establishing an intermediate layer between the caller and the service can move the location of the service without the caller being aware of the change.
Message Conversion: Convert incoming messages from one format to another. For example, a comma-separated message can be converted to SOAP so that data can be passed to a Web service.
Protocol Mediator: Incoming messages are sent from the originating location using a different protocol. For example, incoming messages can use HTTP, while outgoing messages can use WebSphere MQ.
Event Handling: Incoming messages of events are typically distributed to many endpoints through a publish and subscribe model.
Within a given transaction, these major advanced functions are usually combined. For example, the incoming message might be a Web service call using SOAP/HTTP and the destination is a legacy system that requires the use of WebSphere MQ's fixed-length message format. Messages must be translated, protocols must be coordinated, and messages must be routed to the correct location.
Programming an ESB often involves virtual environments and expressing logic as flows of connected activities called message flows or mediation flows. These streams are transactional, using mechanisms like two-phase commit, so that the entire stream can be rolled back on failure, or committed on success. These streams are stateless; typically a message comes in, the stream performs various operations on that message, and then sends the outgoing message.
Due to the stateless transactional nature of an ESB, high performance is a prerequisite. In large organizations, it is not uncommon for ESBs to process millions of messages per day.
IBM offers several software products on the ESB side. WebSphere ESB is built on the WebSphere Application Server Network Deployment platform. It was built to support standards like Web Services, JMS, and XML. WebSphere Message Broker (hereafter called Message Broker) is a non-J2EE product that supports standards in WebSphere ESB, such as Web services, as well as many non-standard-based protocols and data formats. WebSphere DataPower is a hardware tool that performs ESB functions on the fly. In conclusion, all three products can be used as the basis for an ESB.
back to top
BPEL overview
The OASIS standards organization has defined Business Process Execution Language (BPEL) as a standards-based method by which business processes composed of services can be choreographed. In 2007, WS-BPEL 2.0 was approved as a standard language. As an execution language, WS-BPEL defines how to represent activities in business processes, as well as flow control logic, data, message dependencies, exception handling, and more.
IBM's WebSphere Process Server (hereafter referred to as Process Server) includes a business flow orchestrator (ie, a WS-BPEL-based flow engine). The previous release, called WebSphere Business Integration Server Foundation, also included WS-BPEL support.
Process Server includes several key features, including:
Business Processes: Processes can be stateful and long-running, or transactional microflows. Long-running processes cannot be rolled back like microflows, however, they can use compensation handlers to undo previously performed activities. Processes can be used to implement composite services.
Human tasks: A key part of a business process is the ability to bring people into the process. The Human Task Manager enables steps by which people can be invoked as a service. Workflow patterns are supported through external (participating) tasks or inline tasks using BPEL extensions.
Business Rules: An integrated rules engine allows business rules to be created and evaluated instead of hardcoding decisions into business processes. Authorized users can update this rule using a web browser. Administrators can activate updates and export them, so the development environment can be kept in sync with the runtime.
Integrated ESB: Process Server includes the complete WebSphere ESB product. In this article, we only cover the BPEL engine component of Process Server.
SCA: Service invocation in Process Server is done using the Service Component Architecture (SCA) specification. The SCA interface map can be used to invoke a service whose interface is different from the invoking component. Interface mapping also supports advanced features such as relationships. If system A uses "123" as the customer identifier and system B uses "ABC" as the customer identifier, then when converting between the two systems, you can use the relationship to mediate "123" to "ABC", vice versa.
back to top
Decide which runtime to use
As mentioned earlier, there is some functional overlap between Process Server and ESB. Both work with adapters. Both can transform data. Both support the composite service pattern. When faced with the best software to solve a given business problem, you need to decide which one to use.
As an architect, you need to be familiar with the capabilities of both software platforms. Once you have gathered the complete set of requirements, you can begin to decide which process to use.
The first part of the business is to analyze the requirements and determine if the selected process is a more suitable process. For example, if a stateful process is required, the ESB can be excluded immediately. On the other hand, if the message transformation is required to be processed within 0.2 seconds, the ESB should obviously be chosen. Unfortunately, in the real world, not all project requirements can be mechanically applied. Often multiple conditions need to be checked to determine the best option.
Advantages of ESBs
One of the main advantages of an ESB is processing messages. Incoming and outgoing messages may use protocol or format intermediaries. When these requirements clearly require processing messages, using an ESB can provide many advantages, including the ability to handle more complex transactions in transformations. ESBs are the best choice when these requirements require the use of one of ESB's basic functions, such as message routing, transformation, or protocol mediation.
Another advantage of ESBs is performance. The ESB is planned to be able to handle a large number of messages. For example, if the requirement is to process 200,000 messages per day, an ESB is an obvious choice.
If the requirements are data-centric, the ESB is the obvious choice.
Advantages of BPEL
The main advantage of a BPEL engine is the ability to orchestrate business processes. If the requirement is to handle a process with complex logic, BPEL is a better choice. WS-BPEL includes container activities such as while loops and scopes that are not supported by ESB. The logic in ESB is usually very simple, while WS-BPEL can handle more complex situations.
Another advantage of a WS-BPEL engine such as WebSphere Process Server is the ability to keep business processes running and maintain their state for a long time. ESBs should not be used when state is required, as maintaining state would take up a lot of custom code. If the requirement is for stateful processing, the ESB can be excluded from the selection.
If the requirements are process-centric, the obvious choice is WS-BPEL.
Feature Notes
The specific needs of a particular project are the primary factors in determining which environment is best for a given project. Some important questions you need to consider include:
Which transport protocols are required? For example, both Process Server and Message Broker support WebSphere MQ, but only Message Broker supports IP multicasting. If you use JMS, it is also important to know if your system supports a specific JMS provider.
What standards are required? For example, the requirement might be WS-Security, or support for complex industry-standard schemas. You need to know if the server supports these standards. For example, Message Broker may have stronger support for untyped message processing, while Process Server can use J2EE security because it runs in a J2EE container. Different environments may support a standard, but at different levels. Make sure to check all the details before making a decision.
What are Quality of Service (QoS) requirements? Does the system have to be highly available? If Message Broker is installed in a high availability environment, but Process Server is not installed, it may affect your decision. You need to understand the specific environment to understand the QoS available. You need to consider performance requirements as well as things like guaranteed mail delivery.
What routing and messaging styles are required? Various messaging styles can be used in the system, such as synchronous or asynchronous, request/response, one-way, and publish/subscribe. What messaging styles are required? May be a messaging style or a combination of messaging styles. What messaging styles are supported in a given environment? Process Server can perform message publishing through JMS topics, while Message Broker has more advanced capabilities such as content-based subscriptions. These requirements help narrow down runtime choices.
Non-Functional Considerations
In addition to functional requirements, each system also contains a set of non-functional requirements that also need to be considered when choosing the correct runtime.
What is total cost of ownership? You need to consider the initial cost of the software, as well as the associated long-term costs. For example, if the customer can achieve a lower cost by implementing the solution on the J2EE platform, this can influence the decision to choose Process Server.
What are administrative costs? Every system requires an administrator. If the company lets the old administrator manage the new project (without adding new resources), it can affect the total cost of ownership. For example, an MQ administrator might be familiar with Message Broker, but might not know anything about J2EE-based runtimes. Consideration should also be given to how easily the environment can be monitored, as well as the security aspects that administrators deal with.
What skills are required? In addition to admin, developer, tester, other roles are required. The associated time and cost savings can be achieved if existing skills can be leveraged. Learning new skills for a project may require training or coaching.
What is the existing environment? Is the middleware environment a production environment? Are users familiar with the toolset? If new tools are required, how well do these new tools relate to the current tooling environment? Are the current version levels of prerequisite software synchronized? Do I have to upgrade existing assets? Are there any migrations required before using the system?
Does an option offer unique functionality? Unique features in an environment may facilitate your decision. For example, you can perform the mapping in the ESB or through the interface mapping in Process Server. However, only interface mappings in Process Server can provide relational mapping. If you need this capability, only Process Server can provide it.
While both Process Server and ESB may be ideal candidates, non-functional requirements can often narrow the options.
model
E-commerce patterns are a set of proven, reusable assets that can be used to help you accelerate the development and deployment of e-commerce applications. IBM publishes a series of redbooks describing pattern usage and various techniques, such as Patterns: SOA Design Using WebSphere Message Broker and WebSphere ESB.
You can examine your requirements and determine if patterns can be used to achieve them. If you prove that an existing pattern or combination of patterns works for a given runtime, you know that the runtime can handle your needs. This way the risk is reduced because it has been dealt with before, and by applying the pattern the development time can be shortened. If the pattern exists only on one runtime, it can help you decide.
Some typical patterns include:
Message aggregation and de-aggregation (N-to-1 and 1-to-N)
1-of-N response (send multiple requests, pick one response)
Service Virtualization (Location and Identity)
content-based routing
Adapter interaction
The news is enriched
Dynamic registry query
event propagation
Gateway (controls and secures interactions between internal and external domain boundaries)
For example, the referenced redbook describes the usage of DataPower using the Gateway pattern. The XS40 XML Security Gateway addresses the functional and non-functional requirements to support this schema. This can help facilitate your decision if you are looking for a solution to use for implementing the Gateway pattern.
grey area
While filtering using the above criteria can help determine the final decision, there are still situations in which to choose an ESB or a Process Server as the runtime for a given project. In these cases, there are a few other things to consider.
design concept
Certain companies have an overarching design philosophy that guides all projects. For example, you might decide that all transformations will be done in the ESB, while all business logic resides in Process Server.
If a customer places an online order, the business decision to determine whether they qualify for free shipping belongs to the business logic layer running in Process Server. Routing of messages to the corresponding shipping services can be done at the ESB layer. It allows companies to change business logic without having to make changes to integration logic (and vice versa).
This general design philosophy should play an important role in the decision. In other words, what is the strategic direction? All other things being equal, you can use this general design principle as a standard.
cost
Each WS-BPEL process or mediation flow deployed on a server consumes that server's CPU capacity. Of course, in addition to hardware costs, support costs and management costs, etc., the middleware it runs is also a cost. If it is unclear which environment to use, cost can be factored into the deployment location decision.
maintain
When deciding which platform is the best platform to solve a given business problem, you need to consider the overall long-term maintenance of the system. If it's easier to use WS-BPEL correlation sets than to write and maintain code in an ESB to provide similar functionality, then not only can you build solutions quickly, but it's also easier to maintain. If the built-in base functions are available at any time, it is a better design alternative than creating custom code. Solutions that are easy to maintain should be prioritized.
Another aspect of maintenance is how easy it is to upgrade the solution in the future as the middleware product evolves. Often, new major releases are able to use artifacts from previous releases. WebSphere Process Server V6's tool, WebSphere Integration Developer, can import integration projects from WebSphere Business Integration Server Foundation V5.1 and use previous projects as a starting point. The logic in the WS-BPEL process will be transferred to the new environment. However, you may need to perform some manual actions in the process to update the Java™ code. In most cases, using the software product's built-in functionality can make future upgrade paths a bit easier.
reusability
When constructing any asset for SOA, you should know how to facilitate reuse of that asset. When you create an asset, you can use it to solve a business problem. However, if the asset can be reused, development time and cost can be greatly saved.
Using the previous online ordering example, you can perform the entire job using an ESB or WS-BPEL microfluidic process. If the business logic is brought into WS-BPEL and the routing logic in the ESB is used, there is a greater opportunity for reusability. Using both functions in a single environment, the service can only be used once because they are both designed for a specific purpose. By separating functions, there will be a good chance of reusing any of them. For example, an internal shipping system can utilize routing services. The potential reusability of the created solution should be considered when making decisions.
Skill advantage
IT staff may have higher skills for certain products than others. To support your project in the long term, having strong skills can be a key factor in the decision. For example, if you have three experienced ESB developers, but only one is familiar with Process Server, it would be better to choose to use the ESB to build the project.
risk
A successful project is one that reduces risk and has a very small chance of failure. If you have implemented five modules in Process Server, but are just starting to build with Message Broker, at this point you may think that using Process Server is less risky because you have more experience with Process Server.
In the meantime, you can find references and case studies for situations similar to the system you need to implement. If several other users have successfully completed projects with Message Broker, it will be more secure for you to use. If other users are successful on similar projects, your risk is also reduced.
After evaluating all risk factors in your project, you can use lower risk as the final criterion for your decision.
back to top
Summarize
In this article, you learned about ESB and WS-BPEL process engines, and how to determine the process engine that provides the best solution for your environment. You learned the advantages of each environment, how to consider criteria such as functional and non-functional requirements, and how to apply patterns. Finally, you learned some other criteria that might help you make better decisions if both solutions meet your requirements.