Based RocketMQ + Knative driving cloud native application Serverless

Surely we have a better understanding RocketMQ news service, RocketMQ combined with Serverless what will spark it crash? Then we introduce today about how cloud-based RocketMQ + Knative drive Serverless native application. The main authors introduced several convenient:

Cloud native and Serverless
Knative Profile
RocketMQSource
Food distribution scenario example

Cloud native

CNCF look at the definition of native cloud:
cloud-native application technology in favor of the organizations in the public cloud, private cloud and hybrid cloud and other new dynamic environment, build and run elastically extensible. On behalf of native cloud technology including containers, service grid, micro-services, infrastructure and immutable declarative API.

These techniques can be constructed fault-tolerance, easy to manage and easy to observe loosely coupled systems. Combined with reliable automated means, native cloud technology enables engineers to easily make frequent and predictable system for major changes.
In fact, cloud native aims to provide a standardized way of convergence of cloud services cloud vendors and customers. This way for customers to reduce the cost of cross-cloud and cloud migration, so that customers always retain the ability to cloud vendors and bargaining; cloud manufacturers, low cost because the customer migration across the cloud, so long as they provide a more cost-effective high cloud services, it can easily gather a large number of users.

Serverless

Serverless (no server architecture) refers to the server-side logic implemented by the developer, running in stateless computing container, triggered by an event, fully third-party management, its business-level state is stored in a database or other media.
Serverless can be understood as an advanced stage of technological development of the original cloud of them, so that more developers to focus on business logic, while reducing the focus on infrastructure.
Mentioned here is Functions Serverless, in fact, in addition to Functions Serverless, there is another aspect Serverless: Serverless containerized. Compared to Function Serverless, containerized serverless, more portable, developers of complex applications to better control. In addition, for experienced users baptism era container, the container of the serverless those might be a better choice.
For Serverless, the following points need to look at:

Events (event) drive: Serverless is driven by events (event) (eg HTTP, pub / sub) is fully managed computing services.
Auto flexibility: on-demand, peak load shifting.
Per-use billing: Compared with traditional services in the use of resources (ECS example, VM specifications, etc.) billing, under Serverless scene is more in accordance with the amount of use of services (number of calls, duration, etc.) billing
Green computing: the so-called green computing is actually maximized to enhance the efficiency of resource use, reducing waste of resources, do the "energy saving"

Knative

The above mentioned container of serverless, then there is no such Serveless platform framework it? The answer is: Knative

Knative was released in 2018. Google Cloud Next General Assembly based on a Serverless orchestration engine Kubernetes of. Knative a very important goal is to develop a cloud native, cross-platform standard Serverless choreography. Knative is built (or function) through the integration of the container, the Serverless standard workload management (elasticity) and the three event model to achieve. The main contributor to the current Knative communities have Google, Pivotal, IBM, RedHat. Also like CloudFoundry, OpenShift these PAAS providers are actively involved Knative building.

Knative core module

Knative core module includes an event-driven framework Eventing and Serving deployment of workloads.

Serverless Services Engine - Serving

Knative Serving core competency is its simple and efficient application hosting service, which is the basis of its ability to support Serverless. Application hosting services provided by Knative can greatly reduce the complexity and risk of direct manipulation Kubernetes resources, improve iterative efficiency and service delivery applications. Of course, as the ability Severlesss Framework can not do without on-demand resources, Ali cloud container services Knative automatically according to the amount requested expansion of the number of instances of your application at peak times, when after a request to reduce the amount of the automatic volume reduction the number of instances, can be very automation help you save costs.
Serving by binding to Istio also provides a powerful and flexible traffic management capabilities grayscale publishing capabilities. Traffic management capabilities may be able to publish the gradation according to the gradation according to the percentage of the percentage of traffic flow splitting, while also the ability to test the gradation released before on-line self-defined by the tag, their very easy CICD and system integration.

Serving Application Model

Service: abstract application Serverless choreography, Service Lifecycle Management applications
Configuration: The current configuration state expectations. Each update will update Configuration Service
Revision: configuration of each update will create a snapshot, used for version management
Route: route requests to Revision, and forwards traffic to the different proportions of different Revision

Event-driven framework - Eventing

1. Using CloudEvent as an event transport protocol: CloudEvent describing the event data in a common format, providing the ability to cross-platform interactive services. KnativeEventing use CloudEvent as an event transmission standard, which greatly enhance the cross-platform application portability
2. External event source access and registration: Support Github, RocketMQ events such as source and Kafka, of course, the user can define an event source.
3. Subscribe to and triggering events: Introducing Broker and model significance Trigger, the event will not only complex processing to achieve shielding the user, but also provides a wealth of event subscription, filtering mechanism.
4. Compatible with existing messaging systems: KnativeEventing substantially decouple the messaging system's implementation, in addition to the current system supported by the memory-based message channel InMemoryChannel but also supports Kafka, NATSStreaming other messaging services in addition to conventional butt can easily messaging system.

Eventing in Broker / Trigger model

Here are what the Eventing Broker / Trigger model, is not complicated. External event source to the event to a Broker, after Broker receiving events sent to the corresponding Channel (i.e. message buffer, where forwarded as Kafka, InMemoryChannel, etc.), to achieve subscribed events by creating Trigger subscription Broker, further defined in the Trigger in corresponding services, to achieve the ultimate event-driven service.

RocketMQSource

RocketMQSource is RocketMQ event source Knative platform. It can be a message to Cloud Event RocketMQ cluster format in real time forwarded to Knative platform, it is the connector between the Apahe RocketMQ and Knative.

Knative + RocketMQ example scenario - food distribution Scene

Our next example demonstrates to food distribution, food distribution scenario has the following characteristics:

Food distribution day memory obvious peak, trough.
Single large volume at peak times

For this situation, we use the message-driven Serverless, automatic expansion of resources at peak time, reduce the resources at the bottom of the time, on-demand can be greatly improved resource utilization, thereby reducing costs.

A typical architecture is as follows:

When meal time comes, customers ordering single message sent to the next generation RocketMQ, by RocketMQSource obtain single message into an event to the Broker, subscription orders by Trigger Event final drive service orders generated ordering single. With this scheme has the following advantages:

By Knative to RocketMQ technology as the core of the food distribution system Serverless server can reduce operation and maintenance costs and to a great extent.
Knative flexibility can help you easily cope with breakfast, lunch and dinner peak demand resources
RocketMQ do asynchronous decoupling system, to avoid problems long link calls, improve system availability.

operating

Deployment Knative.

See Ali cloud service container deployment Knative

Deployment RocketMQSource

In Knative management component, select RocketMQSource click deployment.

Deployment order service

Reference Example code repository: https://github.com/knative-sample/knative-rocketmq
a key deployment service command as follows:

kubectl apply -f 200-serviceaccount.yaml -f 202-clusterrolebinding.yaml -f 203-secret.yaml -f alirocketmqsource.yaml -f broker.yaml -f ksvc-order-service.yaml -f trigger.yaml

Analog peak ordering orders.

By single simulation, and transmits the message to the RocketMQ can. Message Format Reference:

{"orderId":"123214342","orderStatus":"completed","userPhoneNo":"152122131323","prodId":"2141412","prodName":"test","chargeMoney":"30.0","chargeTime":"1584932320","finishTime":"1584932320"}

Demonstration effect is shown:

Other scenarios

Knative + RocketMQ typical scenario - building Serverless electricity supplier system

Knative resilience can help you easily cope buy, such as double eleven electricity supplier's big promotion activities
RocketMQ system do centered asynchronous decoupling to avoid problems long link calls, improve system availability.

Knative + RocketMQ typical scenario - building alarm monitoring platform

Metric, Log RocketMQ other cluster data push service Knative
Knative service through data analysis will alarm push content such as nails or slack communication tools
Knative service can be Metric or logs data processing, push third-party system

Knative + RocketMQ typical scenario - multiple data format conversion

Processing the data log to generate multiple results as well as derivatives, these results can be used for derivatives operations, marketing and sales.
The content conversion from one format to another format, for example, Microsoft Word will convert to PDF.
Main media files need to be converted to a variety of formats

to sum up

RocketMQ event-driven application Knative Serverless
Scenarios
- Ordering, Takeout
- Buy, limit buy
- Construction of alarm monitoring platform
- Multiple data format conversion
- Logistics
Source
- RocketMQSource：https://github.com/apache/rocketmq-externals/tree/master/rocketmq-knative/source
- Food distribution Example: https://github.com/knative-sample/knative-rocketmq