In the second tutorial , we learned how to use a work queue to assign time-consuming tasks among multiple workers.
However, if we need to run a function on the remote computer and wait for the result? Well, that's another matter. This mode is often referred to as Remote Procedure Call (Remote Procedure Call) or RPC.
In this tutorial, we will use RabbitMQ to build the RPC system: RPC clients and scalable server. Since we do not have any time-consuming task worthy of distribution, we will create a virtual return of Fibonacci numbers RPC service.
1, client interface
To illustrate how to use the RPC service, we will create a simple client class. It is disclosed a method called the call, it sends an RPC request and blocks until a response is received:
FibonacciRpcClient fibonacciRpc = new FibonacciRpcClient();
String result = fibonacciRpc.call("4");
System.out.println( "fib(4) is " + result);Description of the RPC
Although RPC is a very common pattern in the calculation, but it is often criticized. When the programmer does not know the function call is local or slow RPC, problems will arise. Such confusion has led to a system unpredictable, debugging and adds unnecessary complexity. Do not need to simplify the software, misuse RPC causes spaghetti code difficult to maintain.
With that in mind, consider the following recommendations:
- To ensure that it is clear which function calls are local and which are remote.
- Writing documentation for your system, the dependencies between components becomes clear.
- Handle error conditions. When the RPC server is down for a long time, should the client how to react?
Avoid using RPC when in doubt. If you can, you should use asynchronous pipeline - instead of blocking that kind of like rpc
2, the callback queue
In general, an RPC on RabbitMQ is very easy. Client sends a request message, a server replies with a response message. In order to get a response, we need to request a "callback" queue address. We can use the default queue (is exclusive in the Java client). Let's give it a try:
callbackQueueName = channel.queueDeclare().getQueue();
BasicProperties props = new BasicProperties
.Builder()
.replyTo(callbackQueueName)
.build();
channel.basicPublish("", "rpc_queue", props, message.getBytes());
// ... then code to read a response message from the callback_queue ...Message Properties
AMQP 0-9-1 protocol predefined set of attributes 14 for use with the message. Most of the properties are rarely used, with the following exceptions:
- deliveryMode: The message is marked as persistent (value 2) or transient (or any other value). You may recall this second tutorial in this property.
- contentType: used to describe the encoding mime-type. For example, encoding is often used for JSON, the property to:
application/JSON. - replyTo: a name commonly used for the callback queue.
- correlationId: associating RPC response is useful with the request.
We need to introduce a new package:
import com.rabbitmq.client.AMQP.BasicProperties;3、Correlation Id
In the method given above, we recommend creating a callback request queue for each RPC. It is very inefficient, but fortunately there is a better way - let us create a callback queue for each client.
This leads to a new problem, received in response to this queue, it is not clear which request the response belongs. This is why we want to use the correlationIdproperty. Each request we set it to a unique value. Later, when we received a message in the callback queue, we will look at the attribute, based on this, we will be able to match the response according to the request. If we see an unknown correlationId value, we can safely discard the message - it does not belong to our request.
You might ask why we should ignore the callback queue of unknown messages, rather than erroneously failed? This is because the server is possible race condition occurs. Although less likely, RPC server might send us after the death of the answer, but before sending a confirmation message to the request. If this happens, restart the RPC server processes the request again. That's why we have the client gracefully handle duplicate responses, and RPC should be idempotent.
Our RPC will work like this:
- When a client starts, it will create a monopoly anonymous callback queue.
- For RPC request, the client sends a message with two properties: replyTo, it is set to correlationId callback queue and it is set to a unique value for each request.
- Request is sent to a
rpc_queuequeue. - RPC worker (aka: server) listens for requests on the queue. When a request occurs, it will perform the task, and by replyTo field queue will send the results back to the client.
- The client waits for data on the queue callback. When a message appears, it checks the
correlationIdproperty. If it matches the value of the request, it returns a response to the application.
Together they are:
Fibonacci tasks:
private static int fib(int n) {
if (n == 0) return 0;
if (n == 1) return 1;
return fib(n-1) + fib(n-2);
}Fei Ming Feibo that we sound function. Assumption is valid only positive integer inputs. (Do not expect this one to work for the big number, and it may be the slowest recursive implementation).
Our server RPC RPCServer.javacode like this:
public class RPCServer {
private static final String RPC_QUEUE_NAME = "rpc_queue";
private static int fib(int n) {
if (n ==0) return 0;
if (n == 1) return 1;
return fib(n-1) + fib(n-2);
}
public static void main(String[] argv) {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("192.168.20.128");
factory.setPort(5672);
factory.setUsername("carl");
factory.setPassword("198918");
Connection connection = null;
try {
connection = factory.newConnection();
final Channel channel = connection.createChannel();
channel.queueDeclare(RPC_QUEUE_NAME, false, false, false, null);
channel.queuePurge(RPC_QUEUE_NAME);
channel.basicQos(1);
System.out.println(" [x] Awaiting RPC requests");
Consumer consumer = new DefaultConsumer(channel) {
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
AMQP.BasicProperties replyProps = new AMQP.BasicProperties
.Builder()
.correlationId(properties.getCorrelationId())
.build();
String response = "";
try {
String message = new String(body,"UTF-8");
int n = Integer.parseInt(message);
System.out.println(" [.] fib(" + message + ")");
response += fib(n);
}
catch (RuntimeException e){
System.out.println(" [.] " + e.toString());
}
finally {
channel.basicPublish( "", properties.getReplyTo(), replyProps, response.getBytes("UTF-8"));
channel.basicAck(envelope.getDeliveryTag(), false);
// RabbitMq consumer worker thread notifies the RPC server owner thread
synchronized(this) {
this.notify();
}
}
}
};
channel.basicConsume(RPC_QUEUE_NAME, false, consumer);
// Wait and be prepared to consume the message from RPC client.
while (true) {
synchronized(consumer) {
try {
consumer.wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
} catch (IOException | TimeoutException e) {
e.printStackTrace();
}
finally {
if (connection != null)
try {
connection.close();
} catch (IOException _ignore) {}
}
}
}The server code is quite simple:
- As usual, we begin to establish a connection queue, channel and statements.
- We may want to run more than one server process. In order to uniformly distribute the load across multiple servers, we need to
channel.basicQosset theprefetchCountproperty. - We use
basicConsumeto access the queue, where we provide a callback to an object (DefaultConsumer) form, it will complete the work and send back a response.
Our RPC client code RPCClient.java::
public class RPCClient {
private Connection connection;
private Channel channel;
private String requestQueueName = "rpc_queue";
public RPCClient() throws IOException, TimeoutException {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("192.168.20.128");
factory.setPort(5672);
factory.setUsername("carl");
factory.setPassword("198918");
connection = factory.newConnection();
channel = connection.createChannel();
}
public String call(String message) throws IOException, InterruptedException {
final String corrId = UUID.randomUUID().toString();
String replyQueueName = channel.queueDeclare().getQueue();
AMQP.BasicProperties props = new AMQP.BasicProperties
.Builder()
.correlationId(corrId)
.replyTo(replyQueueName)
.build();
channel.basicPublish("", requestQueueName, props, message.getBytes("UTF-8"));
final BlockingQueue<String> response = new ArrayBlockingQueue<String>(1);
String ctag = channel.basicConsume(replyQueueName, true, new DefaultConsumer(channel) {
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
if (properties.getCorrelationId().equals(corrId)) {
response.offer(new String(body, "UTF-8"));
}
}
});
String result = response.take();
channel.basicCancel(ctag);
return result;
}
public void close() throws IOException {
connection.close();
}
public static void main(String[] argv) {
RPCClient fibonacciRpc = null;
String response = null;
try {
fibonacciRpc = new RPCClient();
for (int i = 0; i < 32; i++) {
String i_str = Integer.toString(i);
System.out.println(" [x] Requesting fib(" + i_str + ")");
response = fibonacciRpc.call(i_str);
System.out.println(" [.] Got '" + response + "'");
}
}
catch (IOException | TimeoutException | InterruptedException e) {
e.printStackTrace();
}
finally {
if (fibonacciRpc!= null) {
try {
fibonacciRpc.close();
}
catch (IOException _ignore) {}
}
}
}
}Client code bit more complicated:
- We establish a connection and channel, and declare an exclusive "callback" queue to answer.
- We subscribe to the "callback" queue, so that we can receive the RPC response.
- Our
callmethod of issuing the actual RPC request. - Here, we first generate a unique correlationId number and save it - we realize handleDelivery implemented in the DefaultConsumer will use this value to find the appropriate response.
- Next, we release request message has two properties:
replyToandcorrelationId. - At this time, we can sit down and wait for the appropriate response back.
- Due to our consumers to deliver treatment in a separate thread occurred, so before the arrival of response, we need something to hang the main thread. BlockingQueue usage is one of the possible solutions. Here we create ArrayBlockingQueue, capacity is set to 1, because we only need to wait for a response.
handleDeliveryThe method is very simple to do work for each response message is consumed, it checks whether the information we are looking for. If so, it will respond into BlockingQueue.- At the same time, the main thread is waiting for a response, you can get it from BlockingQueue.
- Finally, we will respond back to the user.
Sending a client request:
RPCClient fibonacciRpc = new RPCClient();
System.out.println(" [x] Requesting fib(30)");
String response = fibonacciRpc.call("30");
System.out.println(" [.] Got '" + response + "'");
fibonacciRpc.close();First start RPCServer.javaof the code:
# => [x] Awaiting RPC requestsRun the client RPCClien.javato request a fibonaccifunction:
[x] Requesting fib(0)
[.] Got '0'
[x] Requesting fib(1)
[.] Got '1'
...
[x] Requesting fib(30)
[.] Got '832040'
[x] Requesting fib(31)
[.] Got '1346269'Here are the RPC service design is not only possible to achieve, but it has some important advantages:
- If the RPC server is too slow, you can extend by running another server. Try a new console to run the second RPCServer.
- In the client, RPC only needs to send and receive a message. You do not need to like
queueDeclarethis synchronous call. Therefore, RPC client RPC request of only one network round trip.
We're still very simple, and does not attempt to solve more complex (but important) issues, such as:
- If the server is not running, the client should be how to react?
- Whether the client should provide some kind of timeout is RPC?
- If a server fails and throws an exception, whether it should be forwarded to the client?
- Protected void prior to processing incoming messages (e.g., check boundary type).
