RocketMQ how to connect multiple servers in the same Java process

Foreword

As we all know, RocketMQ to connect to the server has been restricted in the code level, basically can be understood as a JVM process can only connect a NameServer, but the practical application scenario, we might have been on to RocketMQ functions on architecture design level the division provides service a message processing class a and B class B service processes the message, then how do we solve this problem?

Root of the problem

We from the code level to analyze why in the end produce "a JVM instance can only connect a NameServer".
RocketMQ Client has a core class MQClientManager, when we need to use MQ Client instance, actually through it getAndCreateMQClientInstancecreated the method; the name of a mouthful, is also the Get and the Create , which is not in line with what we call the unity of design principle, but this is not the focus of our discussion, we look at the implementation of this method

    public MQClientInstance getAndCreateMQClientInstance(ClientConfig clientConfig, RPCHook rpcHook) {
        String clientId = clientConfig.buildMQClientId();
        MQClientInstance instance = (MQClientInstance)this.factoryTable.get(clientId);
        if (null == instance) {
            instance = new MQClientInstance(clientConfig.cloneClientConfig(), this.factoryIndexGenerator.getAndIncrement(), clientId, rpcHook);
            MQClientInstance prev = (MQClientInstance)this.factoryTable.putIfAbsent(clientId, instance);
            if (prev != null) {
                instance = prev;
                log.warn("Returned Previous MQClientInstance for clientId:[{}]", clientId);
            } else {
                log.info("Created new MQClientInstance for clientId:[{}]", clientId);
            }
        }

        return instance;
    }

Code is not complicated, we can see that it uses client configuration information to generate a fixed clientId, in order to cache factoryTable find, there will create a new instance.
So, it is understood only a clientID there is a connection instance, and can clientId this is how to generate it? Continue to look at this code tracking

    public String buildMQClientId() {
        StringBuilder sb = new StringBuilder();
        sb.append(this.getClientIP());
        sb.append("@");
        sb.append(this.getInstanceName());
        if (!UtilAll.isBlank(this.unitName)) {
            sb.append("@");
            sb.append(this.unitName);
        }

        return sb.toString();
    }

On the code level to clientId been agreed in the format " ClientIP @ InstanceName " format, unitName not empty when the time will be followed by the " @unitName ."

How to deal with it?

From the analysis we can know the code in order to create a multi-instance, we can

Set different instanceName:
instanceName come from?
```
instanceName = System.getProperty("rocketmq.client.name", "DEFAULT");
```
Read out from the system properties, which is generally set in the JVM starts. . .
You can change it? Of course, you can go do it through code, but to do so, you will lose your ability to make people understand the code, ha ha
this is why the number of RocketMQ Client can only connect to a server of the reason, it does not consider the server is who only care about their own selfish guy!
Set different unitName

In addition there are other solution? I carefully turned round from the network, did not see what good way, is that we did this scene there are other good way to solve it? Welcome to the discussion -

Method 3

In the blog post from the parallel world of salvation , and I made a tool Sandbox , I offer three methods that rely on this tool.
sandbox isolated manner through the code, the class definitions into another sandbox run multiple instances to achieve complete isolation effect.
MQClientManagerBy caching way to clientId key value is stored as instances among themselves, in order to achieve a plurality of Client, then the logic is to modify the first two methods to achieve clientId plurality of instances, the method 3 is logic " Since you have cached the key, I'll change the cache , "essentially" you do not install my pot, I'll change the pot . "

How to do it?

Here I use a springboot project as a demonstration case.
Springboot by Configurationa plurality RocketMQ Client register, and then define a Controller receives a request to send a different message MQ, finally add startup classes.
MQ multi-instance
We start with the introduction of pom file in the package (I have not pushed maven repository, you can from GitHub / gitee download), as follows

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>me.van</groupId>
    <artifactId>rocket-mq-multi-client-test</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>

    <name>测试多个rocketmq client共存</name>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.1.6.RELEASE</version>
    </parent>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <start-class>me.van.App</start-class>
        <java.version>1.8</java.version>
        <lombok.version>1.14.8</lombok.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>${lombok.version}</version>
            <scope>provided</scope>
        </dependency>

        <dependency>
            <groupId>org.apache.rocketmq</groupId>
            <artifactId>rocketmq-client</artifactId>
            <version>4.4.0</version>
        </dependency>
        <dependency>
            <groupId>me.van</groupId>
            <artifactId>sandbox</artifactId>
            <version>1.0.0</version>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

Here we introduced the apache mq rocketmq-client component as a client, that is, the presence of components of the previously mentioned problems.

Startup class

package me.van;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class App {
    public static void main(String[] args) {
        SpringApplication.run(App.class, args);
    }
}

Very simple, nothing presented.

Configuration class

package me.van;

import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.client.producer.DefaultMQProducer;
import org.apache.rocketmq.client.producer.MQProducer;
import org.springframework.beans.factory.annotation.Autowire;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class AppConfig {

    @Bean(autowire = Autowire.BY_NAME, value = "producer")
    MQProducer producer() throws MQClientException {
        DefaultMQProducer producer = new DefaultMQProducer();
        initProducer(producer, "a.io:9876;b.io:9876");
        return producer;
    }

    @Bean(autowire = Autowire.BY_NAME, value = "producer_sandbox1")
    MQProducer producerSandbox1() throws MQClientException, SandboxCannotCreateObjectException {
        DefaultMQProducer producer = createProducerInSandbox();
        initProducer(producer, "x.io:9876;y.io:9876");
        return producer;
    }

    @Bean(autowire = Autowire.BY_NAME, value = "producer_sandbox2")
    MQProducer producerSandbox2() throws MQClientException, SandboxCannotCreateObjectException {
        DefaultMQProducer producer = createProducerInSandbox();
        initProducer(producer, "1.io:9876;2.io:9876");
        return producer;
    }

    private DefaultMQProducer createProducerInSandbox() throws SandboxCannotCreateObjectException {
        Sandbox sandbox = new Sandbox("org.apache.rocketmq.client");
        return sandbox.createObject(DefaultMQProducer.class);
    }

    private void initProducer(DefaultMQProducer producer, String namesrvAddr) throws MQClientException {
        producer.setNamesrvAddr(namesrvAddr);
        producer.setProducerGroup("test-group");
        producer.setRetryAnotherBrokerWhenNotStoreOK(true);
        producer.start();
    }
}

Here can be seen, producerthe object is to direct new out DefaultMQProducer, and producer_sandbox1and producer_sandbox2are created out of different sandboxes; client are connected to three different NameServer while other properties remain consistent.

Controller

package me.van;

import org.apache.rocketmq.client.exception.MQBrokerException;
import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.client.producer.MQProducer;
import org.apache.rocketmq.common.message.Message;
import org.apache.rocketmq.remoting.exception.RemotingException;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class TestController {
    @Autowired
    MQProducer producer;
    @Autowired
    MQProducer producer_sandbox1;
    @Autowired
    MQProducer producer_sandbox2;

    @GetMapping("/")
    public String hello(){
        return "hello world";
    }

    @GetMapping("/send")
    public String send(String msg){
        if(null == msg) return "msg is null";

        String returnMsg = "";
        Message message = new Message("topic-test-multi-mq-client", msg.getBytes());
        try {
            producer.send(message);
            returnMsg += "原生producer发送完成<br/>";

            producer_sandbox1.send(message);
            returnMsg += "第一个沙箱内producer发送完成<br/>";

            producer_sandbox2.send(message);
            returnMsg += "第二个沙箱内producer发送完成<br/>";
        } catch (MQClientException | InterruptedException | RemotingException | MQBrokerException e) {
            returnMsg += "发送过程出现异常：" + e.getMessage();
        }
        return returnMsg;
    }
}

By sendsimultaneously three methods producerto send messages.

have a test

Running App, wait a few seconds to start is completed, visit http: // localhost: 8080 / send , return

msg is null

访问，http://localhost:8080/send?msg=test

Access results

Code address

github: https://github.com/vancoo/multi-mq-demo
gitee: https://gitee.com/vancoo/multi-mq-demo