Dubbo source code chapter 02---explore the principle of Wrapper mechanism from generalization call

---

What is a generalization call

Starting from the traditional three-tier architecture

For the traditional three-tier architecture, the Controller layer is responsible for receiving requests, the Service layer is responsible for processing requests related to business logic, and the Dao layer is responsible for interacting with the database, cooperating with the Model model object to carry business data, passing it in the request context, and finally processing After the data is filled, it is rendered by the View view:

However, in the microservice scenario, RPC remote call requests will inevitably be involved in the Service layer, so the above flow chart becomes as follows:

The above is a case of rpc remote call, and another case is as follows:

In this case, our Controller layer code may be as follows:

@RestController
public class UserController {
    
    
    // 响应码为成功时的值
    public static final String SUCC = "000000";
    
    // 定义访问下游查询用户服务的字段
    @DubboReference
    private UserQueryFacade userQueryFacade;
    
    // 定义URL地址
    @PostMapping("/queryUserInfo")
    public String queryUserInfo(@RequestBody QueryUserInfoReq req){
    
    
        // 将入参的req转为下游方法的入参对象，并发起远程调用
        QueryUserInfoResp resp = 
                userQueryFacade.queryUserInfo(convertReq(req));
        
        // 判断响应对象的响应码，不是成功的话，则组装失败响应
        if(!SUCC.equals(resp.getRespCode())){
    
    
            return RespUtils.fail(resp);
        }
        
        // 如果响应码为成功的话，则组装成功响应
        return RespUtils.ok(resp);
    }
}

In this case, our web server just writes code to package the data, and then sends it to the downstream system. After receiving the content returned by the downstream system, it does nothing and directly returns it to the front end. At this time, the essence of the web server is It's doing something transparent.

Here we only write one interface, but if the controller layer logic of many interfaces is simple transparent transmission, can this logic be extracted into general logic?

---

Reflective calls try to optimize

Let's first try to use reflection to extract the public part of the transparent transmission logic:

• Pass in the service service interface to be called, and the name of the service interface to be called, and then obtain the corresponding Method object through reflection
• Serialize the request parameters into a JSON string, and then convert the JSON string into an input object of the downstream interface through deserialization
• Initiate a real remote call through method.invoke reflection and get the response object
• Take the respCode response code from the response object through the Ognl expression language to make a judgment, and finally return

@RestController
public class UserController {
    
    
    // 响应码为成功时的值
    public static final String SUCC = "000000";
    
    // 定义访问下游查询用户服务的字段
    @DubboReference
    private UserQueryFacade userQueryFacade;
    
    // 定义URL地址
    @PostMapping("/queryUserInfo")
    public String queryUserInfo(@RequestBody QueryUserInfoReq req){
    
    
        // 调用公共方法
        return commonInvoke(userQueryFacade, "queryUserInfo", req);
    }
    
    /**
     * 模拟公共的远程调用方法
     * @param reqObj：下游的接口的实例对象，即通过 @DubboReference 得到的对象。
     * @param mtdName：下游接口的方法名。
     * @param reqParams：需要请求到下游的数据。
     * @return 直接结果数据。
     */
    public static String commonInvoke(Object reqObj, String mtdName, Object reqParams) throws InvocationTargetException, IllegalAccessException {
    
    
        // 通过反射找到 reqObj(例：userQueryFacade) 中的 mtdName(例：queryUserInfo) 方法
        Method reqMethod = ReflectionUtils.findMethod(reqObj.getClass(), mtdName);
        // 并设置查找出来的方法可被访问
        ReflectionUtils.makeAccessible(reqMethod);
        
        // 通过序列化工具将 reqParams 序列化为字符串格式
        String reqParamsStr = JSON.toJSONString(reqParams);
        // 然后再将 reqParamsStr 反序列化为下游对象格式，并反射调用 invoke 方法
        Object resp =  reqMethod.invoke(reqObj, JSON.parseObject(reqParamsStr, reqMethod.getParameterTypes()[0]));
        
        // 判断响应对象的响应码，不是成功的话，则组装失败响应
        if(resp == null || !SUCC.equals(OgnlUtils.getValue(resp, "respCode"))){
    
    
            return RespUtils.fail(resp);
        }
        // 如果响应码为成功的话，则组装成功响应
        return RespUtils.ok(resp);
    }
}

• OGNL (Object-Graph Navigation Language) is an expression language-based Java object graph navigation tool that can be used to simplify object navigation and manipulation in Java applications. OGNL provides a simple yet powerful set of syntax and operators to easily navigate in the Java object graph, access object properties, call methods, and more.

• OGNL was originally developed for the Struts framework for direct access to Java objects in JSP pages. However, due to its flexibility and power, OGNL has become a commonly used tool in many Java frameworks and applications, such as Spring framework, Hibernate
  ORM framework, etc.

• Here are some examples of common OGNL expressions:

  • Access object properties: user.name
  • Call object method: user.getName()
  • Operation collection elements: users.{? #this.age >18}.name
  • Operate array elements: items[0].name
  • Use logical operators: user.age > 18 && user.gender == "male"
  • Use conditional operator: user.age > 18 ? “adult” : “minor”

• Advantages of OGNL include:

  • Simple syntax: OGNL uses an expression language similar to Java, which is easy to understand and use.
  • Easy navigation: OGNL can easily navigate in the Java object graph, access object properties, call methods, etc.
  • Powerful: OGNL provides a set of simple and powerful syntax and operators, which can complete many complex object operations.
  • Extensibility: OGNL can be easily extended and customized to meet different application requirements.

• Overall, OGNL is a powerful, easy-to-use Java object navigation tool that can make the development of Java applications easier and more efficient.

Although we have extracted the common logic of the remote method call process through reflection, making the logic inside a single controller interface much simpler, but we still need to define many request interfaces like queryUserInfo, so is there a way to use a unified request interface as Entrance address, how to process all transparent interface requests uniformly?

---

generalization call

To use a unified request interface as the entry address is actually similar to the idea of ​​DispatchServlet uniformly intercepting and processing all servlet requests, and then DispatcherServlet dispatches them to each controller for request processing according to routing rules: our idea here is to write

a The unified secondary control processor intercepts all requests, initiates an RPC request call according to the general logic encapsulated above, and then returns the remote call result.

The above code modification idea is as follows:

• Define a common secondary control handler CommonController
• Define a unified URL path /gateway/{className}/{mtdName}/request, and use className and mtdName as placeholders for the request path
• Modify the format definition of the request business parameter, and convert the String from the object
• In the original CommonInvoke logic, use the class loader to load the service call interface corresponding to ClassName, and then find a way to find the instance object corresponding to ClassName

@RestController
public class CommonController {
    
    
    // 响应码为成功时的值
    public static final String SUCC = "000000";
    
    // 定义URL地址
    @PostMapping("/gateway/{className}/{mtdName}/request")
    public String commonRequest(@PathVariable String className,
                                @PathVariable String mtdName,
                                @RequestBody String reqBody){
    
    
        // 将入参的req转为下游方法的入参对象，并发起远程调用
        return commonInvoke(className, mtdName, reqBody);
    }
    
    /**
     * 模拟公共的远程调用方法
     * @param className：下游的接口归属方法的全类名。
     * @param mtdName：下游接口的方法名。
     * @param reqParamsStr：需要请求到下游的数据。
     * @return 直接返回下游的整个对象。
     */
    public static String commonInvoke(String className, 
                                      String mtdName, 
                                      String reqParamsStr) throws InvocationTargetException, IllegalAccessException, ClassNotFoundException {
    
    
        // 试图从类加载器中通过类名获取类信息对象
        Class<?> clz = CommonController.class.getClassLoader().loadClass(className);
        // 然后试图通过类信息对象想办法获取到该类对应的实例对象
        Object reqObj = tryFindBean(clz.getClass());
        
        // 通过反射找到 reqObj(例：userQueryFacade) 中的 mtdName(例：queryUserInfo) 方法
        Method reqMethod = ReflectionUtils.findMethod(clz, mtdName);
        // 并设置查找出来的方法可被访问
        ReflectionUtils.makeAccessible(reqMethod);
        
        // 将 reqParamsStr 反序列化为下游对象格式，并反射调用 invoke 方法
        Object resp =  reqMethod.invoke(reqObj, JSON.parseObject(reqParamsStr, reqMethod.getParameterTypes()[0]));
        
        // 判断响应对象的响应码，不是成功的话，则组装失败响应
        if(!SUCC.equals(OgnlUtils.getValue(resp, "respCode"))){
    
    
            return RespUtils.fail(resp);
        }
        // 如果响应码为成功的话，则组装成功响应
        return RespUtils.ok(resp);
    }
}

The key question now is in the tryFindBean method, how do we get the instance object of the service call interface? In other words, how to follow the @DubboReference annotation to get the instance object of the service call?

• At this point, we need to use the generalized call feature provided by Dubbo, that is, when the caller does not have the service call interface provided by the server, call the server and get the call result normally.

---

How to use generalized calls

Environment preparation:

• exposed service interface

public interface HelloService {
    
    
    String sayHello(String arg);
}

• Provide the specific implementation class of the service interface, and at the same time need to implement GenericService, expressed as a generalized call

public class GenericImplOfHelloService implements GenericService,HelloService{
    
    
    @Override
    public Object $invoke(String method, String[] parameterTypes, Object[] args) throws GenericException {
    
    
        System.out.println("进行泛化调用");
        if(method.equals("sayHello")){
    
    
            return "泛化调用结果: "+sayHello(args[0].toString());
        }
        return null;
    }

    @Override
    public String sayHello(String arg) {
    
    
        return "hello "+arg;
    }
}

Steps for the service provider to use the API to use generalized calls:

1. ServiceConfigWhen setting , use setGeneric("true")to enable generalization calls
2. ServiceConfigWhen setting , when using setRef to specify the implementation class, an GenericServiceobject of is to be set. instead of the real service implementation class object
3. Other settings are consistent with normal Api service startup

• Service Provider Full Code

    @Test
    void genericProviderTest() throws InterruptedException {
    
    
        //创建ApplicationConfig
        ApplicationConfig applicationConfig = new ApplicationConfig();
        applicationConfig.setName("generic-impl-provider");
        //创建注册中心配置
        RegistryConfig registryConfig = new RegistryConfig();
        registryConfig.setAddress("zookeeper://8.134.144.48:2181");

        //新建服务实现类，注意要使用GenericService接收
        GenericService helloService = new GenericImplOfHelloService();

        //创建服务相关配置
        ServiceConfig<GenericService> service = new ServiceConfig<>();
        service.setApplication(applicationConfig);
        service.setRegistry(registryConfig);
        service.setInterface("dubbo.dubboSpi.HelloService");
        service.setRef(helloService);
        //重点：设置为泛化调用
        //注：不再推荐使用参数为布尔值的setGeneric函数
        //应该使用referenceConfig.setGeneric("true")代替
        service.setGeneric("true");
        service.export();

        System.out.println("dubbo service started");

        new CountDownLatch(1).await();
    }

Steps for the service consumer to use the API to use generalized calls:

1. ReferenceConfigWhen setting , use setGeneric("true")to enable generalization calls
2. ReferenceConfigAfter configuration , use referenceConfig.get()to get the instance of GenericServicethe class
3. Use its $invokemethod to get the result
4. Other settings are consistent with normal Api service startup

• Service consumer complete code

    @Test
    void genericConsumerTest() throws InterruptedException {
    
    
        //创建ApplicationConfig
        ApplicationConfig applicationConfig = new ApplicationConfig();
        applicationConfig.setName("generic-call-consumer");
        //创建注册中心配置
        RegistryConfig registryConfig = new RegistryConfig();
        registryConfig.setAddress("zookeeper://8.134.144.48:2181");
        //创建服务引用配置
        ReferenceConfig<GenericService> referenceConfig = new ReferenceConfig<>();
        //设置接口
        referenceConfig.setInterface("dubbo.dubboSpi.HelloService");
        applicationConfig.setRegistry(registryConfig);
        referenceConfig.setApplication(applicationConfig);
        //重点：设置为泛化调用
        //注：不再推荐使用参数为布尔值的setGeneric函数
        //应该使用referenceConfig.setGeneric("true")代替
        referenceConfig.setGeneric(true);
        //设置异步，不必须，根据业务而定。
        referenceConfig.setAsync(true);
        //设置超时时间
        referenceConfig.setTimeout(7000);

        //获取服务，由于是泛化调用，所以获取的一定是GenericService类型
        GenericService genericService = referenceConfig.get();

        //使用GenericService类对象的$invoke方法可以代替原方法使用
        //第一个参数是需要调用的方法名
        //第二个参数是需要调用的方法的参数类型数组，为String数组，里面存入参数的全类名。
        //第三个参数是需要调用的方法的参数数组，为Object数组，里面存入需要的参数。
        Object result = genericService.$invoke("sayHello", new String[]{
    
    "java.lang.String"}, new Object[]{
    
    "world"});
        //使用CountDownLatch，如果使用同步调用则不需要这么做。
        CountDownLatch latch = new CountDownLatch(1);
        //获取结果
        CompletableFuture<String> future = RpcContext.getContext().getCompletableFuture();
        future.whenComplete((value, t) -> {
    
    
            System.err.println("invokeSayHello(whenComplete): " + value);
            latch.countDown();
        });
        //由于开启了异步模式,此处打印应该为null
        System.err.println("invokeSayHello(return): " + result);
        //打印结果
        latch.await();
    }

• test
  

---

Using generic calls through Spring

There are many ways to use service exposure and service discovery in Spring, such as xml and annotations. Here is an example of annotations. step:

1. No changes are required on the producer side
2. The original @DubboReferenceannotation specifiesinterfaceClass和generic=true

public interface UserService {
    
    
    User getUserById(String id);
}

public class UserServiceImpl implements UserService {
    
    
    @Override
    public User getUserById(String id) {
    
    
        // Do something to get the user by id
        return user;
    }
}

@Service
public class UserServiceImpl implements UserService {
    
    
    @DubboReference(interfaceClass = UserService.class, generic = true)
    private GenericService genericService;

    @Override
    public User getUserById(String id) {
    
    
        Object result = genericService.$invoke("getUserById", new String[]{
    
    "java.lang.String"}, new Object[]{
    
    id});
        return (User) result;
    }
}

---

Retrofit existing services with generic calls

The core class for the Dubbo consumer to initiate a remote call is ReferenceConfig. The next thing to do is to get the generic object GenericService returned by referenceConfig#get, and then call the GenericService#$invoke method to make a remote call.

@RestController
public class CommonController {
    
    
    // 响应码为成功时的值
    public static final String SUCC = "000000";
    
    // 定义URL地址
    @PostMapping("/gateway/{className}/{mtdName}/{parameterTypeName}/request")
    public String commonRequest(@PathVariable String className,
                                @PathVariable String mtdName,
                                @PathVariable String parameterTypeName,
                                @RequestBody String reqBody){
    
    
        // 将入参的req转为下游方法的入参对象，并发起远程调用
        return commonInvoke(className, parameterTypeName, mtdName, reqBody);
    }
    
    /**
     * 模拟公共的远程调用方法
     * @param className：下游的接口归属方法的全类名。
     * @param mtdName：下游接口的方法名。
     * @param parameterTypeName：下游接口的方法入参的全类名。
     * @param reqParamsStr：需要请求到下游的数据。
     * @return 直接返回下游的整个对象。
     */
    public static String commonInvoke(String className,
                                      String mtdName,
                                      String parameterTypeName,
                                      String reqParamsStr) {
    
    
        // 然后试图通过类信息对象想办法获取到该类对应的实例对象
        ReferenceConfig<GenericService> referenceConfig = createReferenceConfig(className);
        
        // 远程调用
        GenericService genericService = referenceConfig.get();
        Object resp = genericService.$invoke(
                mtdName,
                new String[]{
    
    parameterTypeName},
                new Object[]{
    
    JSON.parseObject(reqParamsStr, Map.class)});
        
        // 判断响应对象的响应码，不是成功的话，则组装失败响应
        if(!SUCC.equals(OgnlUtils.getValue(resp, "respCode"))){
    
    
            return RespUtils.fail(resp);
        }
        
        // 如果响应码为成功的话，则组装成功响应
        return RespUtils.ok(resp);
    }
    
    private static ReferenceConfig<GenericService> createReferenceConfig(String className) {
    
    
        DubboBootstrap dubboBootstrap = DubboBootstrap.getInstance();
        
        // 设置应用服务名称
        ApplicationConfig applicationConfig = new ApplicationConfig();
        applicationConfig.setName(dubboBootstrap.getApplicationModel().getApplicationName());
        
        // 设置注册中心的地址
        String address = dubboBootstrap.getConfigManager().getRegistries().iterator().next().getAddress();
        RegistryConfig registryConfig = new RegistryConfig(address);
        ReferenceConfig<GenericService> referenceConfig = new ReferenceConfig<>();
        referenceConfig.setApplication(applicationConfig);
        referenceConfig.setRegistry(registryConfig);
        referenceConfig.setInterface(className);
        
        // 设置泛化调用形式
        referenceConfig.setGeneric("true");
        // 设置默认超时时间5秒
        referenceConfig.setTimeout(5 * 1000);
        return referenceConfig;
    }
}

• The URL address adds a dimension of the method parameter class name, which means that the background provider can be accessed through the class name, method name, and method parameter class name;
• Create a ReferenceConfig object through the interface class name, and set the core property of generic = true;
• Get the genericService generic object through the referenceConfig.get method;
• Pass the method name, method parameter class name, and business request parameters into the $invoke method of the generalization object for remote Dubbo calls, and return the response object;
• Take the respCode response code judgment from the response object through the Ognl expression language and make the final return

---

Generalization call summary

Generalized call refers to calling the server without the API (SDK) provided by the server, and the call result can be obtained normally.

Generalization uses a unified way to initiate calls to any service method, at least we know it is a way of calling an interface, but this way has a unique name.

What are the application scenarios of generalized calls?

The generalization call is mainly used to implement a general remote service mock framework, which can handle all service requests by implementing the GenericService interface. For example, the following scenario:

• Gateway service: If you want to build a gateway service, then the service gateway should be the calling end of all RPC services. But the gateway itself should not depend on the interface API of the service provider (this will cause the code of the gateway to be modified and redeployed every time a new service is released), so the support of generalized calls is required.

• Test platform: If you want to build a platform that can test RPC calls, the user can test the corresponding RPC service by inputting information such as group name, interface, and method name. Then, for the same reason (that is, every time a new service is released, the code of the gateway needs to be modified and redeployed), the platform itself should not depend on the interface API of the service provider. So support for generalized calls is needed.

For details, you can read the official document: Generalization call (client generalization)

---

Wrapper mechanism

So far, we have used the generalization call to transform the service consumer into a general gateway service, but how does the service provider handle the generalization request?

• The generalization request will carry the interface class name, interface method name, interface method parameter class name, and business request parameters. These four dimensional fields initiate a remote call.
• Service provider services need to receive requests in a unified portal, and then dispatch them to different interface services.

If you want to implement coding for this unified entry, how would you write it?

The easiest way to think of it is to obtain the class object corresponding to the interface class name through the reflection mechanism, and then use the class object to get the corresponding bean from the IOC container, and use the interface method name and interface method parameters to accurately locate the provider interface service which method to process.

@RestController
public class CommonController {
    
    
    // 定义统一的URL地址
    @PostMapping("gateway/{className}/{mtdName}/{parameterTypeName}/request")
    public String recvCommonRequest(@PathVariable String className,
                                    @PathVariable String mtdName,
                                    @PathVariable String parameterTypeName,
                                    @RequestBody String reqBody) throws Exception {
    
    
        // 统一的接收请求的入口
        return commonInvoke(className, parameterTypeName, mtdName, reqBody);
    }

    /**
     * 统一入口的核心逻辑
     *
     * @param className：接口归属方法的全类名。
     * @param mtdName：接口的方法名。
     * @param parameterTypeName：接口的方法入参的全类名。
     * @param reqParamsStr：请求数据。
     * @return 接口方法调用的返回信息。
     */
    public static String commonInvoke(String className,
                                      String mtdName,
                                      String parameterTypeName,
                                      String reqParamsStr) throws Exception {
    
    
        // 通过反射机制可以获取接口类名对应的类对象
        Class<?> clz = Class.forName(className);
        // 接着通过类对象从IOC容器中定位对应的bean实例
        Object cacheObj = SpringCtxUtils.getBean(clz);
        // 通过反射找到方法对应的 Method 对象,并调用执行
        Class<?> methodParamType = Class.forName(parameterTypeName);
        Method method = clz.getDeclaredMethod(mtdName,methodParamType);
        method.setAccessible(true);
        return (String) method.invoke(cacheObj, JSON.parseObject(reqParamsStr,methodParamType));
    }
}

It is very simple to implement reflection calls, but the problem is that reflection calls are time-consuming. Dubbo, as a high-performance call framework, does not use reflection to achieve it. The highest performance implementation should be to directly call the method of the target object, as follows Show:

// 来精准定位需要提供方接口服务中的哪个方法进行处理
if ("sayHello".equals(mtdName) && String.class.getName().equals(parameterTypeName)) {
    
    
    // 真正的发起对源对象（被代理对象）的方法调用
    return ((DemoFacade) cacheObj).sayHello(reqParamsStr);
} else if("say".equals(mtdName) && Void.class.getName().equals(parameterTypeName)){
    
    
    // 真正的发起对源对象（被代理对象）的方法调用
    return ((DemoFacade) cacheObj).say();
}

Obviously, we can't directly move this logic into the commonInvoke method, because we can't use if...else to hard-code all the cases inside the commonInvoke method, which is really unreasonable!

The solution is to move the above logic to the corresponding service provider implementation class, that is, each service implementation class inherits the GenericService interface provided by Dubbo:

public interface GenericService {
    
    
    Object $invoke(String method, String[] parameterTypes, Object[] args) throws GenericException;
    ...
}

Hardcode enumeration of all possible invocations in the $invoke method of the interface, as follows:

public class GenericImplOfHelloService implements GenericService,HelloService{
    
    
    @Override
    public Object $invoke(String method, String[] parameterTypes, Object[] args) throws GenericException {
    
    
        if(method.equals("sayHello")){
    
    
            return "泛化调用结果: "+sayHello(args[0].toString());
        }
        return null;
    }

    @Override
    public String sayHello(String arg) {
    
    
        return "hello "+arg;
    }
}

Using the generalized interface provided by Dubbo to transform the above code, the result is as follows:

@RestController
public class CommonController {
    
    
    // 定义统一的URL地址
    @PostMapping("gateway/{className}/{mtdName}/{parameterTypeName}/request")
    public Object recvCommonRequest(@PathVariable String className,
                                    @PathVariable String mtdName,
                                    @PathVariable String parameterTypeName,
                                    @RequestBody String reqBody) throws Exception {
    
    
        // 统一的接收请求的入口
        return commonInvoke(className, parameterTypeName, mtdName, reqBody);
    }

    /**
     * 统一入口的核心逻辑
     *
     * @param className：接口归属方法的全类名。
     * @param mtdName：接口的方法名。
     * @param parameterTypeName：接口的方法入参的全类名。
     * @param reqParamsStr：请求数据。
     * @return 接口方法调用的返回信息。
     */
    public static Object commonInvoke(String className,
                                      String mtdName,
                                      String parameterTypeName,
                                      String reqParamsStr) throws Exception {
    
    
        // 通过反射机制可以获取接口类名对应的类对象
        Class<?> clz = Class.forName(className);
        // 接着通过类对象的简称获取到对应的接口服务
        GenericService genericService = SpringCtxUtils.getBean(clz);
        // 调用泛化接口的invoke方法
        return genericService.$invoke(mtdName,new String[]{
    
    parameterTypeName},new Object[]{
    
    JSON.parseObject(reqParamsStr,Class.forName(parameterTypeName))});
    }
}

At present, everything seems to be perfect. The only imperfection is that each service implementation class of the service provider needs to implement the GenericService interface, rewrite the invoke method, and hard-code the relevant invocation logic inside the method.

In fact, we can use dynamic proxies to extract the hard-coded repetitive logic above. Dynamic proxies are commonly used with JDK dynamic proxies and Cglib dynamic proxies. Here, JDK dynamic proxies are first excluded, because JDK dynamic proxies also use reflection calls.

The core principle of Cglib is to match the correct method from many method references of the proxy class by executing the callback method of the interceptor (methodProxy.invokeSuper), and execute the method of the proxy class.

This method of Cglib is like dynamically generating a bunch of if...else statements inside the proxy class to call the method of the proxy class, avoiding the hard-coded logic of manual writing of various if...else, saving a lot of hard work. Coding live.

But Dubbo does not use cglib to implement dynamic proxy, because the core implementation of Cglib is to generate various if...else codes to call the methods of the proxy class, but the logic of generating the proxy is not flexible enough and difficult to modify independently.

Dubbo adopts independent implementation based on the idea of ​​Cglib, and does not use the reflection mechanism, to create a simplified version of the mini Cglib proxy tool, so that you can do various logic closely related to the framework in your own proxy tool .

---

custom proxy

Since you want to generate the proxy class yourself, you must first code according to a code template. Let's design the code template:

public interface HelloService {
    
    
    String sayHello(String arg,String name);
    String test();
}


public class HelloServiceImpl implements HelloService{
    
    
    @Override
    public String sayHello(String arg, String name) {
    
    
        return "hello "+arg+" "+name;
    }

    @Override
    public String test() {
    
    
        return "test";
    }
}


//代理类模板
public class $GenericImplOfHelloService_1 extends HelloServiceImpl implements GenericService {
    
    
    public java.lang.Object $invoke(String method, String[] parameterTypes, Object[] args) throws GenericException {
    
    
        if ("test".equals(method) && (parameterTypes == null || parameterTypes != null && parameterTypes.length == 0)) {
    
    
            return test();
        }
        if ("sayHello".equals(method) && (parameterTypes != null && parameterTypes.length == 2 && parameterTypes[0].equals("java.lang.String") && parameterTypes[1].equals("java.lang.String"))) {
    
    
            return sayHello((java.lang.String) args[0], (java.lang.String) args[1]);
        }
        throw new GenericException(new NoSuchMethodException("Method [" + method + "] not found."));
    }
} 

With the code template, we write and generate it in Java language against the code template:

package com.provider.wrapperDemo;
import org.apache.dubbo.rpc.service.GenericException;
import org.apache.dubbo.rpc.service.GenericService;


import javax.tools.JavaCompiler;
import javax.tools.JavaFileObject;
import javax.tools.StandardJavaFileManager;
import javax.tools.ToolProvider;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.lang.reflect.Constructor;
import java.lang.reflect.Method;
import java.net.URL;
import java.net.URLClassLoader;
import java.util.Objects;
import java.util.concurrent.atomic.AtomicInteger;

public class CustomInvokerProxyUtils {
    
    
    private static final String WRAPPER_PACKAGE_NAME = "dubbo.dubboSpi";
    private static final AtomicInteger INC = new AtomicInteger();


    /**
     * 创建源对象（被代理对象）的代理对象
     */
    public static GenericService newProxyInstance(Object sourceTarget) throws Exception {
    
    
        // 代理类文件保存的磁盘路径
        String filePath = getWrapperBasePath();
        // 获取服务接口类
        Class<?> targetClazz = sourceTarget.getClass().getInterfaces()[0];
        // 生成的代理类名称:  $GenericImplOfHelloService_1
        String proxyClassName = "$GenericImplOf" + targetClazz.getSimpleName() + "_" + INC.incrementAndGet();
        // 获取代理的字节码内容
        String proxyByteCode = getProxyByteCode(proxyClassName, targetClazz, sourceTarget.getClass());
        // 缓存至磁盘中
        file2Disk(filePath, proxyClassName, proxyByteCode);
        // 等刷盘稳定后
        Thread.sleep(2000);
        // 再编译java加载class至内存中,返回实例化的对象
        return (GenericService) compileJava2Class(filePath, proxyClassName);
    }

    private static String getWrapperBasePath() {
    
    
        return Objects.requireNonNull(CustomInvokerProxyUtils.class.getResource("/")).getPath()
                + CustomInvokerProxyUtils.class.getPackage().toString().substring("package ".length()).replaceAll("\\.", "/");
    }

    /**
     * 生成代理的字节码内容，其实就是一段类代码的字符串
     */
    private static String getProxyByteCode(String proxyClassName, Class<?> targetClazz, Class<?> sourceClass) {
    
    
        StringBuilder sb = new StringBuilder();
        // java文件第一行是package声明包路径
        String pkgContent = "package " + CustomInvokerProxyUtils.WRAPPER_PACKAGE_NAME + ";";
        //通过import导入代理类中可能会使用到的类
        String importTargetClazzImpl = "import " + sourceClass.getName() + ";";
        String importGenericService = "import " + GenericService.class.getName() + ";";
        String importGenericException = "import " + GenericException.class.getName() + ";";
        String importNoSuchMethodException="import "+ NoSuchMethodException.class.getName()+";";
        // 代理类主体内容构建
        String classHeadContent = "public class " + proxyClassName + " extends " + sourceClass.getSimpleName() + " implements " + GenericService.class.getSimpleName() + " {";
        // 添加内容
        sb.append(pkgContent).append(importTargetClazzImpl).append(importGenericService).append(importGenericException)
                .append(importNoSuchMethodException).append(classHeadContent);
        // 构建invoke方法
        String invokeMethodHeadContent = "public " + Object.class.getName() + " $invoke" +
                "(" + String.class.getSimpleName() + " method, "
                + String.class.getSimpleName() + "[] parameterTypes, "
                + Object.class.getSimpleName() + "[] args) throws " + GenericException.class.getSimpleName() + " {\n";
        sb.append(invokeMethodHeadContent);
        // 组装if...else...逻辑
        for (Method method : targetClazz.getMethods()) {
    
    
            String methodName = method.getName();
            Class<?>[] parameterTypes = method.getParameterTypes();
            String ifHead = "if (\"" + methodName + "\".equals(method)"+buildMethodParamEqual(parameterTypes)+") {\n";
            //组装方法调用逻辑
            String ifContent = buildMethodInvokeContent(methodName, parameterTypes);
            String ifTail = "}\n";
            sb.append(ifHead).append(ifContent).append(ifTail);
        }
        // throw new GenericException("Method [" + method + "] not found.");
        String invokeMethodTailContent = "throw new " + GenericException.class.getSimpleName() + "(new NoSuchMethodException(\"Method [\" + method + \"] not found.\"));\n}\n";
        sb.append(invokeMethodTailContent);
        // 类的尾巴大括号
        String classTailContent = " } ";
        sb.append(classTailContent);
        return sb.toString();
    }

    private static String buildMethodParamEqual(Class<?>[] parameterTypes) {
    
    
        StringBuilder methodParamEqualContent=new StringBuilder();
        methodParamEqualContent.append("&&(");
        //方法参数为0,则可以传入null
        if(parameterTypes.length==0){
    
    
            methodParamEqualContent.append("parameterTypes==null ||");
        }
        //参数类型必须合法
        methodParamEqualContent.append(" parameterTypes!=null&&parameterTypes.length==").append(parameterTypes.length);
        for (int i = 0; i < parameterTypes.length; i++) {
    
    
            methodParamEqualContent.append("&&parameterTypes[").append(i).append("].equals(\"").append(parameterTypes[i].getName()).append("\")");
        }
        methodParamEqualContent.append(")");
        return methodParamEqualContent.toString();
    }

    private static String buildMethodInvokeContent(String methodName, Class<?>[] parameterTypes) {
    
    
        if (parameterTypes.length == 0) {
    
    
            return "return " + methodName + "();\n";
        }
        StringBuilder methodInvokeContent = new StringBuilder();
        methodInvokeContent.append("return ").append(methodName).append("(");
        for (int i = 0; i < parameterTypes.length; i++) {
    
    
            methodInvokeContent.append("(").append(parameterTypes[i].getName()).append(")")
                    .append("args[").append(i).append("]").append(",");
        }
        //删除最后一个多余的,
        methodInvokeContent.delete(methodInvokeContent.length()-1,methodInvokeContent.length());
        methodInvokeContent.append(");\n");
        return methodInvokeContent.toString();
    }

    private static void file2Disk(String filePath, String proxyClassName, String proxyByteCode) throws IOException {
    
    
        File file = new File(filePath + File.separator + proxyClassName + ".java");
        if (!file.exists()) {
    
    
            file.createNewFile();
        }
        FileWriter fileWriter = new FileWriter(file);
        fileWriter.write(proxyByteCode);
        fileWriter.flush();
        fileWriter.close();
    }

    private static Object compileJava2Class(String filePath, String proxyClassName) throws Exception {
    
    
        // 编译 Java 文件
        JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
        StandardJavaFileManager fileManager = compiler.getStandardFileManager(null, null, null);
        Iterable<? extends JavaFileObject> compilationUnits =
                fileManager.getJavaFileObjects(new File(filePath + File.separator + proxyClassName + ".java"));
        JavaCompiler.CompilationTask task = compiler.getTask(null, fileManager, null, null, null, compilationUnits);
        task.call();
        fileManager.close();
        // 加载 class 文件
        URL[] urls = new URL[]{
    
    new URL("file:" + filePath)};
        URLClassLoader urlClassLoader = new URLClassLoader(urls);

        Class<?> clazz = urlClassLoader.loadClass(CustomInvokerProxyUtils.WRAPPER_PACKAGE_NAME + "." + proxyClassName);
        // 反射创建对象，并且实例化对象
        Constructor<?> constructor = clazz.getConstructor();
        return constructor.newInstance();
    }
}

The generated code has three main steps:

• According to the appearance of the code template, a code string is dynamically generated using Java code.
• Save the generated code string to disk.
• Compile the file into a class file according to the disk file path, then use URLClassLoader to load it into the memory to become a Class object, and finally create and instantiate the object through reflection.

Note: If the following exception is thrown

 Caused by: java.lang.ClassNotFoundException: com.sun.tools.javac.processing.JavacProcessingEnvironment

It may be because of the lack of tools.jar dependency, you can introduce related dependencies in maven:

<dependency>
  <groupId>com.sun</groupId>
  <artifactId>tools</artifactId>
  <version>1.8.0</version>
  <scope>system</scope>
  <systemPath>${java.home}/../lib/tools.jar</systemPath>
</dependency>

Note that the system scope will make Maven not download this dependency from the remote repository, but use the specified systemPath instead. You need to modify the value of systemPath according to your environment, make sure it points to the actual path of tools.jar.

test:

    @Test
    void wrapperDemoTest() throws Exception {
    
    
        GenericService genericService = CustomInvokerProxyUtils.newProxyInstance(new HelloServiceImpl());
        Object testMethodRes = genericService.$invoke("test", null, null);
        System.out.println("test method invoke res: "+testMethodRes);
        Object sayHelloRes = genericService.$invoke("sayHello", new String[]{
    
    String.class.getName(),String.class.getName()}, new Object[]{
    
    "参数", "大忽悠"});
        System.out.println("sayHello method invoke res: "+sayHelloRes);
    }

Generated java file:

---

The underlying wrapper principle of dubbo

The GenericService generalization interface implemented above through the proxy class is inspired by the code of the service provider in the Dubbo official document generalization call (client generalization) section. Dubbo’s official implementation idea is similar, but the proxy class does not implement the GenericService interface. , let's take a look below.

The proxy class created by dubbo inherits the Wrapper interface, and the implementation is analogous to the GenericService given above:

// org.apache.dubbo.rpc.proxy.javassist.JavassistProxyFactory#getInvoker
// 创建一个 Invoker 的包装类
@Override
public <T> Invoker<T> getInvoker(T proxy, Class<T> type, URL url) {
    
    
    // 这里就是生成 Wrapper 代理对象的核心一行代码
    final Wrapper wrapper = Wrapper.getWrapper(proxy.getClass().getName().indexOf('$') < 0 ? proxy.getClass() : type);
    // 包装一个 Invoker 对象
    return new AbstractProxyInvoker<T>(proxy, type, url) {
    
    
        @Override
        protected Object doInvoke(T proxy, String methodName,
                                  Class<?>[] parameterTypes,
                                  Object[] arguments) throws Throwable {
    
    
            // 使用 wrapper 代理对象调用自己的 invokeMethod 方法
            // 以此来避免反射调用引起的性能开销
            // 通过强转来实现统一方法调用
            return wrapper.invokeMethod(proxy, methodName, parameterTypes, arguments);
        }
    };
}

Use the Wrapper provided by Dubbo to create a proxy object and perform method call demonstration:

    @Test
    void dubboWrapperTest() throws InvocationTargetException {
    
    
        HelloService helloService = new HelloServiceImpl();
        final Wrapper wrapper = Wrapper.getWrapper(helloService.getClass());
        Object testMethodRes = wrapper.invokeMethod(helloService, "test", new Class[]{
    
    },null);
        System.out.println("test method invoke res: " + testMethodRes);
        Object sayHelloRes = wrapper.invokeMethod(helloService,"sayHello",new Class[]{
    
    String.class, String.class}, new Object[]{
    
    "参数", "大忽悠"});
        System.out.println("sayHello method invoke res: " + sayHelloRes);
    }

We decompile the generated wrapper proxy class file into Java code to see what the generated content looks like. Here we need to use the Arthas tool provided by Ali to complete:

1. Download Arthas: curl -O https://arthas.aliyun.com/arthas-boot.jar
2. Start the test case and call the System.in.read() method at the end of the test method to suspend the current thread
3. Start Arthas: java -jar arthas-boot.jar
   
4. Fuzzy search all proxy classes generated by dubbo

5. View the complete code of the corresponding proxy class

package dubbo.dubboSpi;

import dubbo.dubboSpi.HelloServiceImpl;
import java.lang.reflect.InvocationTargetException;
import java.util.Map;
import org.apache.dubbo.common.bytecode.ClassGenerator;
import org.apache.dubbo.common.bytecode.NoSuchMethodException;
import org.apache.dubbo.common.bytecode.NoSuchPropertyException;
import org.apache.dubbo.common.bytecode.Wrapper;

public class HelloServiceImplDubboWrap0 extends Wrapper implements ClassGenerator.DC {
    
    
    public static String[] pns;
    public static Map pts;
    public static String[] mns;
    public static String[] dmns;
    public static Class[] mts0;
    public static Class[] mts1;

    @Override
    public String[] getPropertyNames() {
    
    
        return pns;
    }

    @Override
    public boolean hasProperty(String string) {
    
    
        return pts.containsKey(string);
    }

    public Class getPropertyType(String string) {
    
    
        return (Class)pts.get(string);
    }

    @Override
    public String[] getMethodNames() {
    
    
        return mns;
    }

    @Override
    public String[] getDeclaredMethodNames() {
    
    
        return dmns;
    }

    @Override
    public void setPropertyValue(Object object, String string, Object object2) {
    
    
        try {
    
    
            HelloServiceImpl helloServiceImpl = (HelloServiceImpl)object;
        }
        catch (Throwable throwable) {
    
    
            throw new IllegalArgumentException(throwable);
        }
        throw new NoSuchPropertyException(new StringBuffer().append("Not found property \"").append(string).append("\" field or setter method in class dubbo.dubboSpi.HelloServiceImpl.").toString());
    }

    @Override
    public Object getPropertyValue(Object object, String string) {
    
    
        try {
    
    
            HelloServiceImpl helloServiceImpl = (HelloServiceImpl)object;
        }
        catch (Throwable throwable) {
    
    
            throw new IllegalArgumentException(throwable);
        }
        throw new NoSuchPropertyException(new StringBuffer().append("Not found property \"").append(string).append("\" field or getter method in class dubbo.dubboSpi.HelloServiceImpl.").toString());
    }

    public Object invokeMethod(Object object, String string, Class[] classArray, Object[] objectArray) throws InvocationTargetException {
    
    
        HelloServiceImpl helloServiceImpl;
        try {
    
    
            helloServiceImpl = (HelloServiceImpl)object;
        }
        catch (Throwable throwable) {
    
    
            throw new IllegalArgumentException(throwable);
        }
        try {
    
    
            if ("sayHello".equals(string) && classArray.length == 2) {
    
    
                return helloServiceImpl.sayHello((String)objectArray[0], (String)objectArray[1]);
            }
            if ("test".equals(string) && classArray.length == 0) {
    
    
                return helloServiceImpl.test();
            }
        }
        catch (Throwable throwable) {
    
    
            throw new InvocationTargetException(throwable);
        }
        throw new NoSuchMethodException(new StringBuffer().append("Not found method \"").append(string).append("\" in class dubbo.dubboSpi.HelloServiceImpl.").toString());
    }
}

In the dynamic proxy class generated by dubbo above, we only need to focus on the logic of the invokeMethod method for the time being. It can be seen that it is actually consistent with the logic we mentioned above.

The source code of the wrapper class will not be expanded in this article, and will be further analyzed in subsequent chapters.

---

summary

We started from the service provider's design of a unified entrance, from reflection call transformation, to trying hard-coded performance, which led to a custom dynamic proxy. Although the implementation logic of the Cglib proxy meets the requirements of the transformation, but for custom generation of proxy classes Flexible requirements are also limited by the Cglib library.

Therefore, after considering the appeal factors, dubbo has customized a mini Cglib proxy tool. The overall implementation idea is as follows:

1. First design a set of common code templates to make it universal for business scenarios and facilitate unified agency
2. Generate a set of dynamic code according to the requirements of the code template by handwriting java code or through bytecode tools
3. Finally, compile the dynamically generated code through JDK or bytecode tools, and finally find a way to generate Class objects
4. Then create an instance with the generated Class object, and use the instance object to make method calls

---

summary

This article refers to the source code analysis section provided by Dubbo's official website, and integrates personal understanding. If there is any error, please point it out in the comment area, or discuss it with me in a private message.