I. Overview
1. Background
Currently, there are many governance tasks in the data governance service. When any of the governance tasks needs to be upgraded or a governance task is added, the data governance service needs to be restarted, which will affect the normal operation of other governance tasks.
2. Target
- Ability to dynamically start and stop any management task
- Ability to dynamically upgrade and add governance tasks
- Starting, stopping governance tasks or upgrading, adding governance tasks cannot affect other tasks
3. Program
- In order to support the decoupling of business code as much as possible, some business functions are loaded into the main program through dynamic loading to meet pluggable loading and combined deployment.
- Cooperate with the xxl-job task scheduling framework, and register the data governance tasks as xxl-job tasks in xxl-job to facilitate unified management.
2. Dynamic loading
1. Custom class loader
URLClassLoader is a special class loader that can load classes and resources from a specified URL. Its main function is to dynamically load external JAR packages or class files, so as to realize the function of dynamically extending the application. In order to facilitate the management of dynamically loaded jar packages, the custom class loader inherits URLClassloader.
package cn.jy.sjzl.util;
import java.lang.reflect.Method;
import java.net.URL;
import java.net.URLClassLoader;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
/**
* 自定义类加载器
*
* @author lijianyu
* @date 2023/04/03 17:54
**/
public class MyClassLoader extends URLClassLoader {
private Map<String, Class<?>> loadedClasses = new ConcurrentHashMap<>();
public Map<String, Class<?>> getLoadedClasses() {
return loadedClasses;
}
public MyClassLoader(URL[] urls, ClassLoader parent) {
super(urls, parent);
}
@Override
protected Class<?> findClass(String name) throws ClassNotFoundException {
// 从已加载的类集合中获取指定名称的类
Class<?> clazz = loadedClasses.get(name);
if (clazz != null) {
return clazz;
}
try {
// 调用父类的findClass方法加载指定名称的类
clazz = super.findClass(name);
// 将加载的类添加到已加载的类集合中
loadedClasses.put(name, clazz);
return clazz;
} catch (ClassNotFoundException e) {
e.printStackTrace();
return null;
}
}
public void unload() {
try {
for (Map.Entry<String, Class<?>> entry : loadedClasses.entrySet()) {
// 从已加载的类集合中移除该类
String className = entry.getKey();
loadedClasses.remove(className);
try{
// 调用该类的destory方法，回收资源
Class<?> clazz = entry.getValue();
Method destory = clazz.getDeclaredMethod("destory");
destory.invoke(clazz);
} catch (Exception e ) {
// 表明该类没有destory方法
}
}
// 从其父类加载器的加载器层次结构中移除该类加载器
close();
} catch (Exception e) {
e.printStackTrace();
}
}
}
- In the custom class loader, in order to facilitate the unloading of classes, a map is defined to save the loaded class information . The key is the ClassName of this class, and the value is the class information of this class.
- At the same time, the unloading method of the class loader is defined. In the unloading method, the class is removed from the collection of loaded classes. Because this class may use system resources or calling threads, in order to avoid memory overflow caused by unreclaimed resources, call the destroy method in this class through reflection to reclaim resources.
- Finally call the close method.
2. Dynamic loading
Since this project uses the spring framework and the mechanism of the xxl-job task calls dynamically loaded code, the following needs to be done
- Read the dynamically loaded jar package into memory
- Classes with spring annotations will be scanned and manually added to the spring container through annotation scanning.
- Manually add the @XxlJob annotated method to the xxljob executor through annotation scanning.
package com.jy.dynamicLoad;
import com.jy.annotation.XxlJobCron;
import com.jy.classLoader.MyClassLoader;
import com.xxl.job.core.executor.impl.XxlJobSpringExecutor;
import com.xxl.job.core.handler.annotation.XxlJob;
import com.xxl.job.core.handler.impl.MethodJobHandler;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.InitializingBean;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.beans.factory.support.AbstractBeanDefinition;
import org.springframework.beans.factory.support.BeanDefinitionBuilder;
import org.springframework.beans.factory.support.DefaultListableBeanFactory;
import org.springframework.context.ApplicationContext;
import org.springframework.core.MethodIntrospector;
import org.springframework.core.annotation.AnnotatedElementUtils;
import org.springframework.stereotype.Component;
import java.io.File;
import java.io.IOException;
import java.lang.reflect.Method;
import java.net.JarURLConnection;
import java.net.URL;
import java.net.URLConnection;
import java.util.Enumeration;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.jar.JarEntry;
import java.util.jar.JarFile;
/**
* @author lijianyu
* @date 2023/04/29 13:18
**/
@Component
public class DynamicLoad {
private static Logger logger = LoggerFactory.getLogger(DynamicLoad.class);
@Autowired
private ApplicationContext applicationContext;
private Map<String, MyClassLoader> myClassLoaderCenter = new ConcurrentHashMap<>();
@Value("${dynamicLoad.path}")
private String path;
/**
* 动态加载指定路径下指定jar包
* @param path
* @param fileName
* @param isRegistXxlJob 是否需要注册xxljob执行器，项目首次启动不需要注册执行器
* @return map<jobHander, Cron> 创建xxljob任务时需要的参数配置
*/
public void loadJar(String path, String fileName, Boolean isRegistXxlJob) throws ClassNotFoundException, InstantiationException, IllegalAccessException {
File file = new File(path +"/" + fileName);
Map<String, String> jobPar = new HashMap<>();
// 获取beanFactory
DefaultListableBeanFactory beanFactory = (DefaultListableBeanFactory) applicationContext.getAutowireCapableBeanFactory();
// 获取当前项目的执行器
try {
// URLClassloader加载jar包规范必须这么写
URL url = new URL("jar:file:" + file.getAbsolutePath() + "!/");
URLConnection urlConnection = url.openConnection();
JarURLConnection jarURLConnection = (JarURLConnection)urlConnection;
// 获取jar文件
JarFile jarFile = jarURLConnection.getJarFile();
Enumeration<JarEntry> entries = jarFile.entries();
// 创建自定义类加载器，并加到map中方便管理
MyClassLoader myClassloader = new MyClassLoader(new URL[] {
url }, ClassLoader.getSystemClassLoader());
myClassLoaderCenter.put(fileName, myClassloader);
Set<Class> initBeanClass = new HashSet<>(jarFile.size());
// 遍历文件
while (entries.hasMoreElements()) {
JarEntry jarEntry = entries.nextElement();
if (jarEntry.getName().endsWith(".class")) {
// 1. 加载类到jvm中
// 获取类的全路径名
String className = jarEntry.getName().replace('/', '.').substring(0, jarEntry.getName().length() - 6);
// 1.1进行反射获取
myClassloader.loadClass(className);
}
}
Map<String, Class<?>> loadedClasses = myClassloader.getLoadedClasses();
XxlJobSpringExecutor xxlJobExecutor = new XxlJobSpringExecutor();
for(Map.Entry<String, Class<?>> entry : loadedClasses.entrySet()){
String className = entry.getKey();
Class<?> clazz = entry.getValue();
// 2. 将有@spring注解的类交给spring管理
// 2.1 判断是否注入spring
Boolean flag = SpringAnnotationUtils.hasSpringAnnotation(clazz);
if(flag){
// 2.2交给spring管理
BeanDefinitionBuilder builder = BeanDefinitionBuilder.genericBeanDefinition(clazz);
AbstractBeanDefinition beanDefinition = builder.getBeanDefinition();
// 此处beanName使用全路径名是为了防止beanName重复
String packageName = className.substring(0, className.lastIndexOf(".") + 1);
String beanName = className.substring(className.lastIndexOf(".") + 1);
beanName = packageName + beanName.substring(0, 1).toLowerCase() + beanName.substring(1);
// 2.3注册到spring的beanFactory中
beanFactory.registerBeanDefinition(beanName, beanDefinition);
// 2.4允许注入和反向注入
beanFactory.autowireBean(clazz);
beanFactory.initializeBean(clazz, beanName);
/*if(Arrays.stream(clazz.getInterfaces()).collect(Collectors.toSet()).contains(InitializingBean.class)){
initBeanClass.add(clazz);
}*/
initBeanClass.add(clazz);
}
// 3. 带有XxlJob注解的方法注册任务
// 3.1 过滤方法
Map<Method, XxlJob> annotatedMethods = null;
try {
annotatedMethods = MethodIntrospector.selectMethods(clazz,
new MethodIntrospector.MetadataLookup<XxlJob>() {
@Override
public XxlJob inspect(Method method) {
return AnnotatedElementUtils.findMergedAnnotation(method, XxlJob.class);
}
});
} catch (Throwable ex) {
}
// 3.2 生成并注册方法的JobHander
for (Map.Entry<Method, XxlJob> methodXxlJobEntry : annotatedMethods.entrySet()) {
Method executeMethod = methodXxlJobEntry.getKey();
// 获取jobHander和Cron
XxlJobCron xxlJobCron = executeMethod.getAnnotation(XxlJobCron.class);
if(xxlJobCron == null){
throw new CustomException("500", executeMethod.getName() + "()，没有添加@XxlJobCron注解配置定时策略");
}
if (!CronExpression.isValidExpression(xxlJobCron.value())) {
throw new CustomException("500", executeMethod.getName() + "(),@XxlJobCron参数内容错误");
}
XxlJob xxlJob = methodXxlJobEntry.getValue();
jobPar.put(xxlJob.value(), xxlJobCron.value());
if (isRegistXxlJob) {
executeMethod.setAccessible(true);
// regist
Method initMethod = null;
Method destroyMethod = null;
xxlJobExecutor.registJobHandler(xxlJob.value(), new CustomerMethodJobHandler(clazz, executeMethod, initMethod, destroyMethod));
}
}
}
// spring bean实际注册
initBeanClass.forEach(beanFactory::getBean);
} catch (IOException e) {
logger.error("读取{} 文件异常", fileName);
e.printStackTrace();
throw new RuntimeException("读取jar文件异常: " + fileName);
}
}
}
The following is a tool class for judging whether the class has spring annotations
apublic class SpringAnnotationUtils {
private static Logger logger = LoggerFactory.getLogger(SpringAnnotationUtils.class);
/**
* 判断一个类是否有 Spring 核心注解
*
* @param clazz 要检查的类
* @return true 如果该类上添加了相应的 Spring 注解；否则返回 false
*/
public static boolean hasSpringAnnotation(Class<?> clazz) {
if (clazz == null) {
return false;
}
//是否是接口
if (clazz.isInterface()) {
return false;
}
//是否是抽象类
if (Modifier.isAbstract(clazz.getModifiers())) {
return false;
}
try {
if (clazz.getAnnotation(Component.class) != null ||
clazz.getAnnotation(Repository.class) != null ||
clazz.getAnnotation(Service.class) != null ||
clazz.getAnnotation(Controller.class) != null ||
clazz.getAnnotation(Configuration.class) != null) {
return true;
}
}catch (Exception e){
logger.error("出现异常：{}",e.getMessage());
}
return false;
}
}
The operation of registering the xxljob executor is modeled after the registration method of XxlJobSpringExecutor in xxljob.
3. Dynamic uninstallation
The process of dynamic unloading is to remove the dynamically loaded code from memory, spring and xxljob.
code show as below:
/**
* 动态卸载指定路径下指定jar包
* @param fileName
* @return map<jobHander, Cron> 创建xxljob任务时需要的参数配置
*/
public void unloadJar(String fileName) throws IllegalAccessException, NoSuchFieldException {
// 获取加载当前jar的类加载器
MyClassLoader myClassLoader = myClassLoaderCenter.get(fileName);
// 获取jobHandlerRepository私有属性,为了卸载xxljob任务
Field privateField = XxlJobExecutor.class.getDeclaredField("jobHandlerRepository");
// 设置私有属性可访问
privateField.setAccessible(true);
// 获取私有属性的值jobHandlerRepository
XxlJobExecutor xxlJobSpringExecutor = new XxlJobSpringExecutor();
Map<String, IJobHandler> jobHandlerRepository = (ConcurrentHashMap<String, IJobHandler>) privateField.get(xxlJobSpringExecutor);
// 获取beanFactory，准备从spring中卸载
DefaultListableBeanFactory beanFactory = (DefaultListableBeanFactory) applicationContext.getAutowireCapableBeanFactory();
Map<String, Class<?>> loadedClasses = myClassLoader.getLoadedClasses();
Set<String> beanNames = new HashSet<>();
for (Map.Entry<String, Class<?>> entry: loadedClasses.entrySet()) {
// 1. 将xxljob任务从xxljob执行器中移除
// 1.1 截取beanName
String key = entry.getKey();
String packageName = key.substring(0, key.lastIndexOf(".") + 1);
String beanName = key.substring(key.lastIndexOf(".") + 1);
beanName = packageName + beanName.substring(0, 1).toLowerCase() + beanName.substring(1);
// 获取bean，如果获取失败，表名这个类没有加到spring容器中，则跳出本次循环
Object bean = null;
try{
bean = applicationContext.getBean(beanName);
}catch (Exception e){
// 异常说明spring中没有这个bean
continue;
}
// 1.2 过滤方法
Map<Method, XxlJob> annotatedMethods = null;
try {
annotatedMethods = MethodIntrospector.selectMethods(bean.getClass(),
new MethodIntrospector.MetadataLookup<XxlJob>() {
@Override
public XxlJob inspect(Method method) {
return AnnotatedElementUtils.findMergedAnnotation(method, XxlJob.class);
}
});
} catch (Throwable ex) {
}
// 1.3 将job从执行器中移除
for (Map.Entry<Method, XxlJob> methodXxlJobEntry : annotatedMethods.entrySet()) {
XxlJob xxlJob = methodXxlJobEntry.getValue();
jobHandlerRepository.remove(xxlJob.value());
}
// 2.0从spring中移除，这里的移除是仅仅移除的bean，并未移除bean定义
beanNames.add(beanName);
beanFactory.destroyBean(beanName, bean);
}
// 移除bean定义
Field mergedBeanDefinitions = beanFactory.getClass()
.getSuperclass()
.getSuperclass().getDeclaredField("mergedBeanDefinitions");
mergedBeanDefinitions.setAccessible(true);
Map<String, RootBeanDefinition> rootBeanDefinitionMap = ((Map<String, RootBeanDefinition>) mergedBeanDefinitions.get(beanFactory));
for (String beanName : beanNames) {
beanFactory.removeBeanDefinition(beanName);
// 父类bean定义去除
rootBeanDefinitionMap.remove(beanName);
}
// 卸载父任务，子任务已经在循环中卸载
jobHandlerRepository.remove(fileName);
// 3.2 从类加载中移除
try {
// 从类加载器底层的classes中移除连接
Field field = ClassLoader.class.getDeclaredField("classes");
field.setAccessible(true);
Vector<Class<?>> classes = (Vector<Class<?>>) field.get(myClassLoader);
classes.removeAllElements();
// 移除类加载器的引用
myClassLoaderCenter.remove(fileName);
// 卸载类加载器
myClassLoader.unload();
} catch (NoSuchFieldException e) {
logger.error("动态卸载的类，从类加载器中卸载失败");
e.printStackTrace();
} catch (IllegalAccessException e) {
logger.error("动态卸载的类，从类加载器中卸载失败");
e.printStackTrace();
}
logger.error("{} 动态卸载成功", fileName);
}
4. Dynamic configuration
When using dynamic loading, in order to avoid losing the loaded task package after the service is restarted, the dynamic configuration method is used to dynamically update the initial loading configuration after loading.
The following provides two configuration methods that I have actually operated.
4.1 Dynamically modify local yml
To dynamically modify the local yml configuration file, you need to add the dependency of snakeyaml
4.1.1 Dependency introduction
<dependency>
<groupId>org.yaml</groupId>
<artifactId>snakeyaml</artifactId>
<version>1.29</version>
</dependency>
4.1.2 Tools
Read the configuration file under the specified path and modify it.
package com.jy.util;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.env.Environment;
import org.springframework.stereotype.Component;
import org.yaml.snakeyaml.DumperOptions;
import org.yaml.snakeyaml.Yaml;
import java.io.*;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
/**
* 用于动态修改bootstrap.yml配置文件
* @author lijianyu
* @date 2023/04/18 17:57
**/
@Component
public class ConfigUpdater {
public void updateLoadJars(List<String> jarNames) throws IOException {
// 读取bootstrap.yml
Yaml yaml = new Yaml();
InputStream inputStream = new FileInputStream(new File("src/main/resources/bootstrap.yml"));
Map<String, Object> obj = yaml.load(inputStream);
inputStream.close();
obj.put("loadjars", jarNames);
// 修改
FileWriter writer = new FileWriter(new File("src/main/resources/bootstrap.yml"));
DumperOptions options = new DumperOptions();
options.setDefaultFlowStyle(DumperOptions.FlowStyle.BLOCK);
options.setPrettyFlow(true);
Yaml yamlWriter = new Yaml(options);
yamlWriter.dump(obj, writer);
}
}
4.2 Dynamically modify nacos configuration
The Spring Cloud Alibaba Nacos component fully supports dynamic configuration modification through code at runtime, and also provides some APIs for developers to implement dynamic configuration modification in the code. Every time the data governance task jar package is dynamically loaded or unloaded, the nacos configuration will be dynamically updated after the execution is successful.
package cn.jy.sjzl.config;
import com.alibaba.nacos.api.NacosFactory;
import com.alibaba.nacos.api.config.ConfigService;
import com.alibaba.nacos.api.exception.NacosException;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Configuration;
import java.util.Properties;
@Configuration
public class NacosConfig {
@Value("${spring.cloud.nacos.server-addr}")
private String serverAddr;
@Value("${spring.cloud.nacos.config.namespace}")
private String namespace;
public ConfigService configService() throws NacosException {
Properties properties = new Properties();
properties.put("serverAddr", serverAddr);
properties.put("namespace", namespace);
return NacosFactory.createConfigService(properties);
}
}
package cn.jy.sjzl.util;
import cn.jy.sjzl.config.NacosConfig;
import com.alibaba.fastjson.JSONObject;
import com.alibaba.nacos.api.config.ConfigService;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.dataformat.yaml.YAMLMapper;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;
import java.util.ArrayList;
import java.util.List;
import java.util.stream.Collectors;
/**
* nacos配置中，修改sjzl-loadjars.yml
*
* @author lijianyu
* @date 2023/04/19 17:59
**/
@Component
public class NacosConfigUtil {
private static Logger logger = LoggerFactory.getLogger(NacosConfigUtil.class);
@Autowired
private NacosConfig nacosConfig;
private String dataId = "sjzl-loadjars.yml";
@Value("${spring.cloud.nacos.config.group}")
private String group;
/**
* 从nacos配置文件中，添加初始化jar包配置
* @param jarName 要移除的jar包名
* @throws Exception
*/
public void addJarName(String jarName) throws Exception {
ConfigService configService = nacosConfig.configService();
String content = configService.getConfig(dataId, group, 5000);
// 修改配置文件内容
YAMLMapper yamlMapper = new YAMLMapper();
ObjectMapper jsonMapper = new ObjectMapper();
Object yamlObject = yamlMapper.readValue(content, Object.class);
String jsonString = jsonMapper.writeValueAsString(yamlObject);
JSONObject jsonObject = JSONObject.parseObject(jsonString);
List<String> loadjars;
if (jsonObject.containsKey("loadjars")) {
loadjars = (List<String>) jsonObject.get("loadjars");
}else{
loadjars = new ArrayList<>();
}
if (!loadjars.contains(jarName)) {
loadjars.add(jarName);
}
jsonObject.put("loadjars" , loadjars);
Object yaml = yamlMapper.readValue(jsonMapper.writeValueAsString(jsonObject), Object.class);
String newYamlString = yamlMapper.writeValueAsString(yaml);
boolean b = configService.publishConfig(dataId, group, newYamlString);
if(b){
logger.info("nacos配置更新成功");
}else{
logger.info("nacos配置更新失败");
}
}
}
3. Separation and packaging
When separating and packaging, modify the following configuration in pom.xml according to the actual situation
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.4</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<filters>
<filter>
<artifact>*:*</artifact>
<includes>
<include>com/jy/job/demo/**</include>
</includes>
</filter>
</filters>
<finalName>demoJob</finalName>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>