foreword
I don’t know if you have encountered such a scenario during the development process. The external project wants to access the jar of the internal nexus private warehouse. Because the private warehouse is not open to the public, the external project cannot be downloaded to the jar of the private warehouse, resulting in The project does not run due to missing jars.
Usually in such a scenario, the common solution is to connect the external project with the internal nexus network, such as through VPN. Or directly download the jar from the private warehouse and give it to an external project. For the second solution, sometimes because the jar in the private warehouse depends on other internal jars, multiple jars need to be downloaded. At this time, for convenience, we may merge these jars into one big jar and give it out. At present, some jars are starters, and there will be some configuration files with the same name, such as spring.factories. If you do not process it and package it directly, the configuration file with the same name will be overwritten.
This article is to talk about how to solve the situation of multiple configuration files with the same name being overwritten when multiple jars are merged into one jar
Solutions
Through the maven-shade-plugin plug-in, the org.apache.maven.plugins.shade.resource.AppendingTransformer of the plug-in is used to process the merging of configuration files with the same name in multiple jar packages. His core is to merge the content of multiple configuration files with the same name, not to overwrite
An example configuration is as follows
<build>
<plugins>
<!-- 防止同名配置文件，在打包时被覆盖，用来处理多个jar包中存在重名的配置文件的合并
参考dubbo：https://github.com/apache/dubbo/blob/master/dubbo-all/pom.xml-->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>META-INF/spring.factories</resource>
</transformer>
<transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>META-INF/spring.handlers</resource>
</transformer>
<transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>META-INF/spring.schemas</resource>
</transformer>
<transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>META-INF/spring.tooling</resource>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
The effect of the packaged configuration file is as follows
Sharp-eyed friends should have discovered that the configuration content of the same name is appended, but only appending, in fact, sometimes it cannot meet the requirements, such as the spring.factories file, the effect he needs to achieve should be as shown in the following figure. I passed maven-
shade -plugin's official example ( https://maven.apache.org/plugins/maven-shade-plugin/examples/resource-transformers.html ) tried to find a solution, but unfortunately, I couldn't find it. So there are two ways in front of me, one is to give up the maven-shade-plugin plug-in, such as choosing other similar plug-ins, such as maven-assembly-plugin, I tried this solution, and found that the maven-assembly-plugin plug-in Extended configuration is more complicated than maven-shade-plugin, so give up. Finally, I chose to expand on the basis of maven-shade-plugin.
Ideas for expansion
I did not directly modify the maven-shade-plugin plug-in, but customized the plug-in on the basis of the packaged maven-shade-plugin. The idea of realization is not difficult, that is, to modify the contents of the spring.factories file after the maven-shade-plugin is packaged into a jar, and the
Adjust the shape as follows
Custom maven plugin spring-factories-merge-plugin
core idea
1. How to read the repeated content of the key in the configuration file spring.factories without being overwritten
If you directly read java.util.properties, when there are duplicate keys in the configuration file, such as when there are multiple org.springframework.boot.autoconfigure.EnableAutoConfiguration, the value will be overwritten in the end.
Solution, we can use org.apacche.commons.configuration.PropertiesConfiguration for processing
Introduce GAV in the pom of the project
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-configuration2</artifactId>
<version>${commons-configuration2}</version>
</dependency>
Read configuration sample code
@SneakyThrows
public static Map<String, Set<String>> readFactoriesFile(InputStream input) {
// 读取 spring.factories 内容
//利用PropertiesConfiguration取配置文件中key重复的内容，而不被覆盖
PropertiesConfiguration properties = new PropertiesConfiguration();
properties.read(new InputStreamReader(input));
Map<String, Set<String>> multiSetMap = new LinkedHashMap<>();
Iterator<String> keys = properties.getKeys();
while(keys.hasNext()) {
String key = keys.next();
String[] values = properties.getStringArray(key);
Set<String> collectSet = new LinkedHashSet<>();
buildKeyValues(values, collectSet);
multiSetMap.put(key,collectSet);
}
return multiSetMap;
}
2. How to rewrite the modified configuration file into the jar
My idea here is to directly use IO to operate
The example is as follows
public static void writeFactoriesFile(String factoriesBaseClassPathDir,String finalJarName) throws IOException {
String jarFilePath = String.format(factoriesBaseClassPathDir + "/target/" + finalJarName).replace("\\", "/").replaceAll("//+", "/");
if(!jarFilePath.endsWith(".jar")){
jarFilePath = jarFilePath + ".jar";
}
JarFile jarFile = new JarFile(jarFilePath);
if(jarFile != null){
List<JarEntry> jarFiles = jarFile.stream().collect(Collectors.toList());
@ Cleanup FileOutputStream fos = new FileOutputStream(jarFile.getName(), true);
@ Cleanup JarOutputStream jos = new JarOutputStream(fos);
for (JarEntry jarEntry : jarFiles) {
if(jarEntry.getName().startsWith(SpringFactoriesLoader.FACTORIES_RESOURCE_LOCATION)){
try {
@ Cleanup InputStream input = jarFile.getInputStream(jarEntry);
Map<String, Set<String>> factoriesMap = readFactoriesFile(input);
jos.putNextEntry(new JarEntry(jarEntry.getName()));
generateFactoriesContent(factoriesMap,jos);
} catch (IOException e) {
e.printStackTrace();
}
}else{
//表示将该JarEntry写入jar文件中 也就是创建该文件夹和文件
jos.putNextEntry(new JarEntry(jarEntry));
jos.write(streamToByte(jarFile.getInputStream(jarEntry)));
}
}
}
}
How to configure plugins in the project
<build>
<plugins>
<!-- 防止同名配置文件，在打包时被覆盖，用来处理多个jar包中存在重名的配置文件的合并
参考dubbo：https://github.com/apache/dubbo/blob/master/dubbo-all/pom.xml-->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>META-INF/spring.factories</resource>
</transformer>
<transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>META-INF/spring.handlers</resource>
</transformer>
<transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>META-INF/spring.schemas</resource>
</transformer>
<transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>META-INF/spring.tooling</resource>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>com.github.lybgeek.jar</groupId>
<artifactId>spring-factories-merge-plugin</artifactId>
<version>0.0.1-SNAPSHOT</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>springFactoriesMerge</goal>
</goals>
</execution>
</executions>
<configuration>
<factoriesBaseClassPathDir>${
basedir}</factoriesBaseClassPathDir>
<finalJarName>${
project.artifactId}-${
project.version}</finalJarName>
</configuration>
</plugin>
</plugins>
</build>
A small detail here is that when the execution life cycle of maven-shade-plugin and spring-factories-merge-plugin are at the same stage, for example, when they are both in the package, the order of maven-shade-plugin must be placed in spring-factories Before -merge-plugin, because spring-factories-merge-plugin performs secondary processing on the packaged results of maven-shade-plugin. If maven-shade-plugin is not placed before spring-factories-merge-plugin, the execution phase of spring-factories-merge-plugin will be later than maven-shade-plugin, such as maven-shade-plugin in package stage execution, the spring-factories-merge-plugin has to be executed in the install or deploy stage
The effect picture after packaging is as follows
Summarize
When I was looking at open source frameworks before, I often focused on the source code instead of paying attention to some maven plug-ins. This time because of the need to make jars. I found that whether it is springboot or dubbo itself integrates some treasure plugins, such as this maven-shade-plugin plugin, I found it on dubbo, and the address is
https://github.com/apache/dubbo/blob/master/dubbo -all/pom.xml
. For example, the version placeholder plug-in flatten-maven-plugin is used in both dubbo and springboot. If there is a demand for maven plug-ins later, it is recommended to search from springboot or dubbo, and it is estimated that there will be unexpected gains
demo link
https://github.com/lyb-geek/springboot-learning/tree/master/springboot-jar-merge