1. Current status and problems
Do you still remember your efforts to troubleshoot jar conflicts?
In order to effectively control the introduction and changes of unknown jars brought about by jar package updates, we often use dependency-tree to view dependencies and troubleshoot problems. Usually, problems occur and then passively analyze and troubleshoot. At this time, the labor cost is huge, and the system is out of date. Problem, no regret medicine.
2. Analyze the reasons
Jar package dependencies are mutated and invisible. Problems caused by jar conflicts often occur, and R&D cannot pay attention to its changes every time.
3. Take action
Adopt "agile" thinking, take small steps, and regularly monitor changes in jar package dependencies every day, so that risks can be pre-empted and unknown problems can be proactively revealed.
Technology solves problems, CI/CD capabilities reduce R&D costs, automatic execution at 23:00 every day, All R&D pays attention to jar doc change every day~
—— We use the dependency tree as a file for git version control, and at the same time maintain it on CI to automatically control changes in jar dependencies, so that changes in dependencies can be discovered immediately. The pipeline regularly triggers the scan of the dependency tree every day to ensure that it is up-to-date every day. When any changes are found, the doc changes are initiated immediately. When R & D pays attention to mr, you can check who changed what the previous day, and effectively manage the jar package.
4. Practical steps
4.1 Create Makefile
Root directory: doc/dependency-tree.txt empty file
Makefile
dependency-tree:
@mvn clean -U package -Dmaven.test.skip=true dependency:tree -Dverbose -DoutputFile=target/dependency-tree.txt --settings settings.xml
@grep -v 'omitted for' wms-outbound-web/target/dependency-tree.txt | grep -vw "tests" | grep -vw "test" | sed -e 's/TEST-SNAPSHOT/SNAPSHOT/g' > doc/dependency-tree.txt
@git add doc/dependency-tree.txt
@git commit -m "fix: [CI make dependency-tree]依赖树变更"
@git push origin HEAD:master
settings.xml
<?xml version="1.0" encoding="UTF-8"?>
<settings
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd"
xmlns="http://maven.apache.org/SETTINGS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<localRepository>./maven/repository</localRepository>
<profiles>
<profile>
<id>Repository</id>
<repositories>
<repository>
<id>nexus</id>
<name>local private nexus</name>
<url>***</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
<repository>
<snapshots>
<enabled>false</enabled>
</snapshots>
<id>central</id>
<name>libs-releases</name>
<url>***</url>
</repository>
<repository>
<snapshots>
<updatePolicy>always</updatePolicy>
</snapshots>
<id>snapshots</id>
<name>libs-snapshots</name>
<url>***</url>
</repository>
</repositories>
</profile>
</profiles>
<activeProfiles>
<activeProfile>***</activeProfile>
</activeProfiles>
</settings>
4.2 Modify the gitignore file
- gitignore add content
/maven
4.3 configure bamboo
Select a timing-triggered pipeline (master pipeline) configuration
Add an atom between the "Download Code" atom and the "Maven Build" atom: "Custom Script" (must be in this order)
Shell code block:
cd ${globalParams.system.APP_IDENTIFIER}
make
- Process control selection: failure to continue (reason: CI modification code requires mr review, so the review mechanism will cause push to fail, no problem)
4.4 configure coding
Add the master permission of xn_testdev_ci account, and add it to the protection branch list permission at the same time
5. Realize the effect
5.1 bamboo log
After running, you can see the log success, just push to initiate a review
5.2 coding MR records
You can view the mr initiated by the bamboo account "Test Development_Continuous Integration", just review it (only change the dependency tree file)
6. Performance improvement
From 2021/10/19 to now, this practice found 42 dependency changes, 7 of which found code problems (R&D has been processed in real time, otherwise each unknown dependency change will correspond to >1 R&D cost)
Efficiency quantification simulation: 2021/10/19~present
| Before the improvement (/person-day) | After efficiency improvement (/person-day) | |
|---|---|---|
| The jar package conflict problem occurs for the first time | 2 (It was discovered today that the problematic jar has been introduced for half a year, and the cost of manpower investigation is huge) | 0.1 (Anomalies have been found and processed in advance, the early cost is extremely low, and this conflict is avoided) |
| The jar package conflict problem occurs for the second time | 2.5 (Tomorrow, it will be found that the mvn dependency tree needs to be checked one by one, and it is found that the jar is introduced earlier and the cost is greater) | 0.5 (Conflicts occur immediately, analyze the git history of doc to directly locate and introduce changes) |
| The jar package conflict problem occurred for the third time | 3 (After many days, it was discovered that the timing of the introduction of the problematic jar could no longer be traced, and the dependency relationship was chaotic, so it could only be researched and developed to ask each other and recall) | 0.5 (same as above, doc git history positioning introduces changes) |
| … | … | … |
| The jar package conflict problem occurs more than n times, the total cost calculation | >2*n | <0.5*n |
7. Brief summary
[jar package conflict] is a problem for every code repo and every researcher!
- If we "can avoid problems and pre-empt risks", then "maintenance costs will inevitably be reduced and daily performance will be improved" in the later stage.
- Using the CI/CD mechanism, the jar package dependency tree is managed and controlled in git as a doc file, and a snapshot is recorded for each change, and the iteration is dismantled and iterated according to the "agile" idea (the cycle is 23:00 every day) to automatically scan dependencies, the earliest Find out and deal with it first, don't be passive anymore, take the initiative to attack!
Author: Zhou Yiru of JD Logistics
Source: Reprinted from Yuanqishuo Tech by JD Cloud developer community, please indicate the source