5 super easy-to-use open source tool libraries to share~

In actual project development, from the perspective of stability and efficiency, reinventing the wheel is not advocated. However, building wheels in the learning process is definitely beneficial to yourself and not harmful. Building wheels is a means that can especially improve your system programming ability.

Today I share several open source tool libraries that I commonly use:

1. MyExcel: A full-featured Excel processing tool with low memory usage and excellent performance.

2. OSHI: A JNA -based (native) operating system and hardware repository for the Java language .

3. JsonPath: Java DSL for reading JSON

4. Caffeine: powerful local cache

5. Hutool: A complete Java tool library

The following is a more detailed introduction. It is recommended that friends read it so that they can get started quickly and use it in their own projects to improve production efficiency.

MyExcel: Excel processing

Project Introduction

Everyone should be familiar with EasyExcel. This is an Excel processing tool open sourced by Ali, which can avoid the memory overflow problem of large files. I have recommended this project several times before.

MyExcel is also an Excel processing tool, which can also avoid the memory overflow problem of large files, and has more comprehensive functions, lower memory usage, and better performance (according to the official test comparison results).

According to the test data in the project wiki:

24.3M Excel, 500,000 rows, 8 columns, 40 reading cycles, the average memory usage is about 75 megabytes, compared with the same file test of Ali EasyExcel (V2.1.6), the memory usage is about one-third of MyExcel

Memory usage comparison between MyExcel and EasyExcel

The core advantages of MyExcel are as follows:

Core Benefits of MyExcel

rely on information

Maven repository address: https://mvnrepository.com/artifact/com.github.liaochong/myexcel .

Maven ：

<dependency>
    <groupId>com.github.liaochong</groupId>
    <artifactId>myexcel</artifactId>
    <version>4.3.0.RC4</version>
</dependency>

Gradle ：

implementation 'com.github.liaochong:myexcel:4.3.0.RC4'

Demo

MyExcel has a lot of functions, here we only select individual introductions, for detailed introductions, please refer to the official wiki: https://github.com/liaochong/myexcel/wiki .

1. Stream export

MyExcel supports streaming export, adopts the producer-consumer mode, allows data to be obtained in batches, and has extremely low memory usage. In addition, streaming export supports unique features such as zip archives.

DefaultStreamExcelBuilder<ArtCrowd> streamExcelBuilder = DefaultStreamExcelBuilder
        .of(ArtCrowd.class) // 如导出Map类型数据，请使用of(Map.class)
        .threadPool(Executors.newFixedThreadPool(10))// 线程池，可选
        .templateHandler(FreemarkerTemplateHandler.class)// 追加模板数据，可选，适合极度个性化数据导出
        .capacity(10_000)// 容量设定，在主动划分excel使用，可选
        .start();

2. Custom style

MyExcel supports custom styles, including the adjustment of width, height, background color, border, font and other styles. For details, please refer to: https://github.com/liaochong/myexcel/wiki/Style-support .

Title (title) style customization:

ExcelColumn(style={"title->color:red","cell->color:green"})
Integer age;

Content row style customization:

@ExcelColumn(style="cell->color:green")
Integer age;

The method call sets the style:

DefaultExcelBuilder.of(ArtCrowd.class)
                   .style("title->color:red","background-color:green;")
                   .build(dataList);

3. Drop-down list

MyExcel supports generating drop-down lists, just pass in List or Array parameters.

@ExcelColumn(title="下拉列表")
List<String> options;

relevant address

Project address: https://github.com/liaochong/myexcel
Official document: https://github.com/liaochong/myexcel/wiki

OSHI: local machine information acquisition

Project Introduction

OSHI is a JNA -based (native) operating system and hardware repository for the Java language .

[JNA(Java Native Access)](https://github.com/java-native-access/jna "JNA(Java Native Access "JNA(Java Native Access)")") is an open source Java framework, is the Sun A technology for invoking local methods launched by the company is a framework based on the classic JNI. The reason why it is said to be a substitute for JNI is that JNA greatly simplifies the process of calling local methods, is very convenient to use, and basically can be completed without leaving the Java environment.

JNI (Java Native Interface) is the Java native interface, which builds a bridge between Java and other programming languages, allowing Java programs to call programs or code libraries written in other languages (especially C/C++). Moreover, the implementation of JDK itself also extensively uses JNI technology to call the local C library.

With OSHI, we don't need to install any other native libraries to view memory and CPU usage, disk and partition usage, devices, sensors, etc.

OSHI aims to provide a cross-platform implementation to retrieve system information, supporting mainstream operating systems such as Windows, Linux, MacOS, and Unix.

The official introduction of oshi is as follows: (translation Chrome plug-in: Mate Translate ):

Using oshi, you can easily create system monitoring functions commonly used in projects, as shown in the following figure:

rely on information

Maven warehouse address: https://mvnrepository.com/artifact/com.github.oshi/oshi-core .

Maven：

<dependency>
  <groupId>com.github.oshi</groupId>
  <artifactId>oshi-parent</artifactId>
  <version>6.4.1</version>
  <type>pom</type>
</dependency>

Gradle：

implementation 'com.github.oshi:oshi-core:6.4.1'

Demo

Get the hardware information object HardwareAbstractionLayer :

//系统信息
SystemInfo si = new SystemInfo();
//操作系统信息
OperatingSystem os = si.getOperatingSystem();
//硬件信息
HardwareAbstractionLayer hal = si.getHardware();
// 磁盘信息
List<HWDiskStore> diskStores = hal.getDiskStores();

Now that we have an object representing hardware information HardwareAbstractionLayer , we can get hardware-related information!

The following is a brief demonstration of obtaining memory and CPU related information.

1. Obtain memory-related information

//内存相关信息
GlobalMemory memory = hal.getMemory();
//获取内存总容量
String totalMemory = FormatUtil.formatBytes(memory.getTotal());
//获取可用内存的容量
String availableMemory = FormatUtil.formatBytes(memory.getAvailable());

With the total memory capacity and available memory capacity, you can calculate the current memory utilization.

2. Obtain CPU related information

//CPU相关信息
CentralProcessor processor = hal.getProcessor();
//获取CPU名字
String processorName = processor.getProcessorIdentifier().getName();
//获取物理CPU数
int physicalPackageCount = processor.getPhysicalPackageCount();
//获取物理核心数
int physicalProcessorCount = processor.getPhysicalProcessorCount();

relevant address

Project address: https://github.com/oshi/oshi
Official website: https://www.oshi.ooo/

JsonPath: A Java DSL for reading JSON

Project Introduction

JsonPath is a query language for JSON structures. Equivalent to XPATH for XML and SQL for relational databases, they are relatively common DSLs.

JsonPath provides implementations in multiple languages, including: Java, Javascript, Python, PHP, Ruby, Go, etc. Here we take the implementation of the Java version as an example.

rely on information

Maven repository address: https://mvnrepository.com/artifact/com.jayway.jsonpath/json-path .

Maven：

<dependency>
    <groupId>com.jayway.jsonpath</groupId>
    <artifactId>json-path</artifactId>
    <version>2.8.0</version>
</dependency>

Gradle：

implementation 'com.jayway.jsonpath:json-path:2.8.0'

Demo

The content of the test json document is as follows:

{
    "store": {
        "book": [
            {
                "category": "reference",
                "author": "Nigel Rees",
                "title": "Sayings of the Century",
                "price": 8.95
            },
            {
                "category": "fiction",
                "author": "Evelyn Waugh",
                "title": "Sword of Honour",
                "price": 12.99
            },
            {
                "category": "fiction",
                "author": "Herman Melville",
                "title": "Moby Dick",
                "isbn": "0-553-21311-3",
                "price": 8.99
            },
            {
                "category": "fiction",
                "author": "J. R. R. Tolkien",
                "title": "The Lord of the Rings",
                "isbn": "0-395-19395-8",
                "price": 22.99
            }
        ],
        "bicycle": {
            "color": "red",
            "price": 19.95
        }
    },
    "expensive": 10
}

To read the authors of all books:

List<String> authors = JsonPath.read(json, "$.store.book[*].author");

Read all books and filter:

// 所有价格低于 10 的书籍
List<Map<String, Object>> books =  JsonPath.parse(json)
                                     .read("$.store.book[?(@.price < 10)]");

// 也可以通过 Filter API 来进行过滤
Filter cheapFictionFilter = filter(
   where("category").is("fiction").and("price").lte(10D)
);

List<Map<String, Object>> books = JsonPath.parse(json).read("$.store.book[?]", cheapFictionFilter);

relevant address

Project address: https://github.com/json-path/JsonPath
Introduction to JSONPath: https://goessner.net/articles/JsonPath/

Caffeine: a powerful local cache

Project Introduction

Caffeine is my most commonly used local cache. It is similar to Caffeine ConcurrentMap, but it provides more comprehensive caching functions and powerful performance.

How good is the performance? A detailed answer has been given in the benchmark test of the official document, address: https://github.com/ben-manes/caffeine/wiki/Benchmarks .

The figure below shows the performance comparison of common local cache implementations when 8 threads perform concurrent reads and writes on a cache configured with the maximum capacity.

Concurrent read:

Common local cache concurrent read performance comparison

Concurrent writes:

Common local cache concurrent write performance comparison

In addition to basic caching functions, Caffeine also provides functions such as expiration and asynchronous loading.

Caffeine is considered to be the best choice for local caching. Well-known open source projects such as Redisson, Cassandra, Hbase, Neo4j, and Druid all use Caffeine.

rely on information

Maven repository address: https://mvnrepository.com/artifact/com.github.ben-manes.caffeine/caffeine .

Used by Java 11 and above 3.x , otherwise 2.x.

Maven：

<dependency>
    <groupId>com.github.ben-manes.caffeine</groupId>
    <artifactId>caffeine</artifactId>
    <version>3.1.5</version>
</dependency>

Gradle：

implementation 'com.github.ben-manes.caffeine:caffeine:3.1.5'

Demo

Caffeine and API are very similar to Guava, borrowing from Guava's design.

Create cache:

Cache<Key, Graph> cache = Caffeine.newBuilder()
    .expireAfterWrite(10, TimeUnit.MINUTES)
    .maximumSize(10_000)
    .build();

Remove the cache manually:

// 单个key
cache.invalidate(key)
// 批量key
cache.invalidateAll(keys)
// 所有key
cache.invalidateAll()

statistics:

// 统计缓存命中率、缓存回收数量、加载新值的平均时间
Cache<String, Object> cache = Caffeine.newBuilder()
    .maximumSize(10_000)
    .recordStats()
    .build();

relevant address

Project address: https://github.com/ben-manes/caffeine
Official documentation: https://github.com/ben-manes/caffeine/wiki

Hutool: A complete Java tool library

Project Introduction

Hutool is a very useful Java tool library, which encapsulates JDK methods such as files, streams, encryption and decryption, transcoding, regularization, threads, and XML.

Really is a good tool library with comprehensive functions, very suitable for use in your own projects.

The official introduction of Hutool is as follows:

Hutool Introduction

rely on information

Maven warehouse address: https://mvnrepository.com/artifact/cn.hutool/hutool-all .

Maven：

<dependency>
    <groupId>cn.hutool</groupId>
    <artifactId>hutool-all</artifactId>
    <version>5.8.16</version>
</dependency>

Gradle：

implementation 'cn.hutool:hutool-all:5.8.16'

Demo

A simple demonstration of a few functions that I personally find more practical.

type conversion

Convert Classes encapsulate conversions to common Java types.

long[] b = {1,2,3,4,5};
String bStr = Convert.toStr(b);//"[1, 2, 3, 4, 5]"

double a = 67556.32;
String digitUppercase = Convert.digitToChinese(a);//"陆万柒仟伍佰伍拾陆元叁角贰分"

mail

Sending emails in Java mainly depends on javax.mail the package, but because it is cumbersome to use, Hutool encapsulates it.

src/main/resourcesCreate a new file under the config directory of the classpath (in a standard Maven project ) mail.setting, and the complete configuration is as follows (the mail server must support and open the SMTP protocol):

# 邮件服务器的SMTP地址，可选，默认为smtp.<发件人邮箱后缀>
host = smtp.yeah.net
# 邮件服务器的SMTP端口，可选，默认25
port = 25
# 发件人（必须正确，否则发送失败）
from = [email protected]
# 用户名，默认为发件人邮箱前缀
user = hutool
# 密码（注意，某些邮箱需要为SMTP服务单独设置授权码，详情查看相关帮助）
pass = q1w2e3

Sending mail is very simple:

MailUtil.send("[email protected]", "测试", "邮件来自Hutool测试", false);

Support group sending:

ArrayList<String> tos = CollUtil.newArrayList(
    "[email protected]",
    "[email protected]",
    "[email protected]",
    "[email protected]");

MailUtil.send(tos, "测试", "邮件来自Hutool群发测试", false);

Support adding one or more attachments:

MailUtil.send("[email protected]", "测试", "<h1>邮件来自Hutool测试</h1>", true, FileUtil.file("d:/aaa.xml"));

In addition to using the configuration file to define the global account, MailUtil.sendthe method also provides an overloaded method to pass in an MailAccountobject, which is a common bean that records the mail server information.

MailAccount account = new MailAccount();
account.setHost("smtp.yeah.net");
account.setPort("25");
account.setAuth(true);
account.setFrom("[email protected]");
account.setUser("hutool");
account.setPass("q1w2e3");

MailUtil.send(account, CollUtil.newArrayList("[email protected]"), "测试", "邮件来自Hutool测试", false);

Unique ID

In a distributed environment, the unique ID generation is widely used, and the generation methods are also varied. Hutool makes a simple package for some common generation strategies.

The unique ID generator tool class provided by Hutool covers:

UUID
ObjectId（MongoDB）
Snowflake（Twitter）

Take UUID for example!

The logic rewritten by Hutool java.util.UUIDcorresponds to class cn.hutool.core.lang.UUID, so that the generation of UUID strings without - does not need to do character replacement, and the performance is improved by about one time .

//生成的UUID是带-的字符串，类似于：a5c8a5e8-df2b-4706-bea4-08d0939410e3
String uuid = IdUtil.randomUUID();

//生成的是不带-的字符串，类似于：b17f24ff026d40949c85a24f4f375d42
String simpleUUID = IdUtil.simpleUUID();

HTTP request tool class

For the most commonly used GET and POST requests, HttpUtil encapsulates two methods,

HttpUtil.get
HttpUtil.post

GET request:

// 最简单的HTTP请求，可以自动通过header等信息判断编码，不区分HTTP和HTTPS
String result1= HttpUtil.get("https://www.baidu.com");

// 当无法识别页面编码的时候，可以自定义请求页面的编码
String result2= HttpUtil.get("https://www.baidu.com", CharsetUtil.CHARSET_UTF_8);

//可以单独传入http参数，这样参数会自动做URL编码，拼接在URL中
HashMap<String, Object> paramMap = new HashMap<>();
paramMap.put("city", "北京");

String result3= HttpUtil.get("https://www.baidu.com", paramMap);

POST request:

HashMap<String, Object> paramMap = new HashMap<>();
paramMap.put("city", "北京");

String result= HttpUtil.post("https://www.baidu.com", paramMap);

File Upload:

HashMap<String, Object> paramMap = new HashMap<>();
//文件上传只需将参数中的键指定（默认file），值设为文件对象即可，对于使用者来说，文件上传与普通表单提交并无区别
paramMap.put("file", FileUtil.file("D:\\face.jpg"));

String result= HttpUtil.post("https://www.baidu.com", paramMap);

cache

Hutool provides the implementation of several common caching strategies:

FIFO(first in first out) : First in first out policy.
LFU(least frequently used) : The least frequently used strategy.
LRU(least recently used) : The strategy has not been used for the longest time.
Timed : Timing strategy.
Weak : Weak reference strategy.

Moreover, Hutool also supports caching small files byte[] into content in the form of , reducing file access, and solving performance problems caused by frequent file reading.

FIFO(first in first out) policy cache uses:

Cache<String,String> fifoCache = CacheUtil.newFIFOCache(3);

//加入元素，每个元素可以设置其过期时长，DateUnit.SECOND.getMillis()代表每秒对应的毫秒数，在此为3秒
fifoCache.put("key1", "value1", DateUnit.SECOND.getMillis() * 3);
fifoCache.put("key2", "value2", DateUnit.SECOND.getMillis() * 3);
fifoCache.put("key3", "value3", DateUnit.SECOND.getMillis() * 3);

//由于缓存容量只有3，当加入第四个元素的时候，根据FIFO规则，最先放入的对象将被移除
fifoCache.put("key4", "value4", DateUnit.SECOND.getMillis() * 3);

//value1为null
String value1 = fifoCache.get("key1");

Console print package

Under normal circumstances, we print information to the console, friends should be familiar with it!

System.out.println("Hello World");

However, this approach does not meet the needs of many scenarios:

Parameters are not supported, and object printing needs to concatenate strings
The array cannot be printed directly, it needs to be called manuallyArrays.toString

For this purpose, Hutool encapsulates Consoleobjects.

ConsoleThe use of objects is more similar to console.log()the method of Javascript, which is also a syntactic sugar borrowed from JS.

String[] a = {"java", "c++", "c"};
Console.log(a);//控制台输出：[java, c++, c]

Console.log("This is Console log for {}.", "test");//控制台输出：This is Console log for test.

encrypt and decode

Hutool supports symmetric encryption, asymmetric encryption, digest encryption, message authentication code algorithm, and national secrets.

Here we take the national secret algorithm as an example. Hutool has Bouncy Castlemade a simplified package for implementing SM2, SM3, and SM4 in the national secret algorithm.

The National Secret Algorithm needs to introduce Bouncy Castlelibrary dependencies:

<dependency>
    <groupId>org.bouncycastle</groupId>
    <artifactId>bcprov-jdk15to18</artifactId>
    <version>1.69</version>
</dependency>

SM2 encrypts or decrypts using a custom key pair :

String text = "JavaGuide:一份涵盖大部分 Java 程序员所需要掌握的核心知识。准备 Java 面试，首选 JavaGuide！";
System.out.println("原文：" + text);

KeyPair pair = SecureUtil.generateKeyPair("SM2");
// 公钥
byte[] privateKey = pair.getPrivate().getEncoded();
// 私钥
byte[] publicKey = pair.getPublic().getEncoded();

SM2 sm2 = SmUtil.sm2(privateKey, publicKey);
// 公钥加密，私钥解密
String encryptStr = sm2.encryptBcd(text, KeyType.PublicKey);
System.out.println("加密后：" + encryptStr);

String decryptStr = StrUtil.utf8Str(sm2.decryptFromBcd(encryptStr, KeyType.PrivateKey));
System.out.println("解密后：" + decryptStr);

SM2 signature and verification :

//加签
String sign = sm2.signHex(HexUtil.encodeHexStr(text));
System.out.println("签名：" + sign);
//验签
boolean verify = sm2.verifyHex(HexUtil.encodeHexStr(text), sign);
System.out.println("验签：" + verify);

Thread Pool

Hutool supports using the builder pattern to create a custom thread pool, which makes it easier to see.

private static ExecutorService pool = ExecutorBuilder.create()
              .setCorePoolSize(10)//初始池大小
              .setMaxPoolSize(20) //最大池大小
              .setWorkQueue(new LinkedBlockingQueue<>(100))//最大等待数为100
              .setThreadFactory(ThreadFactoryBuilder.create().setNamePrefix("IM-Pool-").build())// 线程池命名
              .build();

In actual projects, if an object has many attributes, limited consideration should be given to using the builder pattern to create the object.

Moreover, Hutool also provides a global thread pool, and all asynchronous methods are executed in this thread pool by default.

ThreadUtil.execute : Execute threads directly in the public thread pool
ThreadUtil.execAsync: Execute an asynchronous method
......

Hutool itself is widely used ThreadUtil, such as the sensitive word tool class SensitiveUtil:

public static void init(final Collection<String> sensitiveWords, boolean isAsync){
  if(isAsync){
    // 异步初始化敏感词树
    ThreadUtil.execAsync(new Callable<Boolean>(){
      @Override
      public Boolean call() throws Exception {
        init(sensitiveWords);
        return true;
      }

    });
  }else{
    // 同步初始化敏感词树
    init(sensitiveWords);
  }
}

relevant address

Project address: https://github.com/dromara/hutool
Official website: https://hutool.cn/

postscript

One of the advantages of Java is that it has a particularly good ecology, including many useful tool libraries and frameworks, covering almost all demand scenarios. We don't need to do many things from scratch at all. Using existing stable and reliable tool libraries can greatly improve development efficiency.

For example, for Excel document processing, you can consider the following open source tool libraries:

easyexcel[1] : A fast and simple Java processing Excel tool that avoids OOM.
excel-streaming-reader[2]: Excel streaming code style reading tool (only supports reading XLSX files), based on Apache POI package, while retaining the syntax of the standard POI API.
myexcel[3]: A toolkit that integrates multiple functions such as importing, exporting, and encrypting Excel.

Another example is PDF document processing:

pdfbox[4] : An open source Java tool for processing PDF documents. The project allows creating new PDF documents, operating on existing documents, and extracting content from documents. PDFBox also includes several command line utilities. PDFBox is released under the Apache version 2.0 license.
OpenPDF[5]: OpenPDF is a free Java library for creating and editing PDF files under the LGPL and MPL open source licenses. OpenPDF is based on a fork of iText.
itext7[6]: iText 7 represents a higher level sdk for developers who want to take advantage of PDF. Equipped with a better document engine, high-level and low-level programming features, and the ability to create, edit, and enhance PDF documents, iText 7 benefits nearly every workflow.
FOP[7] : The main output target of the Apache FOP project is PDF.

Some tool libraries commonly used in Java development are summarized on my website, which can be used as a reference: https://javaguide.cn/open-source-project/tool-library.html.

References

[1]

easyexcel: https://github.com/alibaba/easyexcel

[2]

excel-streaming-reader: https://github.com/monitorjbl/excel-streaming-reader

[3]

myexcel: https://github.com/liaochong/myexcel

[4]

pdfbox: https://github.com/apache/pdfbox

[5]

OpenPDF: https://github.com/LibrePDF/OpenPDF

[6]

itext7: https://github.com/itext/itext7

[7]

FOP: https://xmlgraphics.apache.org/fop/