Article directory
Introduction to MinIO and springBoot integration
What is MinIO?
Minio is an open source object storage suite written based on Golang. It is based on the Apache License v2.0 open source agreement. Although it is lightweight, it has good performance. It is compatible with Amazon S3 cloud storage service interface. It can be easily used in combination with other applications, such as NodeJS, Redis, MySQL, etc.
Application scenarios
In addition to being used as an object storage service for private clouds, MinIO's application scenarios can also be used as a gateway layer for cloud object storage, seamlessly connecting Amazon S3or MicroSoft Azure.
)]
Features
-
High performance : As a high-performance storage, under standard hardware conditions, its read and write rates can reach
55Gb/sand respectively35Gb/s. And MinIO supports that an object file can be of any size, ranging from a few kb to a maximum of 5T. -
Scalable : Different MinIO clusters can form a federation and form a global namespace, and support spanning multiple data centers.
-
Cloud native : containerization, K8S-based orchestration, multi-tenant support.
-
Amazon S3 Compatible : Uses Amazon S3 v2/v4 API. Minio server can be accessed using Minio SDK, Minio Client, AWS SDK and AWS CLI.
-
SDK support :
-
- GO SDK:https://github.com/minio/minio-go
- JavaSDK: https://github.com/minio/minio-java
- PythonSDK:https://github.com/minio/minio-py
-
Graphical interface : There is an operation page
-
Supports erasure coding : MinIO uses erasure coding and checksum to prevent hardware errors and silent data pollution. Under the highest redundancy configuration, data can be recovered even if 1/2 of the disk is lost.
“
The function is very powerful. This article is just a starting point. Friends who are interested can explore it by themselves~
”
Install MinIO
The installation is very simple. I use docker to install it. The steps are as follows:
Get image
The execution command is as follows:
docker pull minio/minio
boot image
The execution command is as follows:
docker run -p 9000:9000 -p 9001:9001 --name minio -d --restart=always -e "MINIO_ACCESS_KEY=admin" -e "MINIO_SECRET_KEY=admin" -v /home/data:/data -v /home/config:/root/.minio minio/minio server --console-address ":9000" --address ":9001" /data
The command is explained below:
- -p : 9000 is the port of the graphical interface, 9001 is the port of the API, which is needed when using the SDK connection.
- MINIO_ACCESS_KEY : Specifies the user name of the graphical interface
- MINIO_SECRET_KEY : Specify the password for the graphical interface
Follow the above two steps to start successfully.
Graphical interface operation
After successful installation, directly access the address: http:/ip:9000/login , as follows:
After successfully logging in by entering your username and password, as follows:
There are many menus, so I won’t introduce them in detail here. The author directly creates a bucket as test in the Buckets menu , as shown below:
And set the privacy rule of this bucket to public , as follows:
“
MinIO has been successfully installed and set up now.
”
Spring Boot integrates MinIO to upload files
Although MinIO provides manual upload operations on the graphical interface, it can also be uploaded through the SDK. Here is an introduction to how Spring Boot integrates MinIO to upload files.
Get accessKey and secretKey
The accessKey and secretKey here are not the login name and password of the graphical interface. They are very simple to obtain and can be operated directly in the graphical interface, as shown below:
Add dependencies
Add MinIO dependencies as follows:
<dependency>
<groupId>io.minio</groupId>
<artifactId>minio</artifactId>
<version>8.2.1</version>
</dependency>
Add configuration
Add MInIO related configuration, as follows:
@Configuration
public class MinioConfig {
@Bean
public MinioClient minioClient() {
return MinioClient.builder()
.endpoint("http://xxxxxxxxxx:9090")
.credentials("你自己的key", "你自己的key")//accessKey和secretKey
.build();
}
}
Create a new upload file interface
The author defines an upload file interface here, as follows:
@WebLog
@PostMapping("/upload")
@SneakyThrows
public ApiResponse<Object> upload(MultipartFile file, String bucketName){
System.out.println(file.toString());
String fileType = FileTypeUtils.getFileType(file);
String originalFilename = file.getOriginalFilename();
MinIOUtil.putObject(bucketName,file,"test/"+originalFilename,fileType);
return new ApiResponse<>();
}
test
The above four steps have been integrated. Let’s directly call the interface to upload a picture and try it, as follows:
![[The external link image transfer failed. The source site may have an anti-leeching mechanism. It is recommended to save the image and upload it directly (img-EnZpvzur-1664501166507) (MinIO%E4%BB%8B%E7%BB%8D%E4%BB%A5 %E5%8F%8Aspringboot%E6%95%B4%E5%90%88.assets/image-20220930092521093.png)]](https://img-blog.csdnimg.cn/1140f2670df549a7814779d8870aaeef.png)
The URL returned by the interface is the access address of the file, which can be directly entered into the browser to access.
You can also see the stored files in MInIO, as shown below:
If you need to share it with others, you can also share it manually. The validity period is 7 days. Once this validity period expires, it will become invalid, as follows:
Summarize
Although MInIO is an open source project, it is very powerful. You can use it to implement object storage in small projects, and you can also use MinIO to build a free image bed.
appendix
MinIO tools
package com.example.mallgoods.util;
import com.alibaba.fastjson.JSONObject;
import io.minio.*;
import io.minio.http.Method;
import io.minio.messages.Bucket;
import io.minio.messages.DeleteObject;
import io.minio.messages.Item;
import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Component;
import org.springframework.web.multipart.MultipartFile;
import java.io.ByteArrayInputStream;
import java.io.File;
import java.io.InputStream;
import java.net.URLDecoder;
import java.util.ArrayList;
import java.util.LinkedList;
import java.util.List;
import java.util.Optional;
/**
* 功能描述 文件服务器工具类
*
* @author:
* @date:
*/
@Component
@Slf4j
public class MinIOUtil {
private static MinioClient minioClient;
public MinIOUtil(MinioClient minioClient) {
MinIOUtil.minioClient=minioClient;
}
/**
* 创建bucket
*
* @param bucketName bucket名称
*/
@SneakyThrows
public static void createBucket(String bucketName) {
if (!minioClient.bucketExists(BucketExistsArgs.builder().bucket(bucketName).build())) {
minioClient.makeBucket(MakeBucketArgs.builder().bucket(bucketName).build());
}
}
/**
* 获取存储桶策略
*
* @param bucketName 存储桶名称
* @return json
*/
@SneakyThrows
private JSONObject getBucketPolicy(String bucketName) {
String bucketPolicy = minioClient
.getBucketPolicy(GetBucketPolicyArgs.builder().bucket(bucketName).build());
return JSONObject.parseObject(bucketPolicy);
}
/**
* 获取全部bucket
* <p>
* https://docs.minio.io/cn/java-client-api-reference.html#listBuckets
*/
@SneakyThrows
public static List<Bucket> getAllBuckets() {
return minioClient.listBuckets();
}
/**
* 根据bucketName获取信息
*
* @param bucketName bucket名称
*/
@SneakyThrows
public static Optional<Bucket> getBucket(String bucketName) {
return minioClient.listBuckets().stream().filter(b -> b.name().equals(bucketName)).findFirst();
}
/**
* 根据bucketName删除信息
*
* @param bucketName bucket名称
*/
@SneakyThrows
public static void removeBucket(String bucketName) {
minioClient.removeBucket(RemoveBucketArgs.builder().bucket(bucketName).build());
}
/**
* 预览文件
* @param fileName
* @return
*/
@SneakyThrows
public static String preview(String fileName,String bucketName){
// 查看文件地址
new GetPresignedObjectUrlArgs();
GetPresignedObjectUrlArgs build = GetPresignedObjectUrlArgs
.builder()
.bucket(bucketName)
.object(fileName)
.method(Method.GET)
.build();
return minioClient.getPresignedObjectUrl(build);
}
/**
* 判断文件是否存在
*
* @param bucketName 存储桶
* @param objectName 对象
* @return true：存在
*/
@SneakyThrows
public static boolean doesObjectExist(String bucketName, String objectName) {
boolean exist = true;
try {
minioClient
.statObject(StatObjectArgs.builder().bucket(bucketName).object(objectName).build());
} catch (Exception e) {
exist = false;
}
return exist;
}
/**
* 判断文件夹是否存在
*
* @param bucketName 存储桶
* @param objectName 文件夹名称（去掉/）
* @return true：存在
*/
@SneakyThrows
public static boolean doesFolderExist(String bucketName, String objectName) {
boolean exist = false;
try {
Iterable<Result<Item>> results = minioClient.listObjects(
ListObjectsArgs.builder().bucket(bucketName).prefix(objectName).recursive(false).build());
for (Result<Item> result : results) {
Item item = result.get();
if (item.isDir() && objectName.equals(item.objectName())) {
exist = true;
}
}
} catch (Exception e) {
exist = false;
}
return exist;
}
/**
* 根据文件前置查询文件
*
* @param bucketName bucket名称
* @param prefix 前缀
* @param recursive 是否递归查询
* @return MinioItem 列表
*/
@SneakyThrows
public static List<Item> getAllObjectsByPrefix(String bucketName, String prefix,
boolean recursive) {
List<Item> list = new ArrayList<>();
Iterable<Result<Item>> objectsIterator = minioClient.listObjects(
ListObjectsArgs.builder().bucket(bucketName).prefix(prefix).recursive(recursive).build());
if (objectsIterator != null) {
for (Result<Item> o : objectsIterator) {
Item item = o.get();
list.add(item);
}
}
return list;
}
/**
* 获取文件流（下载文件）
*
* @param bucketName bucket名称
* @param objectName 文件名称
* @return 二进制流
*/
@SneakyThrows
public static InputStream getObject(String bucketName, String objectName) {
return minioClient
.getObject(GetObjectArgs.builder().bucket(bucketName).object(objectName).build());
}
/**
* 断点下载
*
* @param bucketName bucket名称
* @param objectName 文件名称
* @param offset 起始字节的位置
* @param length 要读取的长度
* @return 流
*/
@SneakyThrows
public InputStream getObject(String bucketName, String objectName, long offset, long length) {
return minioClient.getObject(
GetObjectArgs
.builder()
.bucket(bucketName)
.object(objectName)
.offset(offset)
.length(length)
.build());
}
/**
* 获取路径下文件列表
*
* @param bucketName bucket名称
* @param prefix 文件名称
* @param recursive 是否递归查找，如果是false,就模拟文件夹结构查找
* @return 二进制流
*/
@SneakyThrows
public static Iterable<Result<Item>> listObjects(String bucketName, String prefix,
boolean recursive) {
return minioClient.listObjects(
ListObjectsArgs.builder().bucket(bucketName).prefix(prefix).recursive(recursive).build());
}
/**
* 通过MultipartFile，上传文件
*
* @param bucketName 存储桶
* @param file 文件
* @param objectName 对象名
*/
@SneakyThrows
public static ObjectWriteResponse putObject(String bucketName, MultipartFile file,
String objectName, String contentType) {
InputStream inputStream = file.getInputStream();
return minioClient.putObject(
PutObjectArgs.builder().bucket(bucketName).object(objectName).contentType(contentType)
.stream(
inputStream, inputStream.available(), -1)
.build());
}
/**
* 上传本地文件
*
* @param bucketName 存储桶
* @param objectName 对象名称
* @param fileName 本地文件路径
*/
@SneakyThrows
public static ObjectWriteResponse putObject(String bucketName, String objectName,
String fileName) {
return minioClient.uploadObject(UploadObjectArgs
.builder()
.bucket(bucketName)
.object(objectName)
.filename(fileName)
.build());
}
/**
* 通过流上传文件
*
* @param bucketName 存储桶
* @param objectName 文件对象
* @param inputStream 文件流
*/
@SneakyThrows
public static ObjectWriteResponse putObject(String bucketName, String objectName,
InputStream inputStream) {
return minioClient.putObject(
PutObjectArgs
.builder()
.bucket(bucketName)
.object(objectName).stream(inputStream, inputStream.available(), -1)
.build());
}
/**
* 创建文件夹或目录
*
* @param bucketName 存储桶
* @param objectName 目录路径
*/
@SneakyThrows
public static ObjectWriteResponse putDirObject(String bucketName, String objectName) {
return minioClient.putObject(
PutObjectArgs
.builder()
.bucket(bucketName)
.object(objectName)
.stream(new ByteArrayInputStream(new byte[]{
}), 0, -1)
.build());
}
/**
* 获取文件信息, 如果抛出异常则说明文件不存在
* @param bucketName bucket名称
* @param objectName 文件名称
* @return
*/
@SneakyThrows
public static StatObjectResponse statObject(String bucketName, String objectName) {
return minioClient.statObject(StatObjectArgs
.builder()
.bucket(bucketName)
.object(objectName)
.build());
}
/**
* 拷贝文件
*
* @param bucketName bucket名称
* @param objectName 文件名称
* @param srcBucketName 目标bucket名称
* @param srcObjectName 目标文件名称
*/
@SneakyThrows
public static ObjectWriteResponse copyObject(String bucketName, String objectName,
String srcBucketName, String srcObjectName) {
return minioClient.copyObject(
CopyObjectArgs.builder()
.source(CopySource.builder().bucket(bucketName).object(objectName).build())
.bucket(srcBucketName)
.object(srcObjectName)
.build());
}
/**
* 删除文件
*
* @param bucketName bucket名称
* @param objectName 文件名称
*/
@SneakyThrows
public static void removeObject(String bucketName, String objectName) {
minioClient.removeObject(RemoveObjectArgs
.builder()
.bucket(bucketName)
.object(objectName)
.build());
}
/**
* 批量删除文件
*
* @param bucketName bucket
* @param keys 需要删除的文件列表
* @return
*/
@SneakyThrows
public static void removeObjects(String bucketName, List<String> keys) {
List<DeleteObject> objects = new LinkedList<>();
keys.forEach(s -> {
objects.add(new DeleteObject(s));
try {
removeObject(bucketName, s);
} catch (Exception e) {
log.error("批量删除失败！");
}
});
}
/**
* 获取文件外链
* @param bucketName 存储桶
* @param objectName 文件名
* @param expires 过期时间 <=7 秒 （外链有效时间（单位：秒））
* @return url
*
*/
@SneakyThrows
public static String getPresignedObjectUrl(String bucketName, String objectName, Integer expires){
GetPresignedObjectUrlArgs args = GetPresignedObjectUrlArgs
.builder()
.expiry(expires)
.bucket(bucketName)
.object(objectName)
.build();
return minioClient.getPresignedObjectUrl(args);
}
/**
* 获得文件外链
* @param bucketName
* @param objectName
* @return url
*
*/
@SneakyThrows
public static String getPresignedObjectUrl(String bucketName, String objectName){
GetPresignedObjectUrlArgs args = GetPresignedObjectUrlArgs.builder()
.bucket(bucketName)
.object(objectName)
.method(Method.GET).build();
return minioClient.getPresignedObjectUrl(args);
}
/**
* 将URLDecoder编码转成UTF8
*
* @param str
*/
@SneakyThrows
public static String getUtf8ByURLDecoder(String str){
String url = str.replaceAll("%(?![0-9a-fA-F]{2})", "%25");
return URLDecoder.decode(url, "UTF-8");
}
/**
* 将MultipartFile转换为File
* @param multiFile
* @return
*/
@SneakyThrows
public static File multipartFileToFile(MultipartFile multiFile) {
// 获取文件名
String fileName = multiFile.getOriginalFilename();
// 获取文件后缀
assert fileName != null;
String prefix = fileName.substring(fileName.lastIndexOf("."));
// 若须要防止生成的临时文件重复,能够在文件名后添加随机码
File file = File.createTempFile(fileName, prefix);
multiFile.transferTo(file);
return file;
}
}