The OSS service is now a basic service. Many cloud service vendors provide such a service, and the price is not expensive. Songge's own www.javaboy.org uses a similar service.
However, for small and medium-sized companies, in addition to purchasing OSS services, they can also build their own professional file servers. If they build their own specialized file servers, FastDFS used to be a more professional approach, for which Songge had previously recorded videos for distribution. On station B, interested friends can view it by themselves. However, FastDFS is more troublesome to build, and it is very error-prone, so there are more or less thresholds for your friends.
In some project videos recorded by Song Ge in previous articles, if file uploading is involved, they are basically saved locally in the project. This method is more convenient, but the security is not high.
So, today I will introduce you a better gadget, MinIO, and see what surprises this tool brings us.
1. Introduction to MinIO
MinIO is an object storage service based on the Apache License v2.0 open source protocol. It is compatible with the Amazon S3 cloud storage service interface and is very suitable for storing large-capacity unstructured data, such as pictures, videos, log files, backup data and container// A virtual machine image, etc., and an object file can be of any size, ranging from a few KB to a maximum of 5T.
MinIO is a very lightweight service that can be easily integrated with other applications, such as NodeJS, Redis or MySQL.
In short, you can use MinIO to build an object storage service, and MinIO's Java client is compatible with Amazon's S3 cloud storage service client interface. In other words, if you store data on MinIO, you will store data on S3. data.
Features of MinIO:
- Compatible with Amazon S3: MinIO server can be accessed using MinIO SDK, MinIO Client, AWS SDK and AWS CLI.
- Strong data protection: MinIO uses Minio Erasure Code to prevent hardware failure.
- Highly available: MinIO servers can tolerate up to
（N/2)-1node failures in a distributed setup. - Lambda computing is supported.
- Encrypted and tamper-resistant: MinIO provides confidentiality, integrity, and authenticity guarantees for encrypted data with minimal performance overhead. Server-side and client-side encryption is supported using AES-256-GCM, ChaCha20-Poly1305 and AES-CBC.
- Connectable back-end storage: In addition to MinIO's own file system, it also supports DAS, JBODs, NAS, Google Cloud Storage and Azure Blob storage.
2. MinIO installation
No nonsense, let's put on an experience now.
In order to save trouble, let's install it directly with docker. If you are not familiar with docker, please reply to docker in the background of the official account to get Songge's docker tutorial.
We execute the following command to install MinIO:
docker run -p 9000:9000 -p 9001:9001 -d minio/minio server /data --console-address ":9000" --address ":9001"
复制代码Two ports are configured in this startup command: console-address is the web page port for background management; address is the API communication port. Taking the above startup script as an example, after the project is successfully started, the access port on the web page is 9000. If we upload files through Java code, the communication port is 9001.
After the project is started successfully, http://127.0.0.1:9000/loginyou can :
The default login username and password are both minioadmin.
After the login is successful, we first create a bucket, and the files we upload in the future will all be in the bucket, as follows:
After the creation is successful, we also need to set the read permission of the bucket to ensure that the file can be read after the file is uploaded successfully in the future. Click the Settings button in the upper left corner to set, as follows:
After the setting is complete, we can upload resources to this bucket, as shown below:
After the upload is complete, you can see the file you just uploaded:
After the upload is successful, click the file, and then click the Share button on the right, and the access link of the file will pop up. Since we have set the file to be readable, you can ignore the validity period of the link here, and you can access it directly through the front part of the path. The pictures I just uploaded are as follows:
现在文件就可上传可访问了。是不是比 FastDFS 容易多了！
不过前面这种安装方式其实有点小问题，因为我们没有为 docker 容器设置数据卷，所以如果你把 docker 容器不小心删除了，那么数据也就没了！
所以我们要设置数据卷。
修正后的 docker 脚本如下：
docker run -p 9000:9000 -p 9001:9001 -d --name minio -v /Users/sang/minio/data:/data -v /Users/sang/minio/config:/root/.minio -e "MINIO_ROOT_USER=javaboy" -e "MINIO_ROOT_PASSWORD=123@45678" minio/minio server /data --console-address ":9000" --address ":9001"
复制代码主要是加了数据卷映射功能，将 MinIO 的数据和配置文件映射到宿主机上，这样将来即使容器删除了，数据也都还在。
注意上面也自定义了登录用户名和密码。
按照上面的命令，重新创建容器之后，我们也创建一个桶并上传文件，上传成功之后，我们就可以在本地对应的文件夹看到我们上传的文件，如下：
3. 整合 Spring Boot
接下来我们再来看看在 Spring Boot 中如何玩 MinIO。
首先我们创建一个 Spring Boot 项目，引入 Web 依赖，如下：
项目创建成功之后，我们再来手动添加一下 MinIO 的依赖，如下：
<dependency>
<groupId>io.minio</groupId>
<artifactId>minio</artifactId>
<version>8.2.1</version>
</dependency>
复制代码这里我尝试用了最新的版本，但是似乎有一些 BUG，我也没有深究，就换了 8.2.1 这个版本，这个版本是 OK 的。
接下来我们来配置一下 application.yaml，配置一下文件上传所需要的基本信息：
minio:
endpoint: http://localhost:9001
accessKey: javaboy
secretKey: 123@45678
nginxHost: http://local.javaboy.org:9001
复制代码这里四个属性：
- endpoint：这是 MinIO 的 API 通信地址。
- accessKey 和 secretKey 是通信的用户名和密码，这跟网页上登录时候的用户名密码一致。
- nginxHost：这个配置用来生成上传文件的访问路径。对于这个路径，有的小伙伴可能会有疑问，nginxHost 不就是 endpoint 吗？为什么还要单独配置？因为对于文件服务器而言，我们上传文件是通过 MinIO，但是访问的时候不一定通过 MinIO，我们可能会自己搭建一个 Nginx 服务器，通过 Nginx 服务器来访问上传后的资源，大家知道 Nginx 非常擅长于做这个事情，效率非常高。所以这里的 nginxHost 其实是指 Nginx 的访问路径。
接下来我们提供一个 MinioProperties 来接收这里的四个属性，如下：
@ConfigurationProperties(prefix = "minio")
public class MinioProperties {
/**
* 连接地址
*/
private String endpoint;
/**
* 用户名
*/
private String accessKey;
/**
* 密码
*/
private String secretKey;
/**
* 域名
*/
private String nginxHost;
public String getEndpoint() {
return endpoint;
}
public void setEndpoint(String endpoint) {
this.endpoint = endpoint;
}
public String getAccessKey() {
return accessKey;
}
public void setAccessKey(String accessKey) {
this.accessKey = accessKey;
}
public String getSecretKey() {
return secretKey;
}
public void setSecretKey(String secretKey) {
this.secretKey = secretKey;
}
public String getNginxHost() {
return nginxHost;
}
public void setNginxHost(String nginxHost) {
this.nginxHost = nginxHost;
}
}
复制代码将 application.yaml 中相关的配置注入到这个配置类中来。
接下来我们需要提供一个 MinIOClient，通过这个客户端工具可以操作 MinIO，如下：
@Configuration
@EnableConfigurationProperties(MinioProperties.class)
public class MinioConfig {
@Autowired
private MinioProperties minioProperties;
/**
* 获取MinioClient
*/
@Bean
public MinioClient minioClient() {
return MinioClient.builder()
.endpoint(minioProperties.getEndpoint())
.credentials(minioProperties.getAccessKey(), minioProperties.getSecretKey())
.build();
}
}
复制代码这个也没啥好说的，传入通信地址以及用户名密码，就可以构建出一个 MinioClient 出来。
当文件上传成功之后，我们可以通过 MinIO 去访问，也可以通过 Nginx 访问，所以接下来我们就需要提供一个类，来封装这两个地址：
public class UploadResponse {
private String minIoUrl;
private String nginxUrl;
public UploadResponse() {
}
public UploadResponse(String minIoUrl, String nginxUrl) {
this.minIoUrl = minIoUrl;
this.nginxUrl = nginxUrl;
}
public String getMinIoUrl() {
return minIoUrl;
}
public void setMinIoUrl(String minIoUrl) {
this.minIoUrl = minIoUrl;
}
public String getNginxUrl() {
return nginxUrl;
}
public void setNginxUrl(String nginxUrl) {
this.nginxUrl = nginxUrl;
}
}
复制代码再来提供一个 MinIO 文件上传工具类：
@Component
public class MinioUtil {
@Autowired
private MinioProperties minioProperties;
@Autowired
private MinioClient client;
/**
* 创建bucket
*/
public void createBucket(String bucketName) throws Exception {
if (!client.bucketExists(BucketExistsArgs.builder().bucket(bucketName).build())) {
client.makeBucket(MakeBucketArgs.builder().bucket(bucketName).build());
}
}
/**
* 上传文件
*/
public UploadResponse uploadFile(MultipartFile file, String bucketName) throws Exception {
//判断文件是否为空
if (null == file || 0 == file.getSize()) {
return null;
}
//判断存储桶是否存在 不存在则创建
createBucket(bucketName);
//文件名
String originalFilename = file.getOriginalFilename();
//新的文件名 = 存储桶文件名_时间戳.后缀名
assert originalFilename != null;
SimpleDateFormat format = new SimpleDateFormat("yyyy-MM-dd");
String fileName = bucketName + "_" +
System.currentTimeMillis() + "_" + format.format(new Date()) + "_" + new Random().nextInt(1000) +
originalFilename.substring(originalFilename.lastIndexOf("."));
//开始上传
client.putObject(
PutObjectArgs.builder().bucket(bucketName).object(fileName).stream(
file.getInputStream(), file.getSize(), -1)
.contentType(file.getContentType())
.build());
String url = minioProperties.getEndpoint() + "/" + bucketName + "/" + fileName;
String urlHost = minioProperties.getNginxHost() + "/" + bucketName + "/" + fileName;
return new UploadResponse(url, urlHost);
}
/**
* 获取全部bucket
*
* @return
*/
public List<Bucket> getAllBuckets() throws Exception {
return client.listBuckets();
}
/**
* 根据bucketName获取信息
*
* @param bucketName bucket名称
*/
public Optional<Bucket> getBucket(String bucketName) throws IOException, InvalidKeyException, NoSuchAlgorithmException, InsufficientDataException, InvalidResponseException, InternalException, ErrorResponseException, ServerException, XmlParserException, ServerException {
return client.listBuckets().stream().filter(b -> b.name().equals(bucketName)).findFirst();
}
/**
* 根据bucketName删除信息
*
* @param bucketName bucket名称
*/
public void removeBucket(String bucketName) throws Exception {
client.removeBucket(RemoveBucketArgs.builder().bucket(bucketName).build());
}
/**
* 获取⽂件外链
*
* @param bucketName bucket名称
* @param objectName ⽂件名称
* @param expires 过期时间 <=7
* @return url
*/
public String getObjectURL(String bucketName, String objectName, Integer expires) throws Exception {
return client.getPresignedObjectUrl(GetPresignedObjectUrlArgs.builder().bucket(bucketName).object(objectName).expiry(expires).build());
}
/**
* 获取⽂件
*
* @param bucketName bucket名称
* @param objectName ⽂件名称
* @return ⼆进制流
*/
public InputStream getObject(String bucketName, String objectName) throws Exception {
return client.getObject(GetObjectArgs.builder().bucket(bucketName).object(objectName).build());
}
/**
* 上传⽂件
*
* @param bucketName bucket名称
* @param objectName ⽂件名称
* @param stream ⽂件流
* @throws Exception https://docs.minio.io/cn/java-client-api-reference.html#putObject
*/
public void putObject(String bucketName, String objectName, InputStream stream) throws
Exception {
client.putObject(PutObjectArgs.builder().bucket(bucketName).object(objectName).stream(stream, stream.available(), -1).contentType(objectName.substring(objectName.lastIndexOf("."))).build());
}
/**
* 上传⽂件
*
* @param bucketName bucket名称
* @param objectName ⽂件名称
* @param stream ⽂件流
* @param size ⼤⼩
* @param contextType 类型
* @throws Exception https://docs.minio.io/cn/java-client-api-reference.html#putObject
*/
public void putObject(String bucketName, String objectName, InputStream stream, long
size, String contextType) throws Exception {
client.putObject(PutObjectArgs.builder().bucket(bucketName).object(objectName).stream(stream, size, -1).contentType(contextType).build());
}
/**
* 获取⽂件信息
*
* @param bucketName bucket名称
* @param objectName ⽂件名称
* @throws Exception https://docs.minio.io/cn/java-client-api-reference.html#statObject
*/
public StatObjectResponse getObjectInfo(String bucketName, String objectName) throws Exception {
return client.statObject(StatObjectArgs.builder().bucket(bucketName).object(objectName).build());
}
/**
* 删除⽂件
*
* @param bucketName bucket名称
* @param objectName ⽂件名称
* @throws Exception https://docs.minio.io/cn/java-client-apireference.html#removeObject
*/
public void removeObject(String bucketName, String objectName) throws Exception {
client.removeObject(RemoveObjectArgs.builder().bucket(bucketName).object(objectName).build());
}
}
复制代码都是一些常规的 API 调用，我就不逐行解释了，接下来我们来一个文件上传接口：
@RestController
public class FileUploadController {
@Autowired
MinioUtil minioUtil;
@PostMapping("/upload")
public String fileUpload(MultipartFile file) throws Exception {
UploadResponse bucket01 = minioUtil.uploadFile(file, "bucket01");
System.out.println("bucket01.getMinIoUrl() = " + bucket01.getMinIoUrl());
System.out.println("bucket01.getNginxUrl() = " + bucket01.getNginxUrl());
return bucket01.getMinIoUrl();
}
}
复制代码好啦，大功告成。
接下来启动 Spring Boot 项目，然后调用这个接口上传文件，上传成功后，控制台会打印如下信息：
这就表示文件上传成功了。
4. 配置 nginx
前面提到了 MinIO 可以结合 Nginx 来使用，那我们这里就来配一配 Nginx 看看。
为了省事，Nginx 我也选择安装到 docker 容器中，但是前面安装 MinIO 时，我们已经做了数据卷映射，即上传到 MinIO 的文件实际上是保存在宿主机的，所以现在也得给 Nginx 配置数据卷，将来让 Nginx 也去 /Users/sang/minio/data 路径下查找文件。
Nginx 安装指令如下：
docker run --name nginx01 -p 8888:80 -v /Users/sang/minio/data:/usr/share/nginx/html:ro -d nginx
复制代码这里两个关键点：
- 设置 Nginx 端口为 8888。
- 将 MinIO 映射到宿主机的数据卷，再次挂载到 Nginx 上去。
大家知道，默认情况下，当我们访问 Nginx 的时候，Nginx 给我们展示出来的数据其实就是 /usr/share/nginx/html 目录下的，现在该目录其实就相当于我宿主机的 /Users/sang/minio/data 目录，所以我现在都不用修改 Nginx 的配置了，装好之后直接使用 Nginx 即可。
好啦，接下来我们修改一下 application.yaml，如下：
minio:
endpoint: http://localhost:9001
accessKey: javaboy
secretKey: 123@45678
nginxHost: http://local.javaboy.org:8888
复制代码改完之后，再次上传文件，此时打印出来的文件访问路径如下：
Now we can also access the file just uploaded through this Nginx path.
5. Summary
Well, today I will share with my friends the usage of MinIO, and build a simple file server with Nginx. Interested friends can try it.
The public account Jiangnan Yidianyu replied to minio_demo in the background , and obtained the source code download link of this article.