Foreword:
minio is a very lightweight object storage service, which can be counted in the field of cloud native.
The service is written in go language. Therefore, the main file is just one file. Its download and deployment are very simple. Generally, it can be built in two or three steps, but there are some details that need to be used in deployment. Time to pay attention.
This article will explain an available minio storage service deployment in as much detail as possible, and discuss how to implement this technology.
one,
Introduction to minio
MinIO is also a member of CNCF now, and it is one of the current solutions together with ceph in the cloud-native storage part. Therefore, one of the landing directions of this service is to apply it in cloud computing and cloud-native service orchestration management system kubernetes. Secondly, This service can be used as a simple ftp-like server (usually it is more convenient to send a file to the server)
However, due to the characteristics of the storage service (closely connected with the operating system of the host machine, generally an underlying service), the vulnerabilities exposed by this service are usually relatively high-risk. Therefore, we need to pay special attention to version issues when using services such as minio , it is recommended to use a higher version (the higher version has fewer vulnerabilities), if it is an experiment or test, then the version does not matter. (Vulnerability-related issues will not be long-winded here, there are a lot of Baidu)
In fact, the introduction on the official website is more detailed. The official website address: MinIO | High Performance, Kubernetes Native Object Storage
Download address: MinIO China Mirror Station
two,
How to deploy minio
1, rpm deployment
There is nothing to say about this, it is simple and convenient, but it cannot be customized. It is available on the official download website above.
Download address: https://dl.minio.org.cn/server/minio/release/linux-amd64/minio-20230413030807.0.0.x86_64.rpm
Install the rpm package and query the location of the self-starting script and the location of the execution program and join the startup
[root@EULEER ~]# rpm -ivh minio-20230413030807.0.0.x86_64.rpm
Preparing... ################################# [100%]
Updating / installing...
1:minio-0:20230413030807.0.0-1 ################################# [100%]
[root@EULEER ~]# rpm -ql minio-20230413030807.0.0-1.x86_64
/etc/systemd/system/minio.service
/usr/local/bin/minio
[root@EULEER ~]# systemctl enable minio
Created symlink from /etc/systemd/system/multi-user.target.wants/minio.service to /etc/systemd/system/minio.service.
Write the configuration file according to the self-starting script. The content of the self-starting script is as follows:
##It should be noted that User and Group have been changed to minio by me
[root@EULEER ~]# cat /etc/systemd/system/minio.service
[Unit]
Description=MinIO
Documentation=https://docs.min.io
Wants=network-online.target
After=network-online.target
AssertFileIsExecutable=/usr/local/bin/minio
[Service]
WorkingDirectory=/usr/local
User=minio
Group=minio
ProtectProc=invisible
EnvironmentFile=-/etc/default/minio
ExecStartPre=/bin/bash -c "if [ -z \"${MINIO_VOLUMES}\" ]; then echo \"Variable MINIO_VOLUMES not set in /etc/default/minio\"; exit 1; fi"
ExecStart=/usr/local/bin/minio server $MINIO_OPTS $MINIO_VOLUMES
# Let systemd restart this service always
Restart=always
# Specifies the maximum file descriptor number that can be opened by this process
LimitNOFILE=1048576
# Specifies the maximum number of threads this process can create
TasksMax=infinity
# Disable timeout logic and wait until process is stopped
TimeoutStopSec=infinity
SendSIGKILL=no
[Install]
WantedBy=multi-user.target
# Built for ${project.name}-${project.version} (${project.name})
OK, according to the above script content, add a common user minio (this user has no login authority, which can improve the security of the system), and create a configuration file for the minio service: /etc/default/minio
useradd -M -s /bin/nologinThe configuration file defines that minio uses the /data1 directory, the service open port is 39111, the login user on the web side is minio, and the password is omitted here.
#### Note: The data1 directory is not random. The official suggestion is to use a separate directory on the hard disk. In actual production, the data disk must be used. (How to mount the data disk is not nonsense here.) . If the data disk is not used in the cluster, minio will report an error that the data directory cannot be on the same disk as the root root file system, and a separate disk must be used, otherwise the startup will fail . See the picture below (two hard disks, one disk is mounted in the data1 directory):
[root@EULEER ~]# cat /etc/default/minio
MINIO_VOLUMES="/data1"
MINIO_OPTS="--address :39111"
MINIO_ACCESS_KEY=minio
MINIO_SECRET_KEY=密码
Create the above-mentioned data1 directory and authorize it to the minio user. It should be noted that the /data1 directory must be an empty directory:
mkdir /data1
chown -Rf minio. /data1After the above work is completed, you can start the minio service and open the browser to log in to the minio client. The login address is the server address +39111:
Service start and service status:
systemctl start minio
[root@EULEER ~]# systemctl status minio
● minio.service - MinIO
Loaded: loaded (/etc/systemd/system/minio.service; enabled; vendor preset: disabled)
Active: active (running) since Thu 2023-05-18 05:56:36 CST; 9min ago
Docs: https://docs.min.io
Process: 5063 ExecStartPre=/bin/bash -c if [ -z "${MINIO_VOLUMES}" ]; then echo "Variable MINIO_VOLUMES not set in /etc/default/minio"; exit 1; fi (code=exited, status=0/SUCCESS)
Main PID: 5064 (minio)
CGroup: /system.slice/system-hostos.slice/minio.service
└─5064 /usr/local/bin/minio server --address :39111 /data1
May 18 05:56:37 EULEER minio[5064]: Copyright: 2015-2023 MinIO, Inc.
May 18 05:56:37 EULEER minio[5064]: License: GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>
May 18 05:56:37 EULEER minio[5064]: Version: RELEASE.2023-04-13T03-08-07Z (go1.20.3 linux/amd64)
May 18 05:56:37 EULEER minio[5064]: Status: 1 Online, 0 Offline.
May 18 05:56:37 EULEER minio[5064]: API: http://192.168.76.11:39111 http://127.0.0.1:39111
May 18 05:56:37 EULEER minio[5064]: Console: http://192.168.76.11:42317 http://127.0.0.1:42317
May 18 05:56:37 EULEER minio[5064]: Documentation: https://min.io/docs/minio/linux/index.html
May 18 05:56:37 EULEER minio[5064]: Warning: The standard parity is set to 0. This can lead to data loss.
May 18 05:56:38 EULEER minio[5064]: You are running an older version of MinIO released 3 weeks ago
May 18 05:56:38 EULEER minio[5064]: Update: Run `mc admin update`
State of the browser:
The label in the figure above is creating a bucket. The specific process of creating a bucket is as follows:
- bucket name: bucket name.
- versioning: Versioning allows multiple versions of the same object to be kept under the same key.
- object locking: Object locking prevents objects from being deleted. Need to support retention and legal holding. Can only be enabled when creating a bucket.
- quota: The quota is used to limit the amount of data in the bucket.
- Retention: A retention is a rule that prevents object deletion for a period of time. In order to set a bucket retention policy, versioning must be enabled.
Since the installation of minio is too easy, the difficulty is actually in the later use, such as security settings, permission settings, and audit settings.
The following takes the audit function as an example to explain how to configure the audit function of minio:
You can see that the audit is called a message here, but it doesn't matter, it doesn't matter what it is called. There are many audit storages supported by minio. I happen to have redis installed, so I will use redis.
The ports opened by redis are as follows:
[root@EULEER yum.repos.d]# netstat -antup |grep redis
tcp 0 0 192.168.76.11:15379 0.0.0.0:* LISTEN 5581/./redis-server
tcp 0 0 192.168.76.11:15379 192.168.76.1:51519 ESTABLISHED 5581/./redis-server
tcp 0 0 192.168.76.11:15379 192.168.76.1:51542 ESTABLISHED 5581/./redis-server
tcp 0 0 192.168.76.11:15379 192.168.76.1:51570 ESTABLISHED 5581/./redis-server
tcp 0 0 192.168.76.11:15379 192.168.76.1:51541 ESTABLISHED 5581/./redis-server
tcp 0 0 192.168.76.11:15379 192.168.76.1:51689 ESTABLISHED 5581/./redis-server
Back to minio, click the redis icon to enter the following interface:
After saving, a warning is displayed, which means that the minio service needs to be restarted to apply the configuration just modified. Click restart to restart the service:
After restarting, you can see that redis is displayed:
There is still one last shiver, and the bucket specifies the source of the news:
Check the addition, deletion and modification, choose redis first, and save it:
Feel free to upload and delete two files, looking at redis, probably as follows:
two,
OK, the above installation and deployment is only one drive data1 (the folder where minio will store files is called drive)
So, how to drive more?
In fact, it is very simple, other places remain unchanged, just add a set of folders to the configuration file (note that they are separated by spaces, not semicolons):
[root@EULEER data1]# cat /etc/default/minio
MINIO_VOLUMES="/data1/test1 /data1/test2 /data1/test3 /data1/test4"
MINIO_OPTS="--address :39111"
MINIO_ACCESS_KEY=minio
MINIO_SECRET_KEY=shiguang32
Then, we also need to create the above directories that actually exist and assign these directories to the user defined in the startup script, in this case minio:
mkdir /data1/test{1..4}
chown -Rf minio. /data1/Start the minio service again, and you can see the following:
[root@EULEER data1]# ls -al
total 4
drwxr-xr-x 7 minio minio 76 May 19 09:37 .
dr-xr-xr-x. 22 root root 4096 May 17 10:31 ..
drwxr-xr-x 7 minio minio 98 May 18 10:42 .minio.sys
drwx------ 4 minio minio 36 May 19 09:52 test1
drwx------ 4 minio minio 36 May 19 09:52 test2
drwx------ 4 minio minio 36 May 19 09:52 test3
drwx------ 4 minio minio 36 May 19 09:52 test4
[root@EULEER data1]# tree -a --dirsfirst \./
./
├── .minio.sys
│ ├── buckets
│ │ ├── .bloomcycle.bin
│ │ │ └── xl.meta
│ │ ├── test
│ │ │ ├── .metadata.bin
│ │ │ │ └── xl.meta
│ │ │ └── .usage-cache.bin
│ │ │ └── xl.meta
│ │ ├── .usage-cache.bin
│ │ │ └── xl.meta
│ │ └── .usage.json
│ │ └── xl.meta
│ ├── config
│ │ ├── config.json
│ │ │ └── xl.meta
│ │ ├── history
│ │ │ ├── 1ab6e422-56c1-4777-8869-21ccafdfe6c6.kv
│ │ │ │ └── xl.meta
│ │ │ └── 67f69168-adcc-45e5-a5c6-a49ad6c1afa8.kv
│ │ │ └── xl.meta
│ │ └── iam
│ │ ├── format.json
│ │ │ └── xl.meta
│ │ └── sts
│ │ └── RR5YLHZGDAC64AIGJ3U2
│ │ └── identity.json
│ │ └── xl.meta
│ ├── multipart
│ ├── pool.bin
│ │ └── xl.meta
│ ├── tmp
│ │ └── .trash
│ └── format.json
├── test1
│ ├── .minio.sys
│ │ ├── buckets
│ │ │ ├── .background-heal.json
│ │ │ │ └── xl.meta
│ │ │ ├── .bloomcycle.bin
│ │ │ │ └── xl.meta
│ │ │ ├── test
│ │ │ │ ├── .metadata.bin
│ │ │ │ │ └── xl.meta
│ │ │ │ └── .usage-cache.bin
│ │ │ │ └── xl.meta
│ │ │ ├── .usage-cache.bin
│ │ │ │ └── xl.meta
│ │ │ └── .usage.json
│ │ │ └── xl.meta
│ │ ├── config
│ │ │ ├── config.json
│ │ │ │ └── xl.meta
│ │ │ └── iam
│ │ │ ├── format.json
│ │ │ │ └── xl.meta
│ │ │ └── sts
│ │ │ └── 0ESAM6BK1PEF87AZ59E2
│ │ │ └── identity.json
│ │ │ └── xl.meta
│ │ ├── multipart
│ │ ├── pool.bin
│ │ │ └── xl.meta
│ │ ├── tmp
│ │ │ └── .trash
│ │ └── format.json
│ └── test
├── test2
│ ├── .minio.sys
│ │ ├── buckets
│ │ │ ├── .background-heal.json
│ │ │ │ └── xl.meta
│ │ │ ├── .bloomcycle.bin
│ │ │ │ └── xl.meta
│ │ │ ├── test
│ │ │ │ ├── .metadata.bin
│ │ │ │ │ └── xl.meta
│ │ │ │ └── .usage-cache.bin
│ │ │ │ └── xl.meta
│ │ │ ├── .usage-cache.bin
│ │ │ │ └── xl.meta
│ │ │ └── .usage.json
│ │ │ └── xl.meta
│ │ ├── config
│ │ │ ├── config.json
│ │ │ │ └── xl.meta
│ │ │ └── iam
│ │ │ ├── format.json
│ │ │ │ └── xl.meta
│ │ │ └── sts
│ │ │ └── 0ESAM6BK1PEF87AZ59E2
│ │ │ └── identity.json
│ │ │ └── xl.meta
│ │ ├── multipart
│ │ ├── pool.bin
│ │ │ └── xl.meta
│ │ ├── tmp
│ │ │ └── .trash
│ │ └── format.json
│ └── test
├── test3
│ ├── .minio.sys
│ │ ├── buckets
│ │ │ ├── .background-heal.json
│ │ │ │ └── xl.meta
│ │ │ ├── .bloomcycle.bin
│ │ │ │ └── xl.meta
│ │ │ ├── test
│ │ │ │ ├── .metadata.bin
│ │ │ │ │ └── xl.meta
│ │ │ │ └── .usage-cache.bin
│ │ │ │ └── xl.meta
│ │ │ ├── .usage-cache.bin
│ │ │ │ └── xl.meta
│ │ │ └── .usage.json
│ │ │ └── xl.meta
│ │ ├── config
│ │ │ ├── config.json
│ │ │ │ └── xl.meta
│ │ │ └── iam
│ │ │ ├── format.json
│ │ │ │ └── xl.meta
│ │ │ └── sts
│ │ │ └── 0ESAM6BK1PEF87AZ59E2
│ │ │ └── identity.json
│ │ │ └── xl.meta
│ │ ├── multipart
│ │ ├── pool.bin
│ │ │ └── xl.meta
│ │ ├── tmp
│ │ │ └── .trash
│ │ └── format.json
│ └── test
└── test4
├── .minio.sys
│ ├── buckets
│ │ ├── .background-heal.json
│ │ │ └── xl.meta
│ │ ├── .bloomcycle.bin
│ │ │ └── xl.meta
│ │ ├── test
│ │ │ ├── .metadata.bin
│ │ │ │ └── xl.meta
│ │ │ └── .usage-cache.bin
│ │ │ └── xl.meta
│ │ ├── .usage-cache.bin
│ │ │ └── xl.meta
│ │ └── .usage.json
│ │ └── xl.meta
│ ├── config
│ │ ├── config.json
│ │ │ └── xl.meta
│ │ └── iam
│ │ ├── format.json
│ │ │ └── xl.meta
│ │ └── sts
│ │ └── 0ESAM6BK1PEF87AZ59E2
│ │ └── identity.json
│ │ └── xl.meta
│ ├── multipart
│ ├── pool.bin
│ │ └── xl.meta
│ ├── tmp
│ │ └── .trash
│ └── format.json
└── test
110 directories, 56 files
OK, start the minio service, create a bucket at will, and upload a file at will, then check the folder again:
[root@EULEER data1]# tree -a --dirsfirst ./
./
├── .minio.sys
│ ├── buckets
│ │ ├── .bloomcycle.bin
│ │ │ └── xl.meta
│ │ ├── test
│ │ │ ├── .metadata.bin
│ │ │ │ └── xl.meta
│ │ │ └── .usage-cache.bin
│ │ │ └── xl.meta
│ │ ├── .usage-cache.bin
│ │ │ └── xl.meta
│ │ └── .usage.json
│ │ └── xl.meta
│ ├── config
│ │ ├── config.json
│ │ │ └── xl.meta
│ │ ├── history
│ │ │ ├── 1ab6e422-56c1-4777-8869-21ccafdfe6c6.kv
│ │ │ │ └── xl.meta
│ │ │ └── 67f69168-adcc-45e5-a5c6-a49ad6c1afa8.kv
│ │ │ └── xl.meta
│ │ └── iam
│ │ ├── format.json
│ │ │ └── xl.meta
│ │ └── sts
│ │ └── RR5YLHZGDAC64AIGJ3U2
│ │ └── identity.json
│ │ └── xl.meta
│ ├── multipart
│ ├── pool.bin
│ │ └── xl.meta
│ ├── tmp
│ │ └── .trash
│ └── format.json
├── test1
│ ├── .minio.sys
│ │ ├── buckets
│ │ │ ├── .background-heal.json
│ │ │ │ └── xl.meta
│ │ │ ├── .bloomcycle.bin
│ │ │ │ └── xl.meta
│ │ │ ├── test
│ │ │ │ ├── .metadata.bin
│ │ │ │ │ └── xl.meta
│ │ │ │ └── .usage-cache.bin
│ │ │ │ └── xl.meta
│ │ │ ├── .usage-cache.bin
│ │ │ │ └── xl.meta
│ │ │ └── .usage.json
│ │ │ └── xl.meta
│ │ ├── config
│ │ │ ├── config.json
│ │ │ │ └── xl.meta
│ │ │ └── iam
│ │ │ ├── format.json
│ │ │ │ └── xl.meta
│ │ │ └── sts
│ │ │ └── 0ESAM6BK1PEF87AZ59E2
│ │ │ └── identity.json
│ │ │ └── xl.meta
│ │ ├── multipart
│ │ ├── pool.bin
│ │ │ └── xl.meta
│ │ ├── tmp
│ │ │ └── .trash
│ │ └── format.json
│ └── test
│ └── \351\242\234\350\211\262.png
│ └── xl.meta
├── test2
│ ├── .minio.sys
│ │ ├── buckets
│ │ │ ├── .background-heal.json
│ │ │ │ └── xl.meta
│ │ │ ├── .bloomcycle.bin
│ │ │ │ └── xl.meta
│ │ │ ├── test
│ │ │ │ ├── .metadata.bin
│ │ │ │ │ └── xl.meta
│ │ │ │ └── .usage-cache.bin
│ │ │ │ └── xl.meta
│ │ │ ├── .usage-cache.bin
│ │ │ │ └── xl.meta
│ │ │ └── .usage.json
│ │ │ └── xl.meta
│ │ ├── config
│ │ │ ├── config.json
│ │ │ │ └── xl.meta
│ │ │ └── iam
│ │ │ ├── format.json
│ │ │ │ └── xl.meta
│ │ │ └── sts
│ │ │ └── 0ESAM6BK1PEF87AZ59E2
│ │ │ └── identity.json
│ │ │ └── xl.meta
│ │ ├── multipart
│ │ ├── pool.bin
│ │ │ └── xl.meta
│ │ ├── tmp
│ │ │ └── .trash
│ │ └── format.json
│ └── test
│ └── \351\242\234\350\211\262.png
│ └── xl.meta
├── test3
│ ├── .minio.sys
│ │ ├── buckets
│ │ │ ├── .background-heal.json
│ │ │ │ └── xl.meta
│ │ │ ├── .bloomcycle.bin
│ │ │ │ └── xl.meta
│ │ │ ├── test
│ │ │ │ ├── .metadata.bin
│ │ │ │ │ └── xl.meta
│ │ │ │ └── .usage-cache.bin
│ │ │ │ └── xl.meta
│ │ │ ├── .usage-cache.bin
│ │ │ │ └── xl.meta
│ │ │ └── .usage.json
│ │ │ └── xl.meta
│ │ ├── config
│ │ │ ├── config.json
│ │ │ │ └── xl.meta
│ │ │ └── iam
│ │ │ ├── format.json
│ │ │ │ └── xl.meta
│ │ │ └── sts
│ │ │ └── 0ESAM6BK1PEF87AZ59E2
│ │ │ └── identity.json
│ │ │ └── xl.meta
│ │ ├── multipart
│ │ ├── pool.bin
│ │ │ └── xl.meta
│ │ ├── tmp
│ │ │ └── .trash
│ │ └── format.json
│ └── test
│ └── \351\242\234\350\211\262.png
│ └── xl.meta
└── test4
├── .minio.sys
│ ├── buckets
│ │ ├── .background-heal.json
│ │ │ └── xl.meta
│ │ ├── .bloomcycle.bin
│ │ │ └── xl.meta
│ │ ├── test
│ │ │ ├── .metadata.bin
│ │ │ │ └── xl.meta
│ │ │ └── .usage-cache.bin
│ │ │ └── xl.meta
│ │ ├── .usage-cache.bin
│ │ │ └── xl.meta
│ │ └── .usage.json
│ │ └── xl.meta
│ ├── config
│ │ ├── config.json
│ │ │ └── xl.meta
│ │ └── iam
│ │ ├── format.json
│ │ │ └── xl.meta
│ │ └── sts
│ │ └── 0ESAM6BK1PEF87AZ59E2
│ │ └── identity.json
│ │ └── xl.meta
│ ├── multipart
│ ├── pool.bin
│ │ └── xl.meta
│ ├── tmp
│ │ └── .trash
│ └── format.json
└── test
└── \351\242\234\350\211\262.png
└── xl.meta
114 directories, 60 files
Delete the png file under the test1 folder, wait a few seconds, the file will be restored automatically:
[root@EULEER data1]# rm -rf test1/test/颜色.png/
[root@EULEER data1]# ls -al !$
ls -al test1/test/颜色.png/
ls: cannot access test1/test/颜色.png/: No such file or directory
#等待几秒后，发现文件回来了
[root@EULEER data1]# ls -al test1/test/颜色.png/
total 28
drwxr-xr-x 2 minio minio 21 May 19 10:33 .
drwxr-xr-x 3 minio minio 24 May 19 10:33 ..
-rw-r--r-- 1 minio minio 28337 May 19 10:33 xl.meta
OK, a highly available distributed minio cluster has been built. Of course, drives can be evenly distributed on several servers. The following section describes how to fully distribute the distributed minio cluster.