1 简介

MinIO 是一个基于Apache License v2.0开源协议的对象存储服务。它兼容亚马逊S3云存储服务接口，非常适合于存储大容量非结构化的数据，例如图片、视频、日志文件、备份数据和容器/虚拟机镜像等，而一个对象文件可以是任意大小，从几kb到最大5T不等。

MinIO是一个非常轻量的服务,可以很简单的和其他应用的结合，类似 NodeJS, Redis 或者 MySQL。
其提供了多种部署和使用方式，本文将在单机上部署一个minio集群，并对其进行基本使用和测试。

2 部署minio集群

前置条件：安装docker 和 docker-compose

下载配置,初始化目录

$ wget https://github.com/minio/minio/raw/master/docs/orchestration/docker-compose/docker-compose.yaml
或者
$ wget https://raw.githubusercontent.com/minio/minio/master/docs/orchestration/docker-compose/docker-compose.yaml
创建8个目录，模拟每个实例2块硬盘。
$ ls
data1-1  data1-2  data2-1  data2-2  data3-1  data3-2  data4-1  data4-2  minio  test01

此处对其加以修改(修改了nginx配置)：
vim docker-compose.yaml
version: '3.7'

# starts 4 docker containers running minio server instances.
# using nginx reverse proxy, load balancing, you can access
# it through port 9000.
services:
  minio1:
    image: minio/minio:RELEASE.2021-01-16T02-19-44Z
    volumes:
      - /home/minio/data1-1:/data1
      - /home/minio/data1-2:/data2
    ports:
      - "9001:9000"
    expose:
      - "9000"
    environment:
      MINIO_ROOT_USER: minio
      MINIO_ROOT_PASSWORD: minio123
    command: server http://minio{
      
      1...4}/data{
      
      1...2}
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3

  minio2:
    image: minio/minio:RELEASE.2021-01-16T02-19-44Z
    volumes:
      - /home/minio/data2-1:/data1
      - /home/minio/data2-2:/data2
    ports:
      - "9002:9000"
    expose:
      - "9000"
    environment:
      MINIO_ROOT_USER: minio
      MINIO_ROOT_PASSWORD: minio123
    command: server http://minio{
      
      1...4}/data{
      
      1...2}
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3

  minio3:
    image: minio/minio:RELEASE.2021-01-16T02-19-44Z
    volumes:
      - /home/minio/data3-1:/data1
      - /home/minio/data3-2:/data2
    ports:
      - "9003:9000"
    expose:
      - "9000"
    environment:
      MINIO_ROOT_USER: minio
      MINIO_ROOT_PASSWORD: minio123
    command: server http://minio{
      
      1...4}/data{
      
      1...2}
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3

  minio4:
    image: minio/minio:RELEASE.2021-01-16T02-19-44Z
    volumes:
      - /home/minio/data4-1:/data1
      - /home/minio/data4-2:/data2
    ports:
      - "9004:9000"
    expose:
      - "9000"
    environment:
      MINIO_ROOT_USER: minio
      MINIO_ROOT_PASSWORD: minio123
    command: server http://minio{
      
      1...4}/data{
      
      1...2}
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3

  nginx:
    image: nginx:1.19.6
    volumes:
      - ./nginx.conf:/etc/nginx/conf.d/default.conf:ro
    network_mode: "host"
    depends_on:
      - minio1
      - minio2
      - minio3
      - minio4

创建nginx配置文件
该操作也可以省略掉，直接通过ip:port 来访问minio即可。

可以下载该配置文件并加以修改
$ wget https://raw.githubusercontent.com/minio/minio/master/docs/orchestration/docker-compose/nginx.conf

# vim nginx.conf 
upstream minio-server {
      
      
 server 10.214.58.67:9001 weight=3;
 server 10.214.58.67:9002 weight=3;
 server 10.214.58.67:9003 weight=3;
 server 10.214.58.67:9004 weight=3;
 }
server {
      
      
 listen 9000;
 server_name minio;
 # To allow special characters in headers
 ignore_invalid_headers off;
 # Allow any size file to be uploaded.
 # Set to a value such as 1000m; to restrict file size to a specific value
 client_max_body_size 0;
 # To disable buffering
 proxy_buffering off;

 location / {
      
      
   proxy_set_header X-Real-IP $remote_addr;
   proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
   proxy_set_header X-Forwarded-Proto $scheme;
   proxy_set_header Host $http_host;

   proxy_connect_timeout 300;
   # Default is HTTP/1, keepalive is only enabled in HTTP/1.1
   proxy_http_version 1.1;
   proxy_set_header Connection "";
   chunked_transfer_encoding off;

   proxy_pass http:///minio-server; # If you are using docker-compose this would be the hostname i.e. minio
   # Health Check endpoint might go here. See https://www.nginx.com/resources/wiki/modules/healthcheck/
   # /minio/health/live;
 }
}

创建单机集群

# docker-compose -f docker-compose.yaml up -d
/usr/local/lib/python2.7/dist-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release.
  from cryptography.hazmat.backends import default_backend
Starting minio_minio3_1 ... done
Starting minio_minio4_1 ... done
Starting minio_minio2_1 ... done
Starting minio_minio1_1 ... done
Starting minio_nginx_1  ... done

创建集群后，可以直接通过http://10.214.58.67:9000 访问web ui，如下：
该ui提供了用户登录，增删bucket，上传、删除、分享文件等基础功能，复杂的用户权限可以通过mc admin来设置。在这里插入图片描述

3 测试

3.1 mc使用

下载mc

$ wget https://dl.min.io/client/mc/release/linux-amd64/mc
$ chmod +x mc
建议直接cp一份到/usr/local/bin 目录下
$ mc --autocompletion # 添加自动补全
添加自动补全后会在 $HOME/.mc/ 下生成一系列配置文件
$ ls .mc/
certs  config.json  session  share

设置别名

$ mc alias set local http://localhost:9001 minio minio123
Added `local` successfully.

别名会直接在  $HOME/.mc/config.json  中添加一个别名记录，记录中包括url，accessKey，secretKey等信息，也可以参考案例手动添加.
上面命令直接添加了一个local别名，其在 .mc/config.json  中添加了{
      
      "aliases":{
      
      "local":{
      
      别名信息}}} 记录，记录信息如下：
 	"local": {
      
      
 		"url": "http://localhost:9001",
 		"accessKey": "minio",
 		"secretKey": "minio123",
 		"api": "s3v4",
 		"path": "auto"
 	},

查看bucket

$ mc ls local
[2021-01-19 20:10:46 HKT]     0B test01/
[2021-01-19 20:10:59 HKT]     0B test02/

查看使用率

$ mc du --depth=2 local
216B	test01
72KiB	test02
72KiB
$ mc du local/test01

拷贝文件

$ dd if=/dev/zero of=test_file.txt bs=1M count=100
$ mc cp test_file.txt local/test01/
test_file.txt:                100.00 MiB / 100.00 MiB ┃▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓┃ 25.92 MiB/s 3s
$ mc du local/test01
100MiB	test01

具体数据存储格式如下，可见数据被分散到多个节点中，每个节点中的数据又再次被分为多个part。
$ du data1-1/test01/ -h
26M	data1-1/test01/test_file.txt/d30cd52d-09f7-452a-ac1e-19b9bb1cf5cd
26M	data1-1/test01/test_file.txt
38M	data1-1/test01/test_file2.txt/9585ada5-1deb-46c0-9e07-bd261a715450
38M	data1-1/test01/test_file2.txt
63M	data1-1/test01/
minio04:~$ du data1-1/test01/ --max-depth=1 -h
26M	data1-1/test01/test_file.txt
38M	data1-1/test01/test_file2.txt
63M	data1-1/test01/
minio04:~$ ls data1-1/test01/test_file.txt/
d30cd52d-09f7-452a-ac1e-19b9bb1cf5cd  xl.meta
minio04:~$ ls data1-1/test01/test_file.txt/d30cd52d-09f7-452a-ac1e-19b9bb1cf5cd/
part.1  part.10  part.2  part.3  part.4  part.5  part.6  part.7  part.8  part.9

查看集群信息

$ mc admin info local
或者直接json形式输出
$ mc admin info local --json

集群信息如下：
在这里插入图片描述

更多用法参考 github mc

3.2 fuse使用

s3fs可以直接将minio挂载为文件系统形式，更加方便使用。

下载s3fs
```
$ apt-get install s3fs
```

挂载 minio 指定bucket

$ touch ${HOME}/.passwd-s3fs
内容为： 
minio:minio123
$ mkdir -p /home/minio/test01
$ s3fs test01 /home/minio/test01 -o passwd_file=${HOME}/.passwd-s3fs -o url=http://127.0.0.1:9001/ -o use_path_request_style
该命令将bucket test01 挂载到 /home/minio/test01 目录
$ df -h|grep test01
s3fs                    256T     0  256T   0% /home/minio/test01

注： 也可以直接在fstab 中添加挂载信息
mybucket /path/to/mountpoint fuse.s3fs _netdev,allow_other,use_path_request_style,url=https://url.to.s3/ 0 0
但是此种方法需要在etc下配置全局的password：
echo ACCESS_KEY_ID:SECRET_ACCESS_KEY > /etc/passwd-s3fs
chmod 600 /etc/passwd-s3fs

4 补充说明

软件环境
笔者系统为ubuntu16.04
docker 20.10.2
docker-compose version 1.26.2
参考文档
1 minio 官网
2 doc CN
3 doc EN
4 github mc
5 github s3fs

minio笔记01--部署与测试minio