Minio entry series [1] Windows/Linux/K8S stand-alone deployment of Minio

1 Introduction to Minio

MinIO is an object storage service based on the Apache License v2.0 open source agreement. It is compatible with the Amazon S3 cloud storage service interface and is very suitable for storing large-capacity unstructured data, such as pictures, videos, log files, backup data and container/virtual machine images, etc., and an object file can be of any size, ranging from several Ranges from kb to a maximum of 5T.

MinIO is a very lightweight service that can be easily integrated with other applications, such as NodeJS, Redis or MySQL. Currently supports JavaScript, Java, Python, Golang, and .NET.

MinIO is a high-performance, distributed object storage system. It is a software product that can run 100% on standard hardware. Even low-cost machines such as X86 can also run MinIO well.

MinIO is different from traditional storage and other object storage in that it designs its software architecture from the beginning for private cloud standards with higher performance requirements. Because MinIO was only designed for object storage from the beginning. So he designed it in a more user-friendly way. It can realize all the functions required for object storage and is more powerful in performance. It will not compromise for more business functions and lose the ease of use and efficiency of MinIO. . The benefit of this result is that it can more easily implement native object storage services with elastic scalability.

MinIO excels in traditional object storage use cases such as secondary storage, disaster recovery and archiving. At the same time, it is also unique in storage technology in machine learning, big data, private cloud, hybrid cloud, etc. Of course, support for data analysis, high-performance application loads, and native cloud is not excluded.

In China: More than 9,000 companies such as Alibaba, Tencent, Baidu, China Unicom, Huawei, China Mobile, etc. are also using MinIO products

MinIO is now also a member of CNCF and is one of the current solutions together with ceph and others in the cloud native storage part.

2 Minio advantages

2.1 Erasure Coding (elimination coding)

MinIO protects data using per-object inline erasure coding written in assembly code to provide the highest possible performance. MinIO uses Reed-Solomon code to strip objects into data and parity blocks with user-configurable redundancy levels. MinIO's erasure coding performs repair at the object level and can repair multiple objects independently.

With a maximum parity of N/2, MinIO is implemented to ensure uninterrupted read and write operations for only ((N/2)+1) operational drives in the deployment. For example, in a 12-drive setup, MinIO shards objects across 6 data drives and 6 parity drives and can reliably write new objects or rebuild existing objects, leaving only 7 drives in the deployment

2.2 Bitrot Protection

Silent data corruption or bit rot is a serious problem faced by disk drives that causes data to become corrupted without the user's knowledge. The causes are varied (age of the drive, current spikes, disk firmware errors, phantom writes, misreads/writes, driver errors, accidental overwrites) but the result is the same - corrupted data.

MinIO's optimized implementation of the HighwayHash algorithm ensures that it never reads corrupted data - it catches and repairs corrupted objects on the fly. Integrity is ensured end-to-end by computing the hash on read and validating it from the application, network, and memory/drive on write. This implementation is designed for speed and can achieve hash speeds in excess of 10 GB/sec on a single core of an Intel CPU.

2.3 Encryption

Encrypting data in flight is one thing; protecting data at rest is another. MinIO supports multiple sophisticated server-side encryption schemes to protect data – no matter where it resides. MinIO's approach ensures confidentiality, integrity, and authenticity with negligible performance overhead. Server-side and client-side encryption is supported using AES-256-GCM, ChaCha20-Poly1305, and AES-CBC.

Encrypted objects are tamper-proof using AEAD server-side encryption. Additionally, MinIO is compatible and tested with all commonly used key management solutions such as HashiCorp Vault. MinIO uses a Key Management System (KMS) to support SSE-S3.

If the client requests SSE-S3, or if automatic encryption is enabled, the MinIO server encrypts each object using a unique object key that is protected by a KMS-managed master key. Given the extremely low overhead, automatic encryption can be turned on for each application and instance.

2.4 Worms

When WORM is enabled, MinIO disables all APIs that may change object data and metadata. This means that data is tamper-proof once written. This has practical applications for many different regulatory requirements.

2.5 Identity management

MinIO supports the most advanced standards in identity management and integrates with OpenID connect compliant providers as well as major external IDP vendors. This means access is centralized and passwords are temporary and rotated rather than stored in configuration files and databases. Additionally, access policies are fine-grained and highly configurable, which means supporting multi-tenant and multi-instance deployments becomes very simple.

2.6 Continuous replication

The challenge with traditional replication methods is that they cannot scale efficiently beyond a few hundred TiB. That being said, everyone needs a replication strategy to support disaster recovery, and that strategy needs to span geographies, data centers, and clouds.

MinIO's continuous replication is designed for large-scale, cross-data center deployments. By leveraging Lambda calculation notifications and object metadata, it can calculate deltas efficiently and quickly. Lambda notifications ensure that changes are propagated immediately, unlike traditional batch mode.

Continuous replication means that if a failure occurs, data loss is kept to a minimum - even in the face of highly dynamic data sets. Finally, like everything MinIO does, continuous replication is multi-vendor, meaning your backup location can be anywhere from a NAS to a public cloud.

2.7 Global Alliance

Data in modern enterprises is everywhere. MinIO allows these different instances to be combined into a unified global namespace. Specifically, any number of MinIO servers can be combined into a Distributed Mode set, and multiple Distributed Mode sets can be combined into a MinIO Server Federation. Each MinIO Server Federation provides unified management and namespace.

MinIO federated servers support an unlimited number of distributed mode sets. The impact of this approach is that object storage can scale massively for large, geographically distributed enterprises while retaining the ability to accommodate a variety of applications (Splunk, Teradata, Spark, Hive, Presto, TensorFlow, H20) from a single console.

2.8 Multi-cloud gateway

All enterprises are adopting multi-cloud strategies. This also includes private clouds. Therefore, your bare metal virtualized containers and public cloud services (including non-S3 providers such as Google, Microsoft, and Alibaba) must look exactly the same. While modern applications are highly portable, the data that powers these applications is not.

Making data available, no matter where it resides, is the main challenge MinIO solves. MinIO runs on bare metal, network attached storage, and every public cloud. What's more, MinIO ensures that your view of that data is exactly the same from both an application and management perspective via the Amazon S3 API.

With MinIO, you can go even further and make your existing storage infrastructure compatible with Amazon S3. The impact is far-reaching. Now, organizations can truly unify their data infrastructure - from files to blocks, all exposed as objects accessible through the Amazon S3 API, without the need for migration.

3 Minio architecture

MinIO is designed to be cloud native and can run as lightweight containers managed by external orchestration services such as Kubernetes. The entire server is a static binary of approximately 40MB and is very efficient in using CPU and memory resources - even under heavy load. The result is that you can co-host a large number of tenants on shared hardware.

MinIO runs on commodity servers with locally attached drives (JBOD/JBOF). All servers in the cluster have the same capabilities (completely symmetrical architecture). There are no namenodes or metadata servers.

MinIO writes data and metadata together as objects, eliminating the need for a metadata database. Additionally, MinIO performs all functions (erasure code, bit rot checking, encryption) as inline, strictly consistent operations. The result is that MinIO is very resilient.

Each MinIO cluster is a collection of distributed MinIO servers, one process per node. MinIO runs in user space as a single process and uses lightweight coroutines to achieve high concurrency. Drives are grouped into wipe sets (16 drives per group by default), and objects are placed in these sets using a deterministic hashing algorithm.

MinIO is designed for large-scale, multi-data center cloud storage services. Each tenant runs their own MinIO cluster, fully isolated from other tenants, allowing them to be protected from any disruption from upgrades, updates and security events. Each tenant scales independently through cross-geo federated clustering.

4 Windows installation

4.1 Download

Download address: http://dl.minio.org.cn/server/minio/release/

4.2 Open cmd in the minio.exe directory

4.3 Configure parameters and start

# 设置用户名
set MINIO_ACCESS_KEY=admin
# 设置密码（8位）
set MINIO_SECRET_KEY=admin123
# 指定启动端口(未指定默认9000)及存储位置
minio.exe  server  --address 0.0.0.0:9999 D:/data

4.4 Log in to the address IP+9999, enter the user name and password, and the construction is completed.

5 Linux installation

5.1 Download

5.2 Create related directories

# 创建文件存储目录
mkdir -p /data/minio/data
# 创建程序存放目录，并上传minio至此目录
mkdir -p /data/minio
cd /data/minio
# 修改可读权限
chmod +x minio 
# 设置用户名
export  MINIO_ACCESS_KEY=admin
# 设置密码
export  MINIO_SECRET_KEY=admin123

5.3 Start and visit the homepage

 ./minio server /data/minio/data --console-address ":33639" --address ":9000"

5.4 Docker

docker run -p 9999:9000 -e "MINIO_ACCESS_KEY=admin" -e "MINIO_SECRET_KEY=admin123" -v /data/minio:/data -d minio/minio server /data

5.5 Docker compose

version: '3'
services:
   minio:
    container_name: minio
    hostname: minio
    image: "minio/minio:latest"
    volumes:
      - /data/minio:/data
    ports:
      - "9999:9000"
    environment:
      MINIO_ACCESS_KEY: admin
      MINIO_SECRET_KEY: admin123
      TZ: Asia/Shanghai
    command: server /data
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3

6 K8S

Key points: Use yml files to configure resources and use PVC to store minio files

6.1 Create NFS on node 192.168.58.103

# 安装nfs-utils rpcbind
yum install nfs-utils nfs-common rpcbind 
# 创建目录
mkdir -p /data/minio
chmod 666 /data/minio/
chown nfsnobaby /data/minio/
# 添加访问策略
vi /etc/exports
# 输入内容
/data/minio  *(rw,no_root_squash,no_all_squash,sync)
# 启动
systemctl start rpcbind
systemctl start nfs
# 查看
showmount -e

6.2 Create PV (master execution)

# vim minio-pv.yaml 
# 添加
apiVersion: v1
kind: PersistentVolume
metadata:
    name: pv-minio
spec:
    capacity:
      storage: 2Gi
    accessModes:
      - ReadWriteMany
    persistentVolumeReclaimPolicy: Recycle
    nfs:
      path: /data/minio
      server: 192.168.58.103
# 创建
kubectl create -f minio-pv.yaml 
# 查看
kubectl get pv -o wide

6.3 Create PVC (executed by master)

# 
vim minio-pvc.yaml 
# 内容
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-minio
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
# 查看
kubectl get pvc -o wide 

6.4 Create pod and service (executed by master)

# 
vim minio.yaml 
# 添加
apiVersion: v1
kind: Service
metadata:
  name: minio-svc
  labels:
    app: minio
spec:
  type: NodePort
  ports:
  - name: minio-port
    protocol: TCP
    port: 9000
    nodePort: 32600
    targetPort: 9000
  selector:
    app: minio
---
apiVersion: v1
kind: Pod
metadata:
  name: minio
  labels:
    app: minio
spec:
  containers:
  - name: minio
    env:
    - name: MINIO_ACCESS_KEY
      value: "admin"
    - name: MINIO_SECRET_KEY
      value: "admin123"
    image: minio/minio:latest
    args:
    - server
    - /data
    ports:
    - name: minio
      containerPort: 9000
    volumeMounts:
    - name: data
      mountPath: /data
  volumes:
  - name: data
    persistentVolumeClaim:
     claimName: pvc-minio
# 创建
kubectl apply  -f minio.yaml 
# 查看pod 
kubectl get pods -o wide
# 查看service
kubectl get svc -o wide 

6.5 Verification

Because the service uses NodePort, you need to enter the IP+32600 of the node where minio is located to access

file storage: upload a file on the page, enter the server where nfs is located, and then find that the file has been stored in the PVC