k8s环境信息:
| 主机名 | 操作系统 | 私有IP | 可用存储设备 |
|---|---|---|---|
| k8s-0001 | CentOS 7.5 64bit | 172.16.192.127 | |
| k8s-0002 | CentOS 7.5 64bit | 172.16.192.25 | |
| k8s-0003 | CentOS 7.5 64bit | 172.16.192.121 | |
| k8s-0004 | CentOS 7.5 64bit | 172.16.192.243 | /dev/vdb |
| k8s-0005 | CentOS 7.5 64bit | 172.16.192.226 | /dev/vdb |
[root@k8s-0001 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
172.16.192.121 Ready master 3d v1.11.1
172.16.192.127 Ready master 3d v1.11.1
172.16.192.226 Ready node 3d v1.11.1
172.16.192.243 Ready node 3d v1.11.1
172.16.192.25 Ready master 3d v1.11.1
所有节点开启ip_forward:
cat <<EOF > /etc/sysctl.d/ceph.conf
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system
打标签
运行ceph-mon的节点打上：ceph-mon=enabled:
kubectl label nodes {172.16.192.243,172.16.192.226} ceph-mon=enabled
运行ceph-osd的节点，也就是存储节点，打上：ceph-osd=enabled:
kubectl label nodes {172.16.192.243,172.16.192.226} ceph-osd=enabled
运行ceph-mgr的节点打上：ceph-mgr=enabled
kubectl label nodes 172.16.192.243 ceph-mgr=enabled
部署operator:
git clone https://github.com/rook/rook.git
cd rook
git checkout -b release-0.9 remotes/origin/release-0.9
cd cluster/examples/kubernetes/ceph
# edit operator.yaml if you need
kubectl create -f operator.yaml
部署cluster:
kubectl apply -f cluster.yaml
创建storageclass
kubectl apply -f storageclass.yaml
kubectl get storageclasses.storage.k8s.io -n rook-ceph
配置Rook Dashboard
cluster.yaml 中dashboard: enabled: true 配置为 true，就会自动创建 Dashboard 服务，但默认为ClusterIP服务类型。
kubectl get svc -n rook-ceph | grep mgr-dashboard
kubectl create -f dashboard-external-https.yaml
kubectl get svc -n rook-ceph | grep mgr-dashboard
访问登录页面
用户名为admin，获取密码：
kubectl -n rook-ceph get secret rook-ceph-dashboard-password -o jsonpath='{.data.password}' | base64 --decode