centos7 安装ceph12.2.5

1.环境及设置：

ceph-node1 192.168.0.232 这个节点同时用作ceph-deploy
ceph-node2 192.168.0.250
ceph-node3 192.168.0.220
操作系统： centos7.4
内核：3.10.0-693.21.1.el7.x86_64
关闭防火墙，selinux

安装ceph-deploy：

cat >/etc/yum.repos.d/ceph.repo
[ceph-noarch]
name=Ceph noarch packages
baseurl=https://download.ceph.com/rpm/el7/noarch
enabled=1
gpgcheck=1
type=rpm-md
gpgkey=https://download.ceph.com/keys/release.asc

更新操作系统并安装ceph-deploy，其实两台也需要update，不需要安装ceph-deploy

sudo yum update
sudo yum install ceph-deploy

在所有节点创建ceph用户

sudo useradd -d /home/ceph-node -m ceph-node
sudo passwd ceph-node
echo "ceph-node ALL = (root) NOPASSWD:ALL" | sudo tee /etc/sudoers.d/ceph-node
sudo chmod 0440 /etc/sudoers.d/ceph-node

在ceph-deploy节点创建密钥并分发到各节点

ssh-keygen
ssh-copy-id ceph-node@ceph-node2
ssh-copy-id ceph-node@ceph-node1

修改ssh配置文件默认ssh连接是用ceph-node用户

Host ceph-node1
   Hostname ceph-node1
   User ceph-node
Host ceph-node2
   Hostname ceph-node2
   User ceph-node
Host ceph-node3
   Hostname ceph-node3
   User ceph-node

每个节点设置时间同步

yum -y install ntpdate
crontab -e
*/5 * * * * /usr/sbin/ntpdate ntp1.aliyun.com 

2.安装ceph

切换到ceph-deploy节点

mkdir /root/my-cluster
cd /root/my-cluster

如果安装过程有错误或者失败可以用以下命令从头再来

ceph-deploy purge {ceph-node} [{ceph-node}]
ceph-deploy purgedata {ceph-node} [{ceph-node}]
ceph-deploy forgetkeys
rm ceph.*

创建集群

ceph-deploy new ceph-node1 ceph-node2 ceph-node3
vim ceph.conf #加入公共网络网段
public network = 192.168.0.0/24

安装ceph

ceph-deploy install --release luminous ceph-node1 ceph-node2 ceph-node3

部署mon生成密钥文件

ceph-deploy mon create-initial

发送配置文件

ceph-deploy admin ceph-node1 ceph-node2 ceph-node3

创建mgr节点，mgr需要高可用所以创建三个节点

ceph-deploy mgr create ceph-node1
ceph-deploy mgr create ceph-node2
ceph-deploy mgr create ceph-node3

创建osd

ceph-deploy osd create --data /dev/sdb ceph-node1
ceph-deploy osd create --data /dev/sdc ceph-node1
ceph-deploy osd create --data /dev/sdd ceph-node1
ceph-deploy osd create --data /dev/sdb ceph-node2
ceph-deploy osd create --data /dev/sdc ceph-node2
ceph-deploy osd create --data /dev/sdd ceph-node2
ceph-deploy osd create --data /dev/sdb ceph-node3
ceph-deploy osd create --data /dev/sdc ceph-node3
ceph-deploy osd create --data /dev/sdd ceph-node3

使用ceph -s查看集群是否健康，osd是否加入节点

添加额外mon节点，mon节点也需要高可用

ceph-deploy mon add ceph-node2 ceph-node3

至此ceph集群安装完成

3.安装常见错误

• 错误：

  [ceph_deploy][ERROR ] RuntimeError: connecting to host: n1 resulted in errors: IOError cannot send (already closed?)
  解决方法：
  编辑visudo注释下面的
  Defaults    requiretty

• 错误：

  [n2][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
  解决方法：
  在ceph.conf添加
  public network = 192.168.122.0/24

• 错误：

  ceph-deploy安装的时候到yum安装那报错
  解决：可能是源有问题执行yum -y remove centos-release-ceph