Stein Kolla deployment using the OpenStack version (including storage section Ceph)

Stein Kolla deployment using the OpenStack version (including storage section Ceph)
This is my micro letter, there is a problem you can add the exchange
and to the introduction of Kolla not elaborate too much here, there are many online documents. This article is for the latest version of Kolla, can not solve a single-node deployment ceph-osd problem (found in the multi-node test can not be deployed Ceph-osd). In the latest version of kolla, the official website has been given notice to the shelf Ceph, the specific information in the following figure:
Stein Kolla deployment using the OpenStack version (including storage section Ceph)
This piece of article will not solve the problem by modifying the deployment ceph kolla-ansible source.
The deployment process is as follows:

1. Preparations

1.1 Installation Centos7.5 system here. I use virtualbox create a virtual machine, you can use two cards. The first for the management network, you can access the Internet, use the second network as an example, start without the need to configure IP.

[root@openstack ~]# cat /etc/centos-release
CentOS Linux release 7.5.1804 (Core)

1.2 turn off the firewall, set selinux

[root@localhost ~]# systemctl disable firewalld
[root@localhost ~]# systemctl stop firewalld
[root@localhost ~]# getenforce 
Disabled

1.3 restart the machine

[root@localhost ~]# reboo7

1.4 Setting the hostname

[root@localhost ~]# hostnamectl set-hostname openstack
[root@localhost ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.137.2 openstack

1.5 Configuring DNS

[root@openstack ~]# cat /etc/resolv.conf
nameserver 114.114.114.114

Ali added 1.6 yum source

[root@openstack ~]# rm -rfv * /etc/yum.repos.d/*
[root@openstack ~]# curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo

2. Basic System Configuration

2.1 Installation common tools

[root@openstack ~]# yum install vim bash-completion net-tools

2.2 Installation epel

[root@openstack ~]# yum install epel-release

2.3 install dependencies

[root@openstack ~]#yum install python-devel libffi-devel gcc openssl-devel libselinux-python -y

2.4 Installation pip

[root@openstack ~]# yum install python-pip

2.5 Configuring pip, using Ali source

[root@openstack ~]# mkdir ~/.pip
[root@openstack ~]# vim ~/.pip/pip.conf 
[global]
index-url = https://mirrors.aliyun.com/pypi/simple/
[install]
trusted-host=mirrors.aliyun.com

2.6 update pip

[root@openstack ~]# pip install -U pip

3. Install docker

3.1 Add docker source, a source Ali

[root@openstack ~]#yum install -y yum-utils device-mapper-persistent-data lvm2
[root@openstack ~]#yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

3.2 Installation docker

[root@openstack ~]#yum -y install docker-ce

3.3 docker, using Ali accelerator

[root@openstack ~]# mkdir -p /etc/docker/
[root@openstack ~]# vim /etc/docker/daemon.json 
{
  "registry-mirrors": ["https://fl791z1h.mirror.aliyuncs.com"]
}

3.4 Add kolla.conf

[root@openstack ~]# mkdir -p /etc/systemd/system/docker.service.d/
[root@openstack ~]# vim /etc/systemd/system/docker.service.d/kolla.conf 
[Service]
MountFlags=shared

3.5 start docker

[root@openstack ~]# systemctl daemon-reload 
[root@openstack ~]# systemctl enable docker 
[root@openstack ~]# systemctl start docker

4. Installation ansible

4.1 Installation ansible

[root@openstack ~]# yum install ansible

4.2 Configuration ansible

[root@openstack ~]# vim /etc/ansible/ansible.cfg 
[defaults]
host_key_checking=False
pipelining=True
forks=100

5. Installation kolla-ansible

5.1 Installation depends

kolla divided into two warehouses, one for https://github.com/openstack/kolla, the other is https://github.com/openstack/kolla-ansible have requirements.txt file in both the warehouse, need wherein the mounting dependencies

[root@openstack ~]# vim 1.txt 
pbr!=2.1.0,>=2.0.0 # Apache-2.0
docker>=2.4.2 # Apache-2.0
Jinja2>=2.8 # BSD License (3 clause)
GitPython>=1.0.1 # BSD License (3 clause)
six>=1.10.0 # MIT
oslo.config>=5.1.0 # Apache-2.0
oslo.utils>=3.33.0 # Apache-2.0
setuptools!=24.0.0,!=34.0.0,!=34.0.1,!=34.0.2,!=34.0.3,!=34.1.0,!=34.1.1,!=34.2.0,!=34.3.0,!=34.3.1,!=34.3.2,!=36.2.0,>=21.0 # PSF/ZPL
netaddr>=0.7.18 # BSD
[root@openstack ~]# vim 2.txt 
pbr!=2.1.0,>=2.0.0 # Apache-2.0
docker>=2.4.2 # Apache-2.0
Jinja2>=2.10 # BSD License (3 clause)
six>=1.10.0 # MIT
oslo.config>=5.2.0 # Apache-2.0
oslo.utils>=3.33.0 # Apache-2.0
setuptools!=24.0.0,!=34.0.0,!=34.0.1,!=34.0.2,!=34.0.3,!=34.1.0,!=34.1.1,!=34.2.0,!=34.3.0,!=34.3.1,!=34.3.2,!=36.2.0,>=21.0.0 # PSF/ZPL
PyYAML>=3.12 # MIT
netaddr>=0.7.18 # BSD
cryptography>=2.1 # BSD/Apache-2.0
jmespath>=0.9.3 # MIT

Above two documents, I direct the new requirements.txt, copied, can be installed directly

root@openstack ~]# pip install -r 1.txt 
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Looking in indexes: https://mirrors.aliyun.com/pypi/simple/
Requirement already satisfied: pbr!=2.1.0,>=2.0.0 in /usr/lib/python2.7/site-packages (from -r 1.txt (line 1)) (5.4.2)
Requirement already satisfied: docker>=2.4.2 in /usr/lib/python2.7/site-packages (from -r 1.txt (line 2)) (4.0.2)
Requirement already satisfied: Jinja2>=2.8 in /usr/lib64/python2.7/site-packages (from -r 1.txt (line 3)) (2.10.1)
Requirement already satisfied: GitPython>=1.0.1 in /usr/lib/python2.7/site-packages (from -r 1.txt (line 4)) (3.0.2)
ERROR: Package 'GitPython' requires a different Python: 2.7.5 not in '>=3.0, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*'

在安装的时候会提示GitPython无法安装,可以使用pip指定GitPython进行安装

[root@openstack ~]# pip install GitPython==3.0.2

安装第二个依赖

[root@openstack ~]# pip install -r 2.txt

5.2 安装kolla-ansible

[root@openstack ~]# pip install kolla-ansible

在安装kolla-ansible的时候会出现下面的错误

ERROR: Cannot uninstall 'PyYAML'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.

解决方法,在安装kolla-ansible的时候忽略PyYAML

[root@openstack ~]# pip install kolla-ansible --ignore-installed PyYAML

5.3 安装kolla

[root@openstack ~]# pip install kolla

6. 配置kolla-ansible

6.1 创建kolla目录,复制globals.yml,passwords.yml

[root@openstack ~]# mkdir -p /etc/kolla/
[root@openstack ~]# cp /usr/share/kolla-ansible/etc_examples/kolla/* /etc/kolla/

6.2 由于此环境在虚拟机部署,所以virt_type的类型设置为qemu

[root@openstack ~]# mkdir -p /etc/kolla/config/nova/
[root@openstack ~]# vim /etc/kolla/config/nova/nova-compute.conf 
[libvirt]
virt_type = qemu
cpu_mode = none

6.3 配置ceph.conf,设备ceph副本数,此环境为一台节点,3个OSD盘,设置副本为1即可,如果是三台节点,9OSD盘,副本为3。

[root@openstack ~]# vim /etc/kolla/config/ceph.conf
[global]
osd pool default size = 1
osd pool default min size = 1

6.4 配置globals.yml

[root@openstack ~]# cat /etc/kolla/globals.yml  | grep -v ^$| grep -v ^#
kolla_base_distro: "centos"
kolla_install_type: "source"
openstack_release: "stein"
kolla_internal_vip_address: "192.168.137.254"
docker_registry: "registry.cn-hangzhou.aliyuncs.com"
docker_namespace: "openstackbl"
network_interface: "enp0s3"
neutron_external_interface: "enp0s8"
neutron_plugin_agent: "linuxbridge"
enable_ceph: "yes"
ceph_osd_store_type: "bluestore"
enable_ceph_rgw: "yes"
enable_cinder: "yes"
enable_neutron_qos: "yes"
enable_ceph_rgw_keystone: "yes"
glance_backend_ceph: "yes"
glance_enable_rolling_upgrade: "no"
cinder_backend_ceph: "{{ enable_ceph }}"
nova_backend_ceph: "{{ enable_ceph }}"
ironic_dnsmasq_dhcp_range:
tempest_image_id:
tempest_flavor_ref_id:
tempest_public_network_id:
tempest_floating_network_name:

以上的配置中,本人使用了自己在阿里上搭建的docker仓库。里面的镜像,是从docker的官网push而来。这个仓库是私有仓库,要是需要使用的话,请联系本作者。
注:如果此环境所在的网络中还有其它使用Kolla部署的openstack,那么需要修改globals.yml中的

#keepalived_virtual_router_id: "51"

把51改成1到99的之间的整数即可。

7 部署openstack

7.1 本环境使用Ceph作为后端存储,需要添加磁盘,做OSD,并且修改硬盘标签

[root@openstack ~]# lsblk 
NAME            MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda               8:0    0  200G  0 disk 
├─sda1            8:1    0    1G  0 part /boot
└─sda2            8:2    0  199G  0 part 
  ├─centos-root 253:0    0  150G  0 lvm  /
  ├─centos-swap 253:1    0  7.9G  0 lvm  [SWAP]
  └─centos-home 253:2    0 41.1G  0 lvm  /home
sdb               8:16   0  100G  0 disk 
sdc               8:32   0  100G  0 disk 
sdd               8:48   0    100G  0 disk 

可以看到有三块空白盘,修改这三块盘的标签

[root@openstack ~]# parted /dev/sdb -s -- mklabel gpt mkpart KOLLA_CEPH_OSD_BOOTSTRAP 1 -1
[root@openstack ~]# parted /dev/sdc -s -- mklabel gpt mkpart KOLLA_CEPH_OSD_BOOTSTRAP 1 -1
[root@openstack ~]# parted /dev/sdd -s -- mklabel gpt mkpart KOLLA_CEPH_OSD_BOOTSTRAP 1 -1
[root@openstack ~]# lsblk 
NAME            MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda               8:0    0  200G  0 disk 
├─sda1            8:1    0    1G  0 part /boot
└─sda2            8:2    0  199G  0 part 
  ├─centos-root 253:0    0  150G  0 lvm  /
  ├─centos-swap 253:1    0  7.9G  0 lvm  [SWAP]
  └─centos-home 253:2    0 41.1G  0 lvm  /home
sdb               8:16   0  100G  0 disk 
└─sdb1            8:17   0  100G  0 part 
sdc               8:32   0  100G  0 disk 
└─sdc1            8:33   0  100G  0 part 
sdd               8:48   0    100G  0 disk 
└─sdd1            8:49   0    100G  0 part 

7.2 生成openstack密码

[root@openstack ~]# kolla-genpwd

7.3 pull镜像

[root@openstack ~]# kolla-ansible pull
...........
PLAY RECAP ************************************************************************************************************************************************
localhost                  : ok=40   changed=0    unreachable=0    failed=0    skipped=77   rescued=0    ignored=0 

出现以上提示,pull成功。如果在pull中出现问题,可以在评论中留言

7.4 检查Kolla

[root@openstack ~]# kolla-ansible prechecks
PLAY RECAP ************************************************************************************************************************************************
localhost                  : ok=102  changed=0    unreachable=0    failed=0    skipped=147  rescued=0    ignored=0 

出现以上提示,prechecks成功。如果在检查的过程中,有遇到问题,请在评论区留言。

7.5 部署Kolla

[root@openstack ~]# kolla-ansible deploy -vvv
TASK [ceph : Enable ceph dashboard] ***************************************************************************************************************************************
task path: /usr/share/kolla-ansible/ansible/roles/ceph/tasks/start_ceph_dashboard.yml:2
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: root
<localhost> EXEC /bin/sh -c 'echo ~root && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1567411390.78-165388867051828 `" && echo ansible-tmp-1567411390.78-165388867051828="` echo /root/.ansible/tmp/ansible-tmp-1567411390.78-165388867051828 `" ) && sleep 0'
Using module file /usr/lib/python2.7/site-packages/ansible/modules/commands/command.py
<localhost> PUT /root/.ansible/tmp/ansible-local-7809exM0Cg/tmpteinHV TO /root/.ansible/tmp/ansible-tmp-1567411390.78-165388867051828/AnsiballZ_command.py
<localhost> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1567411390.78-165388867051828/ /root/.ansible/tmp/ansible-tmp-1567411390.78-165388867051828/AnsiballZ_command.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python /root/.ansible/tmp/ansible-tmp-1567411390.78-165388867051828/AnsiballZ_command.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /root/.ansible/tmp/ansible-tmp-1567411390.78-165388867051828/ > /dev/null 2>&1 && sleep 0'
fatal: [localhost]: FAILED! => {
    "changed": false, 
    "cmd": [
        "docker", 
        "exec", 
        "ceph_mon", 
        "ceph", 
        "mgr", 
        "module", 
        "enable", 
        "dashboard"
    ], 
    "delta": "0:00:00.990347", 
    "end": "2019-09-02 04:03:11.952770", 
    "invocation": {
        "module_args": {
            "_raw_arams": "docker exec ceph_mon ceph mgr module enable dashboard", 
            "_uses_shell": false, 
            "argv": null, 
            "chdir": null, 
            "creates": null, 
            "executable": null, 
            "removes": null, 
            "stdin": null, 
            "stdin_add_newline": true, 
            "strip_empty_ends": true, 
            "warn": true
        }
    }, 
    "msg": "non-zero return code", 
    "rc": 2, 
    "start": "2019-09-02 04:03:10.962423", 
    "stderr": "Error ENOENT: all mgr daemons do not support module 'dashboard', pass --force to force enablement", 
    "stderr_lines": [
        "Error ENOENT: all mgr daemons do not support module 'dashboard', pass --force to force enablement"
    ], 
    "stdout": "", 
    "stdout_lines": []
}

NO MORE HOSTS LEFT ********************************************************************************************************************************************************

PLAY RECAP ****************************************************************************************************************************************************************
localhost                  : ok=194  changed=111  unreachable=0    failed=1    skipped=105  rescued=0    ignored=0 

加入-vvv参数可以在部署的过程进行调试。在部署此步时提示如上错误,部署中断。解决方法:

[root@openstack ~]# vim /usr/share/kolla-ansible/ansible/roles/ceph/tasks/start_ceph_dashboard.yml 
---
- name: Enable ceph dashboard
  become: true
  command: docker exec ceph_mon ceph mgr module enable dashboard  --force
  changed_when: false
  run_once: true

在command最后加上 --force,即可解决此问题。
把刚才部署好的容器卸载,然后重新部署:

[root@openstack ~]# kolla-ansible destroy --yes-i-really-really-mean-it
Destroy Kolla containers, volumes and host configuration : ansible-playbook -i /usr/share/kolla-ansible/ansible/inventory/all-in-one -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla  /usr/share/kolla-ansible/ansible/destroy.yml 
[DEPRECATION WARNING]: The TRANSFORM_INVALID_GROUP_CHARS settings is set to allow bad characters in group names by default, this will change, but still be user 
configurable on deprecation. This feature will be removed in version 2.10. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
 [WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details

PLAY [Apply role destroy] *************************************************************************************************************************************************

TASK [Gathering Facts] ****************************************************************************************************************************************************
ok: [localhost]

TASK [destroy : include_tasks] ********************************************************************************************************************************************
included: /usr/share/kolla-ansible/ansible/roles/destroy/tasks/validate_docker_execute.yml for localhost

TASK [destroy : Ensure the docker service is running] *********************************************************************************************************************
changed: [localhost]

TASK [destroy : include_tasks] ********************************************************************************************************************************************
included: /usr/share/kolla-ansible/ansible/roles/destroy/tasks/cleanup_containers.yml for localhost

TASK [destroy : Destroying all Kolla containers and volumes] **************************************************************************************************************
changed: [localhost]

TASK [destroy : include_tasks] ********************************************************************************************************************************************
included: /usr/share/kolla-ansible/ansible/roles/destroy/tasks/cleanup_images.yml for localhost

TASK [destroy : Removing Kolla images] ************************************************************************************************************************************
skipping: [localhost]

TASK [destroy : include_tasks] ********************************************************************************************************************************************
included: /usr/share/kolla-ansible/ansible/roles/destroy/tasks/cleanup_host.yml for localhost

TASK [destroy : Destroying Kolla host configuration] **********************************************************************************************************************
changed: [localhost]

PLAY RECAP ****************************************************************************************************************************************************************
localhost                  : ok=8    changed=3    unreachable=0    failed=0    skipped=1    rescued=0    ignored=0   

[root@openstack ~]#
[root@openstack ~]# kolla-ansible deploy -vvv

Stein Kolla deployment using the OpenStack version (including storage section Ceph)
Stein Kolla deployment using the OpenStack version (including storage section Ceph)
During the deployment, we can see that in the implementation of ceph-osd-related tasks, there is no proper feedback execution, specifically as shown above. At this point we can see through the ceph osd tree command

[root@openstack ~]# docker exec ceph_mon ceph osd tree
ID CLASS WEIGHT TYPE NAME    STATUS REWEIGHT PRI-AFF 
-1            0 root default
[root@openstack ~]# docker exec ceph_mon ceph df
RAW STORAGE:
    CLASS     SIZE     AVAIL     USED     RAW USED     %RAW USED 
    TOTAL      0 B       0 B      0 B          0 B             0 

POOLS:
    POOL          ID     STORED     OBJECTS     USED     %USED     MAX AVAIL 
    .rgw.root      1        0 B           0      0 B         0           0 B 
    images         2        0 B           0      0 B         0           0 B 
    volumes        3        0 B           0      0 B         0           0 B 
    backups        4        0 B           0      0 B         0           0 B

By looking, we learned osd currently ceph cluster is not running.
After a while, the deployment is complete, there is no error.

PLAY [Apply role blazar] **************************************************************************************************************************************************
skipping: no hosts matched

PLAY RECAP ****************************************************************************************************************************************************************
localhost                  : ok=411  changed=234  unreachable=0    failed=0    skipped=238  rescued=0    ignored=0   

[root@openstack ~]#

Probably a little more than ten minutes, openstack deployment is complete, there is no error.

7.6 generation admin-openrc.sh

[root@openstack ~]# kolla-ansible post-deploy

7.7 Upload Mirror

As a glance ceph stored image format to use raw, as follows

[root@openstack ~]# yum install qemu-img -y
[root@openstack ~]# qemu-img convert -f qcow2 -O raw cirros-0.3.4-x86_64-disk.img cirros.raw
[root@openstack ~]# source /etc/kolla/admin-openrc.sh
[root@openstack ~]# glance image-create --name cirros --disk-format raw --container-format bare --progress < cirros.raw 
[========>                     ] 29%

In 29% of the time to upload, upload. By looking reason, stars, glance Ceph used as back-end storage, the available space is zero, the following is a glance ceph state log

2019-09-02 04:24:15.349 24 WARNING glance_store._drivers.rbd [req-7ee511be-930b-4e41-9c08-e1a90c375fe2 1148205e534a4832ba8700ea8b410534 d722008e46fa4ef799c68c766b26781b - default default] since image size is zero we will be doing resize-before-write for each chunk which will be considerably slower than normal
[root@openstack ~]# docker exec ceph_mon ceph -s
  cluster:
    id:     055889bd-64a5-4129-a853-340d82bcddc5
    health: HEALTH_WARN
            Reduced data availability: 40 pgs inactive

  services:
    mon: 1 daemons, quorum 192.168.137.2 (age 14m)
    mgr: localhost(active, since 14m)
    osd: 0 osds: 0 up, 0 in

  data:
    pools:   5 pools, 40 pgs
    objects: 0 objects, 0 B
    usage:   0 B used, 0 B / 0 B avail
    pgs:     100.000% pgs unknown
             40 unknown

[root@openstack ~]#

We can see the ceph pg belong to unknown state, indicating that the ceph cluster can not be used.

8. Modify the source kolla-ansible

Well Cluster 8.1 before modifying the source code, we have deployed unloading

[root@openstack ~]# kolla-ansible destroy --yes-i-really-really-mean-it
Destroy Kolla containers, volumes and host configuration : ansible-playbook -i /usr/share/kolla-ansible/ansible/inventory/all-in-one -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla  /usr/share/kolla-ansible/ansible/destroy.yml 
[DEPRECATION WARNING]: The TRANSFORM_INVALID_GROUP_CHARS settings is set to allow bad characters in group names by default, this will change, but still be user 
configurable on deprecation. This feature will be removed in version 2.10. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
 [WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details

PLAY [Apply role destroy] *************************************************************************************************************************************************

TASK [Gathering Facts] ****************************************************************************************************************************************************
ok: [localhost]

TASK [destroy : include_tasks] ********************************************************************************************************************************************
included: /usr/share/kolla-ansible/ansible/roles/destroy/tasks/validate_docker_execute.yml for localhost

TASK [destroy : Ensure the docker service is running] *********************************************************************************************************************
changed: [localhost]

TASK [destroy : include_tasks] ********************************************************************************************************************************************
included: /usr/share/kolla-ansible/ansible/roles/destroy/tasks/cleanup_containers.yml for localhost

TASK [destroy : Destroying all Kolla containers and volumes] **************************************************************************************************************
changed: [localhost]

TASK [destroy : include_tasks] ********************************************************************************************************************************************
included: /usr/share/kolla-ansible/ansible/roles/destroy/tasks/cleanup_images.yml for localhost

TASK [destroy : Removing Kolla images] ************************************************************************************************************************************
skipping: [localhost]

TASK [destroy : include_tasks] ********************************************************************************************************************************************
included: /usr/share/kolla-ansible/ansible/roles/destroy/tasks/cleanup_host.yml for localhost

TASK [destroy : Destroying Kolla host configuration] **********************************************************************************************************************
changed: [localhost]

PLAY RECAP ****************************************************************************************************************************************************************
localhost                  : ok=8    changed=3    unreachable=0    failed=0    skipped=1    rescued=0    ignored=0

8.2 modify bootstrap_osds.yml

Modify the first line 6, line 19

[root@openstack ~]# vim /usr/share/kolla-ansible/ansible/roles/ceph/tasks/bootstrap_osds.yml
-a "partition_name='KOLLA_CEPH_OSD_BOOTSTRAP' match_mode='prefix' use_udev={{ kolla_ceph_use_udev }}"
-a "partition_name='KOLLA_CEPH_OSD_CACHE_BOOTSTRAP' match_mode='prefix' use_udev={{ kolla_ceph_use_udev }}"

Notes 39-42 lines, comment lines 100-102. Add in the back row 42,102

#    - not item.external_journal | bool
#    - item.device.split('/')[2] in ansible_devices  # if there is no device in setup (like loopback, we don't need to warn user
#    - ansible_devices[item.device.split('/')[2]].partitions|count > 1
#    - ceph_osd_wipe_disk != "yes-i-really-really-mean-it"
     - item.external_journal | bool == False
     - ansible_devices[item.device.split('/')[2]].partitions|count > 1
     - ceph_osd_wipe_disk != "yes-i-really-really-mean-it"

#     - not item.external_journal | bool
#     - ansible_devices[item.device.split('/')[2]].partitions|count > 1
#     - ceph_osd_wipe_disk != "yes-i-really-really-mean-it"
      - item.external_journal | bool == False
      - ansible_devices[item.device.split('/')[2]].partitions|count > 1
      -  ceph_osd_wipe_disk != "yes-i-really-really-mean-it"

8.3 modify start_osds.yml

Modify the sixth line

[root@openstack ~]# vim /usr/share/kolla-ansible/ansible/roles/ceph/tasks/start_osds.yml
-a "partition_name='KOLLA_CEPH_DATA_BS' match_mode='prefix' use_udev={{ kolla_ceph_use_udev }}"

9. redeployment openstack

The implementation of the deployment command

[root@openstack ~]# kolla-ansible deploy -vvv

View the deployment process is not started osd, as shown below
Stein Kolla deployment using the OpenStack version (including storage section Ceph)
Stein Kolla deployment using the OpenStack version (including storage section Ceph)
can be seen through the task, osd disk has been created successfully. Osd container can be viewed, ceph following command status

[root@openstack ~]# docker ps | grep ceph
3b765534475d        registry.cn-hangzhou.aliyuncs.com/openstackbl/centos-source-ceph-rgw:stein          "dumb-init --single-…"   About a minute ago   Up About a minute                       ceph_rgw
7b326a2844f2        registry.cn-hangzhou.aliyuncs.com/openstackbl/centos-source-ceph-osd:stein          "dumb-init --single-…"   About a minute ago   Up About a minute                       ceph_osd_2
6abde420cd43        registry.cn-hangzhou.aliyuncs.com/openstackbl/centos-source-ceph-osd:stein          "dumb-init --single-…"   About a minute ago   Up About a minute                       ceph_osd_1
3576d8cf558c        registry.cn-hangzhou.aliyuncs.com/openstackbl/centos-source-ceph-osd:stein          "dumb-init --single-…"   About a minute ago   Up About a minute                       ceph_osd_0
2658dfc87737        registry.cn-hangzhou.aliyuncs.com/openstackbl/centos-source-ceph-mgr:stein          "dumb-init --single-…"   2 minutes ago        Up 2 minutes                            ceph_mgr
c3a97ff4a395        registry.cn-hangzhou.aliyuncs.com/openstackbl/centos-source-ceph-mon:stein          "dumb-init --single-…"   2 minutes ago        Up 2 minutes                            ceph_mon
[root@openstack ~]# docker exec -it ceph_mon ceph -s
  cluster:
    id:     055889bd-64a5-4129-a853-340d82bcddc5
    health: HEALTH_WARN
            too few PGs per OSD (18 < min 30)

  services:
    mon: 1 daemons, quorum 192.168.137.2 (age 3m)
    mgr: localhost(active, since 3m)
    osd: 3 osds: 3 up (since 2m), 3 in (since 2m)
    rgw: 1 daemon active (radosgw.gateway)

  data:
    pools:   7 pools, 56 pgs
    objects: 219 objects, 1.2 KiB
    usage:   3.0 GiB used, 297 GiB / 300 GiB avail
    pgs:     56 active+clean

[root@openstack ~]#

After about ten minutes, openstack successful deployment

PLAY [Apply role blazar] **************************************************************************************************************************************************
skipping: no hosts matched

PLAY RECAP ****************************************************************************************************************************************************************
localhost                  : ok=415  changed=237  unreachable=0    failed=0    skipped=234  rescued=0    ignored=0

So far ceph-osd successfully created. After a few moments, after openstack deployment is successful, then the following operation

10. Test openstack

10.1 Upload Mirror

[root@openstack ~]# kolla-ansible post-deploy
[root@openstack ~]# source /etc/kolla/admin-openrc.sh 
[root@openstack ~]# glance image-create --name cirros --disk-format raw --container-format bare --progress < cirros.raw 
[=============================>] 100%
+------------------+----------------------------------------------------------------------------------+
| Property         | Value                                                                            |
+------------------+----------------------------------------------------------------------------------+
| checksum         | 56730d3091a764d5f8b38feeef0bfcef                                                 |
| container_format | bare                                                                             |
| created_at       | 2019-09-02T09:18:32Z                                                             |
| disk_format      | raw                                                                              |
| id               | 07ab1c4e-d16d-4b55-b1f1-017a0699662e                                             |
| locations        | [{"url": "rbd://055889bd-64a5-4129-a853-340d82bcddc5/images/07ab1c4e-d16d-       |
|                  | 4b55-b1f1-017a0699662e/snap", "metadata": {}}]                                   |
| min_disk         | 0                                                                                |
| min_ram          | 0                                                                                |
| name             | cirros                                                                           |
| os_hash_algo     | sha512                                                                           |
| os_hash_value    | 34f5709bc2363eafe857ba1344122594a90a9b8cc9d80047c35f7e34e8ac28ef1e14e2e3c13d55a4 |
|                  | 3b841f533435e914b01594f2c14dd597ff9949c8389e3006                                 |
| os_hidden        | False                                                                            |
| owner            | 544d94b977234b0295726b0c7163c3e8                                                 |
| protected        | False                                                                            |
| size             | 41126400                                                                         |
| status           | active                                                                           |
| tags             | []                                                                               |
| updated_at       | 2019-09-02T09:18:34Z                                                             |
| virtual_size     | Not available                                                                    |
| visibility       | shared                                                                           |
+------------------+----------------------------------------------------------------------------------+
[root@openstack ~]#

You can see the image can be uploaded successfully

10.2 view mirror in the ceph

[root@openstack ~]# docker exec ceph_mon rbd -p images ls
07ab1c4e-d16d-4b55-b1f1-017a0699662e
[root@openstack ~]# docker exec ceph_mon rbd -p images info 07ab1c4e-d16d-4b55-b1f1-017a0699662e
rbd image '07ab1c4e-d16d-4b55-b1f1-017a0699662e':
    size 39 MiB in 5 objects
    order 23 (8 MiB objects)
    snapshot_count: 1
    id: 11939304e103
    block_name_prefix: rbd_data.11939304e103
    format: 2
    features: layering, exclusive-lock, object-map, fast-diff, deep-flatten
    op_features: 
    flags: 
    create_timestamp: Mon Sep  2 05:18:33 2019
    access_timestamp: Mon Sep  2 05:18:33 2019
    modify_timestamp: Mon Sep  2 05:18:33 2019
[root@openstack ~]#

Storage path can be seen as a mirror ceph images pool

11. Q&A

Guess you like

Origin blog.51cto.com/11229048/2434717