After the devstack server is powered off and restarted, some openstack services cannot be started.
source /opt/stack/devstack/openrc admin admin
nova service-list --binary nova-compute
/opt/stack/devstack/tools/discover_hosts.sh
carried out:
systemctl restart devstack@*
After restarting the service, the nova service is in the up state. However, the cinder-volume service started to be in the up state, and after a while it was in the down state.
View the status of the cinder service:
cinder service-list
Start the cinder service up and
query the log:
sudo journalctl -a -u [email protected] | grep "ERROR"
display:
Aug 25 12:04:05 computer cinder-volume[5124]: ERROR cinder.utils [None req-5f09d1d8-9f85-4f77-82f0-eff577d4de0e None None] Volume driver LVMVolumeDriver not initialized
Aug 25 12:04:05 computer cinder-volume[5124]: ERROR cinder.volume.manager [None req-5f09d1d8-9f85-4f77-82f0-eff577d4de0e None None] Cannot complete RPC initialization because driver isn't initialized properly.: cinder.exception.DriverNotInitialized: Volume driver not ready.
Aug 25 12:04:15 computer cinder-volume[5124]: ERROR cinder.service [-] Manager for service cinder-volume computer@lvmdriver-1 is reporting problems, not sending heartbeat. Service will appear "down".
Execute after a while:
cinder service-list
Output:
+------------------+------------------------+------+---------+-------+----------------------------+-----------------+
| Binary | Host | Zone | Status | State | Updated_at | Disabled Reason |
+------------------+------------------------+------+---------+-------+----------------------------+-----------------+
| cinder-scheduler | controller | nova | enabled | up | 2020-08-25T06:19:08.000000 | - |
| cinder-volume | controller@lvmdriver-1 | nova | enabled | down | 2020-08-25T06:19:00.000000 | - |
| cinder-volume | computer@lvmdriver-1 | nova | enabled | down | 2020-08-25T06:19:06.000000 | - |
+------------------+------------------------+------+---------+-------+----------------------------+-----------------+
At first I thought it was a time synchronization problem, but after checking the time, I found that it was not a time synchronization problem.
Execute under the controller:
vgs
no output.
Under computer,
execute:
vgs
There is also no output.
Respectively execute under controller and computer:
sudo losetup /dev/loop0 /opt/stack/data/stack-volumes-default-backing-file
Output:
losetup: /opt/stack/data/stack-volumes-default-backing-file: failed to set up loop device: Device or resource busy
Restart the server:
repeat the above steps
Execute again (#Tie /opt/stack/data/stack-volumes-lvmdriver-1-backing-file to the first free loop):
sudo losetup -f /opt/stack/data/stack-volumes-lvmdriver-1-backing-file
And then execute separately
vgs
display:
VG #PV #LV #SN Attr VSize VFree
stack-volumes-lvmdriver-1 1 1 0 wz--n- <24.00g <1.15g
Restart the cinder service again:
systemctl restart devstack@c*
cinder service-list
display:
+------------------+------------------------+------+---------+-------+----------------------------+-----------------+
| Binary | Host | Zone | Status | State | Updated_at | Disabled Reason |
+------------------+------------------------+------+---------+-------+----------------------------+-----------------+
| cinder-scheduler | controller | nova | enabled | up | 2020-08-25T06:19:08.000000 | - |
| cinder-volume | controller@lvmdriver-1 | nova | enabled | up | 2020-08-25T06:19:00.000000 | - |
| cinder-volume | computer@lvmdriver-1 | nova | enabled | up | 2020-08-25T06:19:06.000000 | - |
+------------------+------------------------+------+---------+-------+----------------------------+-----------------+
problem solved.
However, the specific cause of the problem remains to be investigated.
Reference document:
"cinder-volume down"
Delete the redundant cinder volume:
cinder-manage service remove cinder-volume computer@lvmdriver-2