Generating orphans pod of
Under circumstances where OOM node or nodes abnormal crash, pod failed to clean up orphaned normally caused.
Tip follows
Orphaned pod found - but volume paths are still present on disk
K8s the pod into the directory
cd /var/lib/kubelet/pods/
Solve the problem
Orphans pod filter the log, delete the pod.
#!/bin/bash
num=$(grep "errors similar to this. Turn up verbosity to see them." /var/log/messages |tail -1 | awk '{print $12}' |sed 's/"//g')
echo $num
while [ $num ]
do
[ -d "/var/lib/kubelet/pods/${num}" ] && rm -rf /var/lib/kubelet/pods/${num}
sleep 2s
num=$(grep "errors similar to this. Turn up verbosity to see them." /var/log/messages |tail -1 | awk '{print $12}' |sed 's/"//g')
[ -d "/var/lib/kubelet/pods/${num}" ] || num=
echo "$num remaining"
done
However, this method has some 危险性
, it is not confirmed whether there is 数据丢失
risk, if confirmed, then execute.
If you have already mounted the PVC and other related storage, perform umount and then delete.
Go to view the log, you will find syslog not painted like a log.