在Docker中安装ElasticSearch时出现no space left on device问题原因及解决办法

曾几何时,使用docker pull docker.elastic.co/elasticsearch/elasticsearch:7.10.2命令拉取ES时,本来一切挺正常的,但拉取进程走到大约一半时,却冷不丁出现磁盘空间不足的异常。详细记录如下:

[root@localhost /]# docker pull docker.elastic.co/elasticsearch/elasticsearch:7.10.2
7.10.2: Pulling from elasticsearch/elasticsearch
ddf49b9115d7: Downloading [======>                                            ]  11.35MB/84.72MB
e736878e27ad: Downloading [=============>                                     ]  6.554MB/25.12MB
7487c9dcefbe: Download complete
9ccb7e6e1f0c: Downloading [==================================================>]  345.7MB/345.7MB
dcec6dec98db: Waiting
8a10b4854661: Waiting
1e595aee1b7d: Waiting
06cc198dbf22: Waiting
55b9b1b50ed8: Waiting
write /var/lib/docker/tmp/GetImageBlob549619752: no space left on device

先亮出底牌吧, 解决方案就是: 修改文件C:\Users\Administrator\.vagrant.d\boxes\centos-VAGRANTSLASH-7\2004.01\virtualbox\Vagrantfile中vagrant的同步目录。详情如下:

Vagrantfile原始的原始配置为:

Vagrant.configure("2") do |config|
  config.vm.base_mac = "5254004d77d3"
  config.vm.synced_folder ".", "/vagrant", type: "rsync"
end

新建目录C:\Users\Administrator\vagrantSync,并将上述配置文件第三行修改为:

config.vm.synced_folder "./vagrantSync", "/vagrant", type: "rsync"

参考博文: 《使用Vagrant 后发现虚拟机磁盘空间爆满的血泪填坑记》(并在此向博主“大厨无盐煮”致谢!)
~~~~~~~~~~~~~~~~~~~~~~~一条不甘做咸鱼的分割线--------------------------------------------
以下是小编呕心沥血寻找到问题端倪,幸运搜到上述博文,从而最终药到病除的悲惨经历:
出现空间不足异常时,使用docker infodf -hl命令查看Docker的根目录及根目录空间使用情况:

[root@localhost vagrant]# docker info
......
Docker Root Dir: /var/lib/docker

[root@localhost /]# df -lh /var/lib/docker/
FilesystemSize  Used Avail Use% Mounted on
/dev/sda1        40G   40G   28M 100% /
......

居然这么快就用了40G,我可是什么事都没干,造化弄人?
为了洗清这不白之冤,我决心要查个水落石出。
接着,查看CentOS7的根目录空间剩余情况:

[root@localhost /]# df -hl
Filesystem      Size  Used Avail Use% Mounted on
devtmpfs        912M     0  912M   0% /dev
tmpfs           919M     0  919M   0% /dev/shm
tmpfs           919M  8.6M  911M   1% /run
tmpfs           919M     0  919M   0% /sys/fs/cgroup
/dev/sda1        40G   40G   28M 100% /
tmpfs           184M     0  184M   0% /run/user/1000
overlay          40G   40G   28M 100% /var/lib/docker/overlay2/825c0e959e357450e22584d4c640d86b14112b233661c61350b5c5547c41ed3a/merged
overlay          40G   40G   28M 100% /var/lib/docker/overlay2/421dcb93c267339d1748b863098df0815d71d72315dea15691bbf941a83bc468/merged

这两个overlay2是什么鬼?吃了我40G的空间!
回忆此前所做的一系列操作,难道对应着我安装过的MySQL和Redis?但此时我并没有找到二者和本文问题的关联。
多方尝试,并四处碰壁之后,尝试参考博文《docker容器存放目录磁盘空间满了》,将docker的默认目录从/var/lib/docker修改到了/home/docker/lib/docker。
修改docker目录并重启docker成功之后,使用docker imagesdocker ps查看之前下载的MySQL和Redis镜像及其运行情况,二者均已踪影全无。此时,感觉已经凉凉,之前自己的捣鼓数据库所做的努力或许已经付诸东流了。
绝望之际,尝试搜了一下/dev/sda1导致磁盘爆满的原因,看到了一篇名为《vagrant中虚拟机linux的/dev/sda1硬盘满了》的文章提供了一条查询大文件的命令:du -h --max-depth=1,就在CentOS7的根目录里试了一下,结果如下:

[root@localhost /]# du -h --max-depth=1
0       ./dev
du: cannot access ‘./proc/1514/task/1514/fd/3’: No such file or directory
du: cannot access ‘./proc/1514/task/1514/fdinfo/3’: No such file or directory
du: cannot access ‘./proc/1514/fd/4’: No such file or directory
du: cannot access ‘./proc/1514/fdinfo/4’: No such file or directory
0       ./proc
8.6M    ./run
0       ./sys
36M     ./etc
48K     ./root
130M    ./var
0       ./tmp
1.2G    ./usr
27M     ./boot
477M    ./home
0       ./media
0       ./mnt
0       ./opt
0       ./srv
36G     ./vagrant
208M    ./mydata
40G     .

看到结果中的“36G ./vagrant”突然眼前一亮,难道./vagrant就是问题症结所在?当时居然还小小激动了一阵!于是,赶紧追踪:

[root@localhost /]# cd ./vagrant/
[root@localhost vagrant]# ls
20201119_222738.mp4    Music                                                                                         source
20201204_215951.mp4    My Documents                                                                                  SpringCloudAlibaba-xxx.mp4
20201217_224243.mp4    MySQL数据结构-xxx.mp4                                                                      SpringCloud.mp4
20201218_202319.mp4    nacos                                                                                         SpringCloud-xxx.mp4
3D Objects             NetHood                                                                                       spring源码1-xxx.mp4
anaconda3              node_modules                                                                                  spring源码2-xxx.mp4
AppData                NTUSER.DAT{
    
    18549daa-3d78-11e9-85da-005056c00008}.TM.blf                                       Templates
Application Data       NTUSER.DAT{
    
    18549daa-3d78-11e9-85da-005056c00008}.TMContainer00000000000000000001.regtrans-ms  UIDowner
Contacts               NTUSER.DAT{
    
    18549daa-3d78-11e9-85da-005056c00008}.TMContainer00000000000000000002.regtrans-ms  Vagrantfile
Cookies                ntuser.ini                                                                                    Videos
Desktop                ntuser.pol                                                                                    VirtualBox VMs
Documents              OneDrive                                                                                      workspace
Downloads              package-lock.json                                                                             Workspaces
eclipse                Pictures                                                                                      xxx创新型资源.mp4
EVAVEdit               PrintHood                                                                                     多线程-可见性、有序性、原子性.mp4
Favorites              PUTTY.RND                                                                                     「开始」菜单
IntelGraphicsProfiles  PycharmProjects                                                                               微服务演化过程-xxx1.mp4
Links                  Recent                                                                                        微服务演化过程-xxx2.mp4
Local Settings         Redis-MySQL-锁.mp4                                                                            微服务结构设计-xxx2.mp4
logs                   Saved Games                                                                                   微服务结构设计-xxx.mp4
MicrosoftEdgeBackups   Searches
mm.cfg                 SendTo

看着这些熟悉的文件名和文件夹,我终于知道了问题的主要原因:这些文件和文件夹和Vagrantfile文件同处于C:\Users\Administrator中。
于是,讲这些视频文件转移到别处,重启虚拟机,看到了一丝曙光,终于腾出了一些空间。
接着,就尝试重新创建Redis实例并重启:

[root@localhost /]# docker run -p 6379:6379 --name redis -v /mydata/redis/data:/data     -v /mydata/redis/conf/redis.conf:/etc/redis/redis.conf     -d redis redis-server /etc/redis/redis.conf
Unable to find image 'redis:latest' locally
latest: Pulling from library/redis
a076a628af6f: Pull complete
f40dd07fe7be: Pull complete
ce21c8a3dbee: Pull complete
ee99c35818f8: Pull complete
56b9a72e68ff: Pull complete
3f703e7f380f: Pull complete
Digest: sha256:0f97c1c9daf5b69b93390ccbe8d3e2971617ec4801fd0882c72bf7cad3a13494
Status: Downloaded newer image for redis:latest
874e61200f4e3f18eee354f33eb0d8340fd6110d120942895443b7cfc77b9ee3
[root@localhost /]# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES
874e61200f4e        redis               "docker-entrypoint.s…"   49 seconds ago      Up 44 seconds       0.0.0.0:6379->6379/tcp   redis
[root@localhost /]# cat /mydata/redis/conf/redis.conf
appendonly yes
[root@localhost /]# docker exec -it redis redis-cli
127.0.0.1:6379> get a
"12341234123"

居然成功恢复了Redis,并且数据也没有丢失!此时,一下子来了信心,顿感胜利在望!
紧接着尝试恢复MySQL:

[root@localhost mydata]# docker run -p 3306:3306 --name mysql -v /mydata/mysql/log:/var/log/mysql -v /mydata/mysql/data:/var/lib/mysql -v /mydata/mysql/conf:/etc/mysql -e MYSQL_ROOT_PASSWORD=root -d mysql:5.7
Unable to find image 'mysql:5.7' locally
5.7: Pulling from library/mysql
a076a628af6f: Already exists
f6c208f3f991: Retrying in 8 seconds
f6c208f3f991: Retrying in 5 seconds

看着上面的进度逐渐停顿,并且查看空间剩余情况,发现再次爆满,我的心情也再次跌下悬崖!
之后,查看一篇文章说,可以将C:\Users\Administrator\.vagrant.d\boxes\centos-VAGRANTSLASH-7\2004.01\virtualbox\Vagrantfile中的type: "rsync"修改为:type: “virtualbox”。测试了一下,重启虚拟时提示异常:

mount -t vboxsf -o uid=1000,gid=1000 vagrant /vagrant
The error output from the command was:
mount: unknown filesystem type 'vboxsf'

尝试在宿主机中安装了vagrant-vbguest插件:

C:\Users\Administrator>vagrant plugin install vagrant-vbguest
Installing the 'vagrant-vbguest' plugin. This can take a few minutes...
Fetching micromachine-3.0.0.gem
Fetching vagrant-vbguest-0.29.0.gem
Installed the plugin 'vagrant-vbguest (0.29.0)'!

然而,并未解决问题。

此时,脑子里的问号越积越多,试着搜索了一下“vagrant为什么会把物理机上的文件当做虚拟机的文件”,偶然发现了博文 《Vagrant将C盘用户下的文件挂载到vagrant目录里,导致虚拟机空间爆满》,并且此文标明了对 《使用Vagrant 后发现虚拟机磁盘空间爆满的血泪填坑记》的参考。

按照上述神一般的文章的指引,新建文件夹:C:\Users\Administrator\vagrantSync,并将C:\Users\Administrator\.vagrant.d\boxes\centos-VAGRANTSLASH-7\2004.01\virtualbox\Vagrantfile中的:
config.vm.synced_folder ".", "/vagrant", type: "rsync"修改为:config.vm.synced_folder "./vagrantSync", "/vagrant", type: "rsync",问题迎刃而解。

文末疑问: type为"rsync"据说可以保证很高的共享速度,那么,除了“rsync”共享类型之外,还有没有其它类型呢?

猜你喜欢

转载自blog.csdn.net/shinyolive/article/details/113577905