MongoDB内存限制

用过MongoDB的人应该会发现一个问题，就是随着时间的推移，MongoDB占用的物理内存会越来越大，甚至到达不可想象的地步。或者在短时间内，用压力测试MongoDB的性能，内存也会飚的很高，而且会一直保持在最高的状态。

下面的测试是在我的虚拟机上进行的，配置是1核4G，压力测试工具是YCSB，测试命令是

./bin/ycsb load mongodb -s -P workloads/workloada -threads 50 > outputLoad.txt

./bin/ycsb run mongodb -s -P workloads/workloada -threads 50 > outputRun.txt

workloada文件里的测试数据总量改成了100w，测试结果如下：

可以看到MongoDB占用内存达到了33.5%，虽然只有1.3G，但是在32G的服务器上占用的比例也可以达到30%~40%，也就是10G左右，这样就容易影响到其他进程的正常运行。

看官网上关于内存的介绍：

https://docs.mongodb.com/v3.4/core/wiredtiger/index.html

Memory Use

With WiredTiger, MongoDB utilizes both the WiredTiger internal cache and the filesystem cache.

Starting in 3.4, the WiredTiger internal cache, by default, will use the larger of either:

50% of (RAM - 1 GB), or
256 MB.

By default, WiredTiger uses Snappy block compression for all collections and prefix compression for all indexes. Compression defaults are configurable at a global level and can also be set on a per-collection and per-index basis during collection and index creation.

Different representations are used for data in the WiredTiger internal cache versus the on-disk format:

Data in the filesystem cache is the same as the on-disk format, including benefits of any compression for data files. The filesystem cache is used by the operating system to reduce disk I/O.
Indexes loaded in the WiredTiger internal cache have a different data representation to the on-disk format, but can still take advantage of index prefix compression to reduce RAM usage. Index prefix compression deduplicates common prefixes from indexed fields.
Collection data in the WiredTiger internal cache is uncompressed and uses a different representation from the on-disk format. Block compression can provide significant on-disk storage savings, but data must be uncompressed to be manipulated by the server.

Via the filesystem cache, MongoDB automatically uses all free memory that is not used by the WiredTiger cache or by other processes.

To adjust the size of the WiredTiger internal cache, seestorage.wiredTiger.engineConfig.cacheSizeGB and --wiredTigerCacheSizeGB. Avoid increasing the WiredTiger internal cache size above its default value.

从3.4版本开始，默认情况下，WieldGigd内部缓存将使用下面2种中更大的一种：50% of (RAM - 1 GB) 和256 MB。通过文件系统缓存，MongoDB的自动使用未被wiredtiger缓存或由其他进程使用所有可用内存。调整WiredTiger内部缓存的方法：storage.wiredTiger.engineConfig.cacheSizeGB 和 --wiredTigerCacheSizeGB

看来不设置的话，默认会使用50% of (RAM - 1 GB)的内存。于是在配置文件设置了storage.wiredTiger.engineConfig.cacheSizeGB为0.5，也就是500M，再看测试结果：

vim /etc/mongod.conf

storage:
  dbPath: /var/lib/mongodb
  journal:
    enabled: true
#  engine:
#  mmapv1:
  wiredTiger:
    engineConfig:
      cacheSizeGB: 0.5

可以看到，MongoDB所占的物理内存稳定在了630M左右，说明设置确实生效了。

Memory Use

猜你喜欢