Statistical analysis [original] big data base of Kudu (6) kudu tserver memory footprint

After the memory for high kudu tserver will reject write requests, log is as follows:

19/06/01 13:34:12 INFO AsyncKuduClient: Invalidating location 34b1c13d04664cc8bae6689d39b08b77($kudu_tserver:7050) for tablet 858c055c456549569af77d14eaf997e5: Service unavailable: Soft memory limit exceeded (at 92.36% of capacity). See https://kudu.apache.org/releases/1.7.0-cdh5.16.1/docs/troubleshooting.html

1 View tserver detailed memory footprint

http: // kudu_tserver $ 8050 / mem-trackers

Page is divided into two parts:

Process memory usage for the overall occupancy

Memory usage by subsystem is detailed accounting, which at first glance more complicated, in fact, is a tree structure to cascade through the parent,

For example, root is fully occupied as 1.6G, root below logcache takes up 277K, and then look down

block_cache-sharded_lru_cache occupied 246M, server occupation 1.4G, server below is divided into a number of tablet, in addition to the tablet, as well as

log_block_manager occupied 438M;

 

If you find some tablet take up more memory, which want to see in the next table, you can adopt the following commands:

 

View all 2 tablet on the tserver

sudo -u should be remote_replica list $ kudu_tserver

such as:

Tablet id: e064f41775084680ab269b3cb3c21c76
State: RUNNING
Table name: impala::test_db.test_table1
Partition: RANGE (dt) PARTITION 20190531 <= VALUES < 20190532
Estimated on disk size: 30.71M
Schema: Schema [
        0:co1[int32 NOT NULL],
        1:co2[string NOT NULL],
        2:co3[string NOT NULL],
        3:co4[string NOT NULL],
        4:co5[string NULLABLE]
]
...

 

关于kudu的内存估算，和数据量大小以及tablet多少以及活跃replica多少都有关系，详见：

https://kudu.apache.org/docs/scaling_guide.html#memory