Kudu calculation formula:
Assuming: The maximum data storage capacity of 5
tablet servers is 32 , and after replication and compression, each tablet server is 9.6TB. Each tablet server manages 3000 tablets, including a copy of the tablet.
Based on the above restrictions, the following can be inferred:
Partition = size of each row * total number of
tablets <= 3.2G tablets (limited to 1500 partitions per tablet-server), currently the total number of kudu tables is 1546, and a total of 32 tablet-servers
3000*32/1546=62, each table cannot exceed 62 partitions, including replicas 62/3 20 partitions
The total size of each table, partition table total points size
3.2 20 = 64G, recommended that each table can not exceed the total size 64G
The total amount of data stored in Kudu is recommended as: the total number of tablet servers. The amount of data for a single tablet server = 32 24TB 0.4=614TB/3 = 102TB
. The amount of data for a single tablet is: the amount of data for a single tablet server / the amount of tablet in each tablet server Total=24TB 0.4/3000=3.2G.