1. The purpose of the preparation of documents
Yarn CDH use in dynamic resource pools, users will distinguish the distribution of cluster resources based on time (eg: in the night hours tend to cluster resources are mainly run batch jobs, daytime businesses tend to cluster resources are mainly real-time computing jobs). How to configure for such needs in the CDH? This article describes how to plan Fayson rules CM Configuration Yarn dynamic resource pools.
- Content Overview
1. Create a resource pool configuration set
2. Modify the resource allocation for each set and verify the configuration
3. Summary
- test environment
1.CM and CDH version 5.15
2. Create a resource pool configuration set
The default configuration set only a resource pool in CDH cluster, the next Fayson create multiple resource pools configuration set by the CM, Fayson create a total of three in this example configuration set
- 21:00 to 9:00 in the morning using pool_nigth resource pool configuration set
- 9 am to 18:00 using pool_day resource pool configuration set
- Other time periods set using the default resource pool configuration
1. Log into the CM dynamic resource pool management interface
2. Click the "planning mode" to enter the configuration interface
3. Click the "Create Program Rules", create two new Configuration Set
Need to be copied from an existing configuration set when you create a new configuration set, you can select the configuration set repetition period "daily, weekly, monthly," repetition time in hours to choose.
4. The configuration set list has been created as follows
Just finished the above steps a plurality of configuration set unreasonable periods of use.
3. Modify each configuration set resource allocation
1. Go to the main interface resource pool configuration
2. The default configuration set root.default root.users resource pools and 50% and accounted for 50% modification
3. Select pool_nigth resource pool, the resource pool root.users root.defaut and modifications proportion of 80% and 20%
4. Select pool_day resource pool, the resource pool root.users root.default and modifications accounted 80% and 20%
5. Modify the configuration set after the completion of each resource pool configuration, click "Refresh Dynamic Resource Pools
Each of the above steps A Configuration complete resource pool.
4. Verify configuration set to take effect
The configuration rule set resource pool configuration, the current time is 10 pm, this time should be used to set resource pool configuration pool_nigth.
1. Log Yarn 8088 interface to view the accounting resource pool
2. Click the Scheduler resource pool to see the proportion of cases
可以看到root.default的资源池是root.users资源池占比的4倍，与配置集的计划规则一致。
5.总结
1.通过CM的动态资源池配置，可以方便的创建多个配置集。
2.根据不同时段对资源池的使用进行重新分配，动态的调整不需要手动触发。