CDH集群日志垃圾清理记录

CDH

• Activity Monitor、Service Monitor、Host Monitor

for i in `ls /var/log/cloudera-scm-firehose/*.log.out`;do cat /dev/null > $i;done

• Alert Publisher

for i in `ls /var/log/cloudera-scm-alertpublisher/*.log.out`;do cat /dev/null > $i;done

• Event Server

for i in `ls /var/log/cloudera-scm-eventserver/*.log.out`;do cat /dev/null > $i;done

HDFS

• NameNode、SecondaryNameNode、DataNode

for i in `ls /var/log/hadoop-hdfs/*.log.out`;do cat /dev/null > $i;done
for i in `ls /var/log/hadoop-hdfs/*.log`;do cat /dev/null > $i;done

Hive

• HiveServer2、Hive Metastore Server

for i in `ls /var/log/hive/*.log.out`;do cat /dev/null > $i;done

Hue

• Hue Server

for i in `ls /var/log/hue/*.log`;do cat /dev/null > $i;done

• Load Balancer

for i in `ls /var/log/hue-httpd/*_log`;do cat /dev/null > $i;done

impala

• Impala Catalog Server

for i in `ls /var/log/catalogd/catalogd.*`;do cat /dev/null > $i;done

• Impala StateStore

for i in `ls /var/log/statestore/statestored.*`;do cat /dev/null > $i;done

• Impala Daemon

for i in `ls /var/log/impalad/impalad.*`;do cat /dev/null > $i;done

Kudu

• Master

for i in `ls /var/log/kudu/kudu-master.*`;do cat /dev/null > $i;done

• Tablet Server

for i in `ls /var/log/kudu/kudu-tserver.*`;do cat /dev/null > $i;done

Yarn

• ResourceManager、NodeManager

for i in `ls /var/log/hadoop-yarn/*.log.out`;do cat /dev/null > $i;done

• JobHistory Server

for i in `ls /var/log/hadoop-mapreduce/*.log.out`;do cat /dev/null > $i;done

ZooKeeper

• Server

for i in `ls /var/log/zookeeper/*.log`;do cat /dev/null > $i;done

其他

cdh

rm -rf /var/lib/cloudera-host-monitor/ts/*/partition*/*
rm -rf /var/lib/cloudera-service-monitor/ts/*/partition*/*

yarn cache

• yarn.nodemanager.localizer.cache.cleanup.interval-ms：这是缓存清理间隔时间。默认值为 600000 毫秒。此间隔时间后，如果缓存大小超出 yarn.nodemanager.localizer.cache.target-size-mb 中设置的值，则将删除未被运行容器使用的文件。
• yarn.nodemanager.localizer.cache.target-size-mb：这是缓存允许的最大磁盘空间。默认值为 10240 MB。当缓存磁盘大小超出此值时，将按照 yarn.nodemanager.localizer.cache.cleanup.interval-ms 中设置的时间间隔删除未被运行容器使用的文件。

yarn.nodemanager.localizer.cache.cleanup.interval-ms 400000
yarn.nodemanager.localizer.cache.target-size-mb 5120