Environment Description: redhat7.3 CDH5.15.1 deployed using parcels
Error Description: airflow scheduler, recently two weeks occasional error, error types are Class 2: 1, can not initialize cluster configuration; 2, reads the configuration permissions issue
A error:
Launching Job 1 out of 1 Number of reduce tasks is set to 0 since there's no reduce operator java.io.IOException: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses. at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:143) at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:108) at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:101) at org.apache.hadoop.mapred.JobClient.init(JobClient.java:477) at org.apache.hadoop.mapred.JobClient.(JobClient.java:455)
Two error
19/12/21 04:06:27 ERROR conf.Configuration: error parsing conf core-site.xml java.io.FileNotFoundException: /etc/hive/conf.cloudera.hive/core-site.xml (Permission denied) at java.io.FileInputStream.open0(Native Method) at java.io.FileInputStream.open(FileInputStream.java:195) at java.io.FileInputStream.(FileInputStream.java:138) at java.io.FileInputStream.(FileInputStream.java:93) at sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90)
Being given three
setfacl: Permission denied. user=dip is not the owner of inode=.hive-staging_hive_2019-12-22_07-44-25_997_2557429548076828737-1 java.lang.RuntimeException: java.io.FileNotFoundException: /etc/hive/conf.cloudera.hive/hive-site.xml (Permission denied) at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2811) at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2663) at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2559) at org.apache.hadoop.conf.Configuration.get(Configuration.java:1340) orgapachehadoophiveconfHiveConfgetVar at (HiveConfjava: 2756 ) orgapachehadoophiveconfHiveConfgetVar at (HiveConfjava: 2777 ) at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:2849) at org.apache.hadoop.hive.conf.HiveConf.(HiveConf.java:2792)
All machines in the cluster to check and found a machine configuration file has been updated crazy:
/etc/hive/conf.cloudera.hive
/etc/hadoop/conf.cloudera.yarn
/etc/hbase/conf.cloudera.hbase
an examination:
The machine: / var / lib / alternatives available file following
operating:
Delete / var / lib / alternatives all of the following documents,
Restart agent
cd / var / lib / alternatives rm -rf * systemctl restart cloudera-scm-agent.service
Observation: The configuration file is no longer crazy update, the problem is solved.