table of Contents
One, the problem leads
In the process of enabling HDFS HA, the Secondary NameNone has not been deleted due to artificial or unexpected interruption. It took 15 minutes for the namenode to start and I forced it out. It’s really cheap. After checking the online articles and failing, you can retry.
Reference document: https://www.jianshu.com/p/bc2c6ebef51c
I had no choice but to think about rolling back the previous operation. At that time, there were only three JournalNodes installed, and there was one more NameNode without ZKFC. To be specific, I can use the following method to query.
1. Role View
#查看namenode主机信息
curl -u admin:admin -i http://namenode:8080/api/v1/clusters/sfyp/host_components?HostRoles/component_name=NAMENODE
#查看secondarynamenode主机信息
curl -u admin:admin -i http://namenode:8080/api/v1/clusters/sfyp/host_components?HostRoles/component_name=SECONDARY_NAMENODE
#查看journalnode主机信息
curl -u admin:admin -i http://namenode:8080/api/v1/clusters/sfyp/host_components?HostRoles/component_name=JOURNALNODE
#查看zkfc主机信息
curl -u admin:admin -i http://namenode:8080/api/v1/clusters/sfyp/host_components?HostRoles/component_name=ZKFCHDFS information view, then this is not useful
ZKFC Information View
SECONDARY_NAMENODE
Why look at these?
In fact, to prepare for the subsequent deletion, clearly install the above four components on that machine.
Second, the solution steps
What I did was to clean up the extra NameNode, because if you didn’t clean it up, you found that you want to choose HA on the original machine, that is, the node where you plan to install the snamenode can no longer be selected. I can’t clean up the subsequent JournalNode, either. I didn't care, I didn't install ZKFC either.
1. Clean up extra NAMENODE
#获取NAMENODE节点信息
curl -u admin:admin -H "X-Requested-By: ambari" -X GET http://namenode:8080/api/v1/clusters/sfyp/host_components?HostRoles/component_name=NAMENODE
#选择额外的namenode节点删除
curl -u admin:admin -H "X-Requested-By: ambari" -X DELETE http://namenode:8080/api/v1/clusters/sfyp/hosts/snamenode/host_components/NAMENODE2. Restore secondary_namenode
#step1
curl -u admin:admin -H "X-Requested-By: ambari" -X POST -d '{"host_components" : [{"HostRoles":{"component_name":"SECONDARY_NAMENODE"}}] }' http://namenode:8080/api/v1/clusters/sfyp/hosts?Hosts/host_name=snamenode
#step2 启用SECONDARY_NAMENODE
curl -u admin:admin -H "X-Requested-By: ambari" -X PUT -d '{"RequestInfo":{"context":"Enable Secondary NameNode"},"Body":{"HostRoles":{"state":"INSTALLED"}}}' http://namenode:8080/api/v1/clusters/sfyp/hosts/snamenode/host_components/SECONDARY_NAMENODE
#step3
curl -u admin:admin -H "X-Requested-By: ambari" -X GET "http://namenode:8080/api/v1/clusters/sfyp/host_components?HostRoles/component_name=SECONDARY_NAMENODE&fields=HostRoles/state"
secondary_namenode view
3. NameNodeHA starts
Because before half of the HA startup process is done, when I clean up the redundant namenode, I can continue to configure HA on the original node.
