1. Preface
Every time building a cluster environment is like a battle. Any error in the details of the battle will lead to serious consequences. Therefore, the configurations that need to be done during the construction are system configuration, network configuration (remember to turn off the firewall), user permissions, files. Permissions, configuration files, etc., must be very careful to kill mistakes in the cradle, avoid a minute saved due to carelessness or want to increase the speed, and spend a few hours of debugging time later, because the small loses the big It's not worth it, just remember to check, check and check.
Today, let’s talk about a common problem encountered when building a ZooKeeper cluster, Connection refused, which is also a problem that many friends on the Internet also have. The performance of the problem may be the same, but the causes of the problem are different, so we should pay attention to it. If you report errors in your own log file, you should concentrate on solving the errors in the log after confirming that there is no external environment.
2. My question
The following is the error when I set up the cluster:
This is an ostensible error
2016-12-08 05:11:07,890 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0 :0:0:0:0:0:2181:QuorumCnxManager@382] - Cannot open channel to 5 at election address lzx15/192.168.80.175:3888
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:368)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:402)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader (FastLeaderElection.java:840)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:762)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:368)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:402)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader (FastLeaderElection.java:840)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:762)
This is the actual error
java.io.FileNotFoundException: /home/hadoop/data/zookeeper/zkdata/version-2/acceptedEpoch.tmp(Permissoin denied)
3. Someone else's solution:
Mainly for the following two points:
1 ) The firewall is not closed, which leads to the failure of communication and cannot connect to other nodes. I have checked and confirmed that the firewall is all closed, so this method is invalid for my problem.
Command line: chkconfig iptables off (turn off firewall)
service iptables status (view firewall status)
2 ) Configuration file problem
a. /zookeeper/conf/zoo.cfg file configuration problem
For example , the format of dataDir is wrong, or the file specified by dataDir is not created , etc., so check your configuration file carefully. As for the content of this file, please Baidu by yourself.
b. Create a myid file in the directory specified by dataDir, and then write the corresponding server number of the node in zoo.cfg to the file, such as server.1 , then write 1 in myid .
4. My solution to findzookeeper'slogfilezookeeper.outwhere? The normal path is your zookeper installation directory /zookeeper.out . I can only find the superficial errorin thiszookeeper.outConnection refused, so I have not been able to find the problem, and finally I found it by accident
[hadoop@Test hadoop]$ cd /home/hadoop [hadoop@Test ~]$ ls app data tools zookeeper.out
I found the real error of my problem in this zookeeper.out , /home/hadoop/data/zookeeper/zkdata/version-2/acceptedEpoch.tmp(Permissoin denied) , so I went to check the permissions of the verison-2 folder, I found that he was actually the root user, I immediately transferred it to the hadoop user, and then restarted the cluster, the three nodes started successfully.
5. Summary
Many times we will find solutions to the surface problems we encounter, but many problems are only caused by problems in other places, so when you think that you have found a solution to the current problem, you still cannot solve the problem, Think about making a mistake somewhere ahead, look elsewhere for the answer, and remember that the key is usually not in the lock.