hadoop-2.7.3 + Centos 6.7 64-bit environment using some small things in the process of setting up a Hadoop environment.
(1) Start checking
[hadoop@xxx6 ~]$ hadoop checknative -a
16/03/21 23:12:46 WARN bzip2.Bzip2Factory: Failed to load/initialize native-bzip2 library system-native, will use pure-Java version
16/03/21 23:12:46 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
Native library checking:
hadoop: true /home/hadoop/hadoop-2.7.2/lib/native/libhadoop.so
zlib: true /lib64/libz .so.1
snappy: false
lz4: true revision:99
bzip2: false
openssl: false Cannot load libcrypto.so (libcrypto.so: Cannot open shared object file: No such file or directory)!
16/03/21 23:12 :46 INFO util.ExitUtil: Exiting with status 1
(2) Problem finding
https://issues.apache.org/jira/browse/HADOOP-12845
After searching through Google, I found out that this is a BUG in the hadoop-2.7.2 version, which still needs to be solved. At present, I can only avoid the problem.
On RedHat system (Fedora/Centos/...) the /usr/lib64/libcrypto.so is a link provided by
openssl-devel RPM package which is fine on a build/development host, but devel packages are
not supposed to be installed on Production servers (Hadoop Cluster) and the openssl RPM package
don't include that link which is a problem.
(3) Problem solving
You can create the following symbolic link:
[hadoop@xxx6 ~]$ cd /usr/lib64/
[hadoop@xxx lib64]$ ln -s libcrypto.so.1.0.1e libcrypto.so
After establishing the symbolic link and then checking hadoop native, the phenomenon has changed!
[hadoop@xxx6 ~]$ hadoop checknative -a
16/03/21 23:13:51 WARN bzip2.Bzip2Factory: Failed to load/initialize native-bzip2 library system-native, will use pure-Java version
16/03/21 23:13:51 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
Native library checking:
hadoop: true /home/hadoop/hadoop-2.7.2/lib/native/libhadoop.so
zlib: true /lib64/libz.so.1
snappy: false
lz4: true revision:99
bzip2: false
openssl: true /usr/lib64/libcrypto.so
16/03/21 23:13:51 INFO util.ExitUtil: Exiting with status 1