解决hdfs命令报错:WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... usi

今天在集群上执行hdfs命令时,总是出现这样的报错信息

19/04/06 22:03:55 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

更改日志级别

export HADOOP_ROOT_LOGGER=DEBUG,console

再次执行hdfs命令,日志信息如下:

[root@Linux004 mapreduce]# hadoop fs -ls /
19/04/06 22:20:04 DEBUG util.Shell: setsid exited with exit code 0
19/04/06 22:20:05 DEBUG conf.Configuration: parsing URL jar:file:/opt/apps/hadoop-2.6.5/share/hadoop/common/hadoop-common-2.6.5.jar!/core-default.xml
19/04/06 22:20:05 DEBUG conf.Configuration: parsing input stream sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@3b58fa97
19/04/06 22:20:05 DEBUG conf.Configuration: parsing URL file:/opt/apps/hadoop-2.6.5/etc/hadoop/core-site.xml
19/04/06 22:20:05 DEBUG conf.Configuration: parsing input stream java.io.BufferedInputStream@6f8cc276
19/04/06 22:20:05 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with annotation @org.apache.hadoop.metrics2.annotation.Metric(value=[Rate of successful kerberos logins and latency (milliseconds)], about=, valueName=Time, type=DEFAULT, always=false, sampleName=Ops)
19/04/06 22:20:05 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with annotation @org.apache.hadoop.metrics2.annotation.Metric(value=[Rate of failed kerberos logins and latency (milliseconds)], about=, valueName=Time, type=DEFAULT, always=false, sampleName=Ops)
19/04/06 22:20:05 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with annotation @org.apache.hadoop.metrics2.annotation.Metric(value=[GetGroups], about=, valueName=Time, type=DEFAULT, always=false, sampleName=Ops)
19/04/06 22:20:05 DEBUG impl.MetricsSystemImpl: UgiMetrics, User and group related metrics
19/04/06 22:20:06 DEBUG security.Groups:  Creating new Groups object
19/04/06 22:20:06 DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library...
19/04/06 22:20:06 DEBUG util.NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: /opt/apps/hadoop-2.6.5/lib/native/libhadoop.so.1.0.0: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /opt/apps/hadoop-2.6.5/lib/native/libhadoop.so.1.0.0)
19/04/06 22:20:06 DEBUG util.NativeCodeLoader: java.library.path=/opt/apps/hadoop-2.6.5/lib/native
19/04/06 22:20:06 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
19/04/06 22:20:06 DEBUG util.PerformanceAdvisory: Falling back to shell based
19/04/06 22:20:06 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
19/04/06 22:20:06 DEBUG security.Groups: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000; warningDeltaMs=5000
19/04/06 22:20:06 DEBUG security.UserGroupInformation: hadoop login
19/04/06 22:20:06 DEBUG security.UserGroupInformation: hadoop login commit
19/04/06 22:20:06 DEBUG security.UserGroupInformation: using local user:UnixPrincipal: root
19/04/06 22:20:06 DEBUG security.UserGroupInformation: Using user: "UnixPrincipal: root" with name root
19/04/06 22:20:06 DEBUG security.UserGroupInformation: User entry: "root"
19/04/06 22:20:06 DEBUG security.UserGroupInformation: UGI loginUser:root (auth:SIMPLE)
19/04/06 22:20:06 DEBUG hdfs.BlockReaderLocal: dfs.client.use.legacy.blockreader.local = false
19/04/06 22:20:06 DEBUG hdfs.BlockReaderLocal: dfs.client.read.shortcircuit = false
19/04/06 22:20:06 DEBUG hdfs.BlockReaderLocal: dfs.client.domain.socket.data.traffic = false
19/04/06 22:20:06 DEBUG hdfs.BlockReaderLocal: dfs.domain.socket.path = 
19/04/06 22:20:06 DEBUG hdfs.DFSClient: No KeyProvider found.
19/04/06 22:20:06 DEBUG retry.RetryUtils: multipleLinearRandomRetry = null
19/04/06 22:20:06 DEBUG ipc.Server: rpcKind=RPC_PROTOCOL_BUFFER, rpcRequestWrapperClass=class org.apache.hadoop.ipc.ProtobufRpcEngine$RpcRequestWrapper, rpcInvoker=org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker@6cd827df
19/04/06 22:20:06 DEBUG ipc.Client: getting client out of cache: org.apache.hadoop.ipc.Client@6e6fdd08
19/04/06 22:20:07 DEBUG util.PerformanceAdvisory: Both short-circuit local reads and UNIX domain socket are disabled.
19/04/06 22:20:07 DEBUG sasl.DataTransferSaslUtil: DataTransferProtocol not using SaslPropertiesResolver, no QOP found in configuration for dfs.data.transfer.protection
19/04/06 22:20:07 DEBUG ipc.Client: The ping interval is 60000 ms.
19/04/06 22:20:07 DEBUG ipc.Client: Connecting to Linux004/192.168.149.134:9000
19/04/06 22:20:07 DEBUG ipc.Client: IPC Client (1066456457) connection to Linux004/192.168.149.134:9000 from root: starting, having connections 1
19/04/06 22:20:07 DEBUG ipc.Client: IPC Client (1066456457) connection to Linux004/192.168.149.134:9000 from root sending #0
19/04/06 22:20:07 DEBUG ipc.Client: IPC Client (1066456457) connection to Linux004/192.168.149.134:9000 from root got value #0
19/04/06 22:20:07 DEBUG ipc.ProtobufRpcEngine: Call: getFileInfo took 75ms
19/04/06 22:20:07 DEBUG ipc.Client: IPC Client (1066456457) connection to Linux004/192.168.149.134:9000 from root sending #1
19/04/06 22:20:07 DEBUG ipc.Client: IPC Client (1066456457) connection to Linux004/192.168.149.134:9000 from root got value #1
19/04/06 22:20:07 DEBUG ipc.ProtobufRpcEngine: Call: getListing took 2ms
Found 3 items
drwxr-xr-x   - root supergroup          0 2019-04-06 22:05 /data
drwx------   - root supergroup          0 2019-04-06 22:17 /tmp
drwxr-xr-x   - root supergroup          0 2019-04-06 22:13 /wcdata
19/04/06 22:20:07 DEBUG ipc.Client: stopping client from cache: org.apache.hadoop.ipc.Client@6e6fdd08
19/04/06 22:20:07 DEBUG ipc.Client: removing client from cache: org.apache.hadoop.ipc.Client@6e6fdd08
19/04/06 22:20:07 DEBUG ipc.Client: stopping actual client because no more references remain: org.apache.hadoop.ipc.Client@6e6fdd08
19/04/06 22:20:07 DEBUG ipc.Client: Stopping client
19/04/06 22:20:07 DEBUG ipc.Client: IPC Client (1066456457) connection to Linux004/192.168.149.134:9000 from root: closed
19/04/06 22:20:07 DEBUG ipc.Client: IPC Client (1066456457) connection to Linux004/192.168.149.134:9000 from root: stopped, remaining connections 0

-------------------------------------------------------------------------------------------------------------------------

查看红色的部分,可知缺少了glibc-2.14包。接下来手动安装

安装glibc-2.14包

1、下载链接:http://ftp.gnu.org/gnu/glibc/glibc-2.14.tar.gz

2、tar -zxvf /opt/source/glibc-2.14.tar.gz -C ../apps/

3、cd glibc-2.14 && mkdir build && cd build 

4、../configure --prefix=/opt/apps/glibc-2.14

5、make -j4 

6、make install

7、cp /etc/ld.so.c* /opt/glibc-2.14/etc/

8、ln -sf /opt/glibc-2.14/lib/libc-2.14.so /lib64/libc.so.6

9、strings /lib64/libc.so.6 | grep GLIBC




#第四步时间很长,第9步验证是否已经支持glibc-2.14







 

猜你喜欢

转载自blog.csdn.net/pengzonglu7292/article/details/89055414
今日推荐