本文是我学习Hadoop搭建过程中的各种笔记，内容来自于各种公开的教程，起点非常低，从Linux基础开始，直至在PC上搭建Hadoop成功，是真正的从零开始。
感谢过程中帮助我的各位认识的和不认识的老师。
40、Ssh免登陆配置：
上面集群搭建好，也已经测试成功，但是在启停集群服务时需要输入密码，如果是几十台的就要一直输入密码，密码多次输入会导致服务不能正常启动或停止
借助ssh的免登陆：ssh：安全的一个shell，用它进行免登陆，在此启停服务，操作此台机子，跨机子操作另一台服务器都不用密码了
配置ssh: ssh-keygen –t rsa
[root@hadoop01 hadoop-2.7.1]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): （按回车）
Enter passphrase (empty for no passphrase): （按回车）
Enter same passphrase again: （按回车）
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is: （按回车）
ac:b6:58:3c:95:ec:4c:42:3a:6d:16:90:dd:b2:11:0a root@hadoop01
The key's randomart image is: （按回车）
+--[ RSA 2048]----+
| E .o.o |
| ..o+ . |
| . o+ |
| +.+ . |
| o = S |
| = B |
| * o |
| + o |
| . . |
+-----------------+
rsa指加密的类型。 按回车、回车、回车，生成一种图时可以了
此时多了两个目录id_rsa和id_rsa.pub： ll ~/ .ssh/
相当于生成公钥，私钥的对id_rsa和id_rsa.pub known-hosts：记载着访问过这台主机的一些主机名】
[root@hadoop01 hadoop-2.7.1]# ll ~/.ssh/
total 12
-rw-------. 1 root root 1675 Apr 26 10:24 id_rsa
-rw-r--r--. 1 root root 395 Apr 26 10:24 id_rsa.pub
-rw-r--r--. 1 root root 1218 Apr 26 08:41 known_hosts
在hadoop02上没有任何文件的：
[root@hadoop02 local]# ll ~/.ssh/
total 0
设置免密码.ssh的三种方式：
选用第三种方式：
拷贝前，此时需要输密码 root： ssh hadoop01
[root@hadoop01 hadoop-2.7.1]# ssh hadoop01
root@hadoop01's password:
Last login: Thu Apr 26 10:37:47 2018 from hadoop01
退出去： exit
[root@hadoop01 ~]# exit
logout
Connection to hadoop01 closed.
把公钥拷贝到当前的机子上：ssh-copy-id hadoop01需要输密码
[root@hadoop01 hadoop-2.7.1]# ssh-copy-id hadoop01
root@hadoop01's password:
Now try logging into the machine, with "ssh 'hadoop01'", and check in:
.ssh/authorized_keys
to make sure we haven't added extra keys that you weren't expecting.
看有没有这个文件 ll ~/ .ssh/
[root@hadoop01 ~]# ll ~/.ssh/
total 16
-rw-------. 1 root root 395 Apr 26 10:41 authorized_keys
-rw-------. 1 root root 1675 Apr 26 10:24 id_rsa
-rw-r--r--. 1 root root 395 Apr 26 10:24 id_rsa.pub
-rw-r--r--. 1 root root 1218 Apr 26 08:41 known_hosts
刚才执行的命令，产生了authorized_keys这个文件，相当于把公钥复制的这个authorized_keys钥匙的文件里面
再次访问hadoop01，此时就不需要密码了 ssh hadoop01
[root@hadoop01 ~]# ssh hadoop01
Last login: Thu Apr 26 10:59:58 2018 from hadoop01
-bash: my: command not found 问
在hadoop01远程登录hadoop02： ssh hadoop02
此时不能过来，需要输入密码
[root@hadoop01 ~]# ssh hadoop02
root@hadoop02's password:
Last login: Thu Apr 26 09:13:22 2018 from 192.168.216.1
-bash: my: command not found
退出去：exit
[root@hadoop02 ~]# exit
logout
Connection to hadoop02 closed.
在hadoop02上查看： ll ~/ .ssh/
[root@hadoop02 local]# ll ~/.ssh/
total 0
在hadoop01执行：ssh-copy-id hadoop02
[root@hadoop01 ~]# ssh-copy-id hadoop02
root@hadoop02's password:
Now try logging into the machine, with "ssh 'hadoop02'", and check in:
.ssh/authorized_keys
to make sure we haven't added extra keys that you weren't expecting.
在hadoop02上查看：ll ~/ .ssh/
[root@hadoop02 local]# ll ~/.ssh/
total 4
-rw-------. 1 root root 395 Apr 26 11:10 authorized_keys
此时就多了authorized_keys文件
在hadoop01上，此时就不需要密码了，证明ssh做好 ssh hadoop02
[root@hadoop01 ~]# ssh hadoop02
Last login: Thu Apr 26 11:03:33 2018 from hadoop01
-bash: my: command not found
退出去exit
同样的方法:
ssh hadoop03 输密码
exit 退出
ssh-copy-id hadoop03 输密码
ssh hadoop03 此时就到hadoop03的机子上了
exit 退出
查看hadoop01、02、03的jps，在hadoop01上操作停止，此时hadoop01,02,03都在停止，并且因为设置了ssh，所以不需要输入密码 ： stop-all . sh
[root@hadoop01 ~]# stop-all.sh
This script is Deprecated. Instead use stop-dfs.sh and stop-yarn.sh
18/04/26 11:22:08 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Stopping namenodes on [hadoop01]
hadoop01: stopping namenode
hadoop02: stopping datanode
hadoop03: stopping datanode
hadoop01: stopping datanode
Stopping secondary namenodes [hadoop01]
hadoop01: stopping secondarynamenode
18/04/26 11:22:29 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
stopping yarn daemons
stopping resourcemanager
hadoop02: no nodemanager to stop
hadoop03: no nodemanager to stop
hadoop01: stopping nodemanager
hadoop01: nodemanager did not stop gracefully after 5 seconds: killing with kill -9
no proxyserver to stop
启动： start-all . sh
[root@hadoop01 ~]# start-all.sh
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
18/04/26 11:24:46 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [hadoop01]
hadoop01: starting namenode, logging to /usr/local/hadoop-2.7.1/logs/hadoop-root-namenode-hadoop01.out
hadoop02: starting datanode, logging to /usr/local/hadoop-2.7.1/logs/hadoop-root-datanode-hadoop02.out
hadoop03: starting datanode, logging to /usr/local/hadoop-2.7.1/logs/hadoop-root-datanode-hadoop03.out
hadoop01: starting datanode, logging to /usr/local/hadoop-2.7.1/logs/hadoop-root-datanode-hadoop01.out
Starting secondary namenodes [hadoop01]
hadoop01: starting secondarynamenode, logging to /usr/local/hadoop-2.7.1/logs/hadoop-root-secondarynamenode-hadoop01.out
18/04/26 11:25:07 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
starting yarn daemons
starting resourcemanager, logging to /usr/local/hadoop-2.7.1/logs/yarn-root-resourcemanager-hadoop01.out
hadoop03: starting nodemanager, logging to /usr/local/hadoop-2.7.1/logs/yarn-root-nodemanager-hadoop03.out
hadoop02: starting nodemanager, logging to /usr/local/hadoop-2.7.1/logs/yarn-root-nodemanager-hadoop02.out
hadoop01: starting nodemanager, logging to /usr/local/hadoop-2.7.1/logs/yarn-root-nodemanager-hadoop01.out
查看hadoop01、02、03服务是否正常 ： jps
hadoop01正常：
[root@hadoop01 ~]# jps
16608 NameNode
16737 DataNode
17154 NodeManager
17477 Jps
16872 SecondaryNameNode
17050 ResourceManager
hadoop02正常：
[root@hadoop02 local]# jps
14014 NodeManager
14143 Jps
13919 DataNode
hadoop03正常：
[root@hadoop03 local]# jps
13971 NodeManager
13876 DataNode
14100 Jps
41、hdfs的常用shell：
什么把集群已经搭建好了，下面怎么用它
shell相当于hdfs shell的一个脚本，相当于在xshell里面，用shell的方式操做文件系统。
FS Shell
调用文件系统（FS）Shell命令应使用 bin/hadoop fs<args>的形式。使所有的FS shell命令使用URI路径作为参数。URI格式是 scheme://authority/path。对HDFS文件系统，scheme是hdfs，对本地文件系统，scheme是file。其中scheme和authority参数都是可选的，如果未加指定，就会使用配置中指定的默认scheme。一个HDFS文件或目录，比如 /parent/child 可以表示成 hdfs:namenode:namenodeport/parent/child，或者/parent/child （假设你配置文件中的默认值是 namenode:namenodeport）。大多数FS Shell命令的行为和对应的Unix Shell 命令类似，不同之处会在下面介绍各命令使用详情时指出。
出错信息时会输出到stderr，其他信息输出到stdout
1、hdfs 的 shell
hdfs dfs –ls /
【/ 查看目录 ； -……，-……是怎么知道的】
hdfs dfs – 出来的是-后面可以跟的
[root@hadoop01 ~]# hdfs dfs -
Usage: hadoop fs [generic options]
[-appendToFile <localsrc> ... <dst>]
appendToFile：追加文件到某个目标地去 ；localsrc：本地文件（Linux文件系统）; dst：hdfs文件系统
[-cat [-ignoreCrc] <src> ...] cat：读取文件内容
[-checksum <src> ...] checksum：检验校验核
[-chgrp [-R] GROUP PATH...]
chgrp：改变文件或目录的组；-R：递归改；GROUP：组名；PATH：路径
[-chmod [-R] <MODE[,MODE]... | OCTALMODE> PATH...]
chmod：改权限，同Linux系统
[-chown [-R] [OWNER][:[GROUP]] PATH...]
chown：改拥有者，也可以加上组，同Linux
[-copyFromLocal [-f] [-p] [-l] <localsrc> ... <dst>]
copyFromlocal：把本地的文件copy到hdfs文件系统
[-copyToLocal [-p] [-ignoreCrc] [-crc] <src> ... <localdst>]
copyToLocal：把hdfs文件系统copy到本地
[-count [-q] [-h] <path> ...]
[-cp [-f] [-p | -p[topax]] <src> ... <dst>]
[-createSnapshot <snapshotDir> [<snapshotName>]]
[-deleteSnapshot <snapshotDir> <snapshotName>]
[-df [-h] [<path> ...]] 查看文件大小
[-du [-s] [-h] <path> ...]
[-expunge]
[-find <path> ... <expression> ...]
[-get [-p] [-ignoreCrc] [-crc] <src> ... <localdst>] 下载
[-getfacl [-R] <path>]
[-getfattr [-R] {-n name | -d} [-e en] <path>]
[-getmerge [-nl] <src> <localdst>]
[-help [cmd ...]]
[-ls [-d] [-h] [-R] [<path> ...]]
[-mkdir [-p] <path> ...]
[-moveFromLocal <localsrc> ... <dst>]
[-moveToLocal <src> <localdst>]
[-mv <src> ... <dst>]
[-put [-f] [-p] [-l] <localsrc> ... <dst>]
[-renameSnapshot <snapshotDir> <oldName> <newName>]
[-rm [-f] [-r|-R] [-skipTrash] <src> ...]
[-rmdir [--ignore-fail-on-non-empty] <dir> ...]
[-setfacl [-R] [{-b|-k} {-m|-x <acl_spec>} <path>]|[--set <acl_spec> <path>]]
[-setfattr {-n name [-v value] | -x name} <path>]
[-setrep [-R] [-w] <rep> <path> ...]
[-stat [format] <path> ...]
[-tail [-f] <file>]
[-test -[defsz] <path>]
[-text [-ignoreCrc] <src> ...]
[-touchz <path> ...]
[-truncate [-w] <length> <path> ...]
[-usage [cmd ...]]
通常有两种方式：
hdfs dfs -（东西）/（+路径）
hadoop fs -（东西）/（+路径）
查看目录，查看文件系统有什么：hdfs dfs –ls / 或 hadoop fs –ls /
[root@hadoop01 ~]# hdfs dfs -ls /
18/04/26 11:46:21 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 3 items
-rw-r--r-- 3 root supergroup 1366 2018-04-26 09:30 /README.txt
drwxr-xr-x - root supergroup 0 2018-04-26 09:58 /out
drwx------ - root supergroup 0 2018-04-26 09:57 /tmp
[root@hadoop01 ~]# hadoop fs -ls /
18/04/26 11:47:41 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 3 items
-rw-r--r-- 3 root supergroup 1366 2018-04-26 09:30 /README.txt
drwxr-xr-x - root supergroup 0 2018-04-26 09:58 /out
drwx------ - root supergroup 0 2018-04-26 09:57 /tmp
递归查看：hdfs dfs –lsr /
[root@hadoop01 ~]# hdfs dfs -lsr /
lsr: DEPRECATED: Please use 'ls -R' instead.
18/04/26 11:50:46 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
-rw-r--r-- 3 root supergroup 1366 2018-04-26 09:30 /README.txt
drwxr-xr-x - root supergroup 0 2018-04-26 09:58 /out
drwxr-xr-x - root supergroup 0 2018-04-26 09:58 /out/00
-rw-r--r-- 3 root supergroup 0 2018-04-26 09:58 /out/00/_SUCCESS
-rw-r--r-- 3 root supergroup 1306 2018-04-26 09:58 /out/00/part-r-00000
drwx------ - root supergroup 0 2018-04-26 09:57 /tmp
drwx------ - root supergroup 0 2018-04-26 09:57 /tmp/hadoop-yarn
drwx------ - root supergroup 0 2018-04-26 09:58 /tmp/hadoop-yarn/staging
drwxr-xr-x - root supergroup 0 2018-04-26 09:58 /tmp/hadoop-yarn/staging/history
drwxrwxrwt - root supergroup 0 2018-04-26 09:58 /tmp/hadoop-yarn/staging/history/done_intermediate
drwxrwx--- - root supergroup 0 2018-04-26 09:58 /tmp/hadoop-yarn/staging/history/done_intermediate/root
-rwxrwx--- 3 root supergroup 33646 2018-04-26 09:58 /tmp/hadoop-yarn/staging/history/done_intermediate/root/job_1524706682045_0001-1524707867807-root-word+count-1524707938102-1-1-SUCCEEDED-default-1524707894739.jhist
-rwxrwx--- 3 root supergroup 349 2018-04-26 09:58 /tmp/hadoop-yarn/staging/history/done_intermediate/root/job_1524706682045_0001.summary
-rwxrwx--- 3 root supergroup 113579 2018-04-26 09:58 /tmp/hadoop-yarn/staging/history/done_intermediate/root/job_1524706682045_0001_conf.xml
drwx------ - root supergroup 0 2018-04-26 09:57 /tmp/hadoop-yarn/staging/root
drwx------ - root supergroup 0 2018-04-26 09:58 /tmp/hadoop-yarn/staging/root/.staging
创建test目录：hdfs dfs –makdir /test （一定要跟 根/）
[root@hadoop01 ~]# hdfs dfs -mkdir /test
18/04/26 11:54:46 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
查看创建的test目录是否储存：hdfs dfs -ls
[root@hadoop01 ~]# hdfs dfs -ls /
18/04/26 11:55:07 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 4 items
-rw-r--r-- 3 root supergroup 1366 2018-04-26 09:30 /README.txt
drwxr-xr-x - root supergroup 0 2018-04-26 09:58 /out
drwxr-xr-x - root supergroup 0 2018-04-26 11:54 /test
drwx------ - root supergroup 0 2018-04-26 09:57 /tmp
在test目录下，递归创建文件01,02，加-p：hdfs dfs –mkdir –p /test/01/02
[root@hadoop01 ~]# hdfs dfs -mkdir -p /test/01/02
18/04/26 12:00:57 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
查看在test下面创建的01 02：
[root@hadoop01 ~]# hdfs dfs -lsr /test/
lsr: DEPRECATED: Please use 'ls -R' instead.
18/04/26 12:01:30 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
drwxr-xr-x - root supergroup 0 2018-04-26 12:00 /test/01
drwxr-xr-x - root supergroup 0 2018-04-26 12:00 /test/01/02
在test目录下，创建一个空的：-touchz
[root@hadoop01 ~]# hdfs dfs -touchz /test/te.txt
18/04/26 12:05:42 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
查看：
[root@hadoop01 ~]# hdfs dfs -lsr /test
lsr: DEPRECATED: Please use 'ls -R' instead.
18/04/26 12:05:57 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
drwxr-xr-x - root supergroup 0 2018-04-26 12:00 /test/01
drwxr-xr-x - root supergroup 0 2018-04-26 12:00 /test/01/02
-rw-r--r-- 3 root supergroup 0 2018-04-26 12:05 /test/te.txt
复制：-cp 将test下的te.txt复制到01：（加空格复制多个）
[root@hadoop01 ~]# hdfs dfs -cp /test/te.txt /test/01
18/04/26 12:08:56 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
18/04/26 12:08:57 WARN hdfs.DFSClient: DFSInputStream has been closed already
查看 复制是否到01下：
[root@hadoop01 ~]# hdfs dfs -ls /test/01
18/04/26 12:09:36 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 2 items
drwxr-xr-x - root supergroup 0 2018-04-26 12:00 /test/01/01
-rw-r--r-- 3 root supergroup 0 2018-04-26 12:08 /test/01/te.txt
查看 test是否还有te.txt：
[root@hadoop01 ~]# hdfs dfs -lsr /test
lsr: DEPRECATED: Please use 'ls -R' instead.
18/04/26 12:14:42 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
drwxr-xr-x - root supergroup 0 2018-04-26 12:08 /test/01
drwxr-xr-x - root supergroup 0 2018-04-26 12:00 /test/01/01
-rw-r--r-- 3 root supergroup 0 2018-04-26 12:08 /test/01/te.txt
-rw-r--r-- 3 root supergroup 0 2018-04-26 12:05 /test/te.txt
移动：将test下01下的te.txt，移动到test下01下的02下面
[root@hadoop01 ~]# hdfs dfs -mv /test/01/te.txt /test/01/02
18/04/26 15:00:47 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
查看：
[root@hadoop01 ~]# hdfs dfs -lsr /test
lsr: DEPRECATED: Please use 'ls -R' instead.
18/04/26 15:01:17 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
drwxr-xr-x - root supergroup 0 2018-04-26 14:34 /test/01
-rw-r--r-- 3 root supergroup 0 2018-04-26 12:08 /test/01/02
-rw-r--r-- 3 root supergroup 0 2018-04-26 14:34 /test/01/02/te.txt
-rw-r--r-- 3 root supergroup 0 2018-04-26 12:05 /test/te.txt
移动并且改名字：将test下01下的02下的te.txt，移动到test下01，改名字为te
[root@hadoop01 ~]# hdfs dfs -mv /test/01/02/te.txt /test/01/te
18/04/26 15:36:43 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[root@hadoop01 ~]# hdfs dfs -lsr /test
lsr: DEPRECATED: Please use 'ls -R' instead.
18/04/26 15:37:18 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
drwxr-xr-x - root supergroup 0 2018-04-26 14:34 /test/01
-rw-r--r-- 3 root supergroup 0 2018-04-26 12:08 /test/01/02
-rw-r--r-- 3 root supergroup 0 2018-04-26 14:34 /test/01/te
-rw-r--r-- 3 root supergroup 0 2018-04-26 12:05 /test/te.txt
上传：-put 将当前目录的if.sh 上传到/test根目录下的test：
[root@hadoop01 ~]# cd /home/
[root@hadoop01 home]# cd ./shell/
[root@hadoop01 shell]# ll
total 60
……………………
-rwxr--r--. 1 root root 267 Apr 24 15:01 if.sh
……………………
[root@hadoop01 shell]# hdfs dfs -put ./if.sh /test
18/04/26 15:42:01 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[root@hadoop01 shell]# hdfs dfs -ls /test
18/04/26 15:42:34 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 3 items
drwxr-xr-x - root supergroup 0 2018-04-26 14:34 /test/01
-rw-r--r-- 3 root supergroup 267 2018-04-26 15:42 /test/if.sh
-rw-r--r-- 3 root supergroup 0 2018-04-26 12:05 /test/te.txt
移动上传：-copyFromLocal 移动当前目录下的for.sh 到 hdfs文件系统的test目录下：
[root@hadoop01 shell]# hdfs dfs -copyFromLocal ./for.sh /test
18/04/26 15:44:44 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[root@hadoop01 shell]# hdfs dfs -ls /test
18/04/26 15:44:56 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 4 items
drwxr-xr-x - root supergroup 0 2018-04-26 14:34 /test/01
-rw-r--r-- 3 root supergroup 71 2018-04-26 15:44 /test/for.sh
-rw-r--r-- 3 root supergroup 267 2018-04-26 15:42 /test/if.sh
-rw-r--r-- 3 root supergroup 0 2018-04-26 12:05 /test/te.txt
下载：-get 将hdfs文件系统的test下的if.sh，下载到home下，改名为并if：
[root@hadoop01 shell]# hdfs dfs -get /test/if.sh /home/if
18/04/26 15:46:41 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
18/04/26 15:46:42 WARN hdfs.DFSClient: DFSInputStream has been closed already
[root@hadoop01 shell]# ll /home/
total 387576
………………
-rw-r--r--. 1 root root 267 Apr 26 15:46 if
………………
下载：-copyToLocal 将hdfs文件系统的test下的for.sh，下载到home目录下：
[root@hadoop01 shell]# hdfs dfs -copyToLocal /test/for.sh /home/
18/04/26 15:49:17 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
18/04/26 15:49:18 WARN hdfs.DFSClient: DFSInputStream has been closed already
[root@hadoop01 shell]# ll /home/
total 387580
………………
-rw-r--r--. 1 root root 71 Apr 26 15:49 for.sh
………………
查看文件（有多个）查看hdfs正真的文件内容： -text、-tail、-cat
[root@hadoop01 shell]# hdfs dfs -text /test/if.sh
18/04/26 15:51:54 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[root@hadoop01 shell]# hdfs dfs -tail /test/if.sh
18/04/26 15:53:27 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[root@hadoop01 shell]# hdfs dfs -cat /test/if.sh
18/04/26 15:55:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
都是下面这个结果：
#!/bin/bash
a=10
b=20
c=15
if [ $a -gt $b ]
then
echo "$a > $b"
fi
if [ $a -lt $b ]
then
echo "$a < $b"
else
echo "$a is not sammer $b"
fi
if [ $a -lt $b ]
then
echo "$a < $b"
elif [ $a -gt $b -a $b -lt $c ]
then
echo "$a > $b and $b < $c"
else
echo "$a < $b"
fi
查看每个目录的大小：-du
[root@hadoop01 shell]# hdfs dfs -du /test
18/04/26 15:58:36 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
0 /test/01
71 /test/for.sh
267 /test/if.sh
0 /test/te.txt
查看每个目录大小的总和：-du -s
[root@hadoop01 shell]# hdfs dfs -du -s /test
18/04/26 15:59:16 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
338 /test
删除：hdfs dfs –rmr [-R] /
删除前：
[root@hadoop01 shell]# hdfs dfs -ls /
18/04/27 09:23:10 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 4 items
-rw-r--r-- 3 root supergroup 1366 2018-04-26 09:30 /README.txt
drwxr-xr-x - root supergroup 0 2018-04-26 09:58 /out
drwxr-xr-x - root supergroup 0 2018-04-26 15:44 /test
drwx------ - root supergroup 0 2018-04-26 09:57 /tmp
删除：test
[root@hadoop01 shell]# hdfs dfs -rm -R /test
18/04/27 09:23:52 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
18/04/27 09:23:53 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval = 0 minutes, Emptier interval = 0 minutes.
Deleted /test
删除后查看：
[root@hadoop01 shell]# hdfs dfs -ls /
18/04/27 09:24:05 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 3 items
-rw-r--r-- 3 root supergroup 1366 2018-04-26 09:30 /README.txt
drwxr-xr-x - root supergroup 0 2018-04-26 09:58 /out
drwx------ - root supergroup 0 2018-04-26 09:57 /tmp