[] HDFS HDFS basic operations

Basic operation
by

在/目录下创建一个test1文件夹
hadoop fs -mkdir /test1
在Hadoop中的test1文件夹中创建一个file.txt文件
hadoop fs -touchz /test1/file.txt

delete

删除Hadoop根下的test1目录
hadoop fs -rm -r /test1

change

将Hadoop根下test1目录中的file.txt文件,移动到根下并重命名为file2.txt
hadoop fs -mv /test1/file.txt /file2.txt

将Hadoop根下的file2.txt文件复制到test1目录下
hadoop fs -cp /file2.txt /test1

向data.txt文件写入hello hadoop!
echo hello hadoop! >> data.txt

将Linux本地/data目录下的data.txt文件,上传到HDFS中的/test1目录下
hadoop fs -put /data/data.txt /test1

将Hadoop中/test1目录下的data.txt文件,下载到Linux本地/apps目录中
hadoop fs -get /test1/data.txt /apps

使用chown方法,改变Hadoop中/test1目录中的data.txt文件拥有者为root,使用-R将使改变在目录结构下递归进行。
hadoop fs -chown root /test1/data.txt

使用chmod方法,赋予Hadoop中/test1目录中的data.txt文件777权限
hadoop fs -chmod 777 /test1/data.txt

text方法可以将源文件输出为文本格式。允许的格式是zip和TextRecordInputStream。
hadoop fs -text /test1/data.txt

stat方法可以返回指定路径的统计信息,有多个参数可选,当使用-stat选项但不指定format时候,只打印文件创建日期,相当于%y
hadoop fs -stat /test1/data.txt

The following lists the format of the form:

% B: print file size (directory 0)

% N: Print the file name

% O: print block size (we want value)

% R: print back up

% Y: Print UTC date yyyy-MM-dd HH: mm: ss

% Y: print UTC microseconds since January 1, 1970 of

% F: directory print directory, print file regular file
search

查看根目录下所有文件
hadoop fs -ls /

查看Hadoop中/test1目录下的data.txt文件
hadoop fs -cat /test1/data.txt

使用tail方法查看Hadoop中/test1目录下的data.txt文件
hadoop fs -tail /test1/data.txt

查看Hadoop中/test1目录下的data.txt文件大小
hadoop fs -du -s /test1/data.txt

-du behind can not add -s, direct write directory representation view all files in the directory size

查看一下/apps目录下是否存在data.txt文件
ls /apps

Trash

当在Hadoop中设置了回收站功能时,删除的文件会保留在回收站中,可以使用expunge方法清空回收站。
hadoop fs -expunge

Safe Mode

When distributed file system starts, there will be time to start in safe mode, when the case of a distributed file system in safe mode, the contents of the file system does not allow modification can not be deleted until the safe mode ends. Safety mode is used to check the validity of each data block DataNode on system startup, also copy or delete part of the data block is necessary according to the policy. Runtime can also enter Safe Mode command. In practice, when the system starts to modify and delete files there will be a security model does not allow modification of the error message, you can just wait for a while.

在HDFS的根下创建in目录,并将/data下的data.txt文件上传到HDFS中的in目录
hadoop fs -put /data/data.txt /in
查看HDFS中的/out目录
hadoop fs -ls /out
hadoop fs -cat /out/*
进入Hadoop安全模式
hdfs dfsadmin -safemode enter
退出Hadoop安全模式
hdfs dfsadmin -safemode leave
关闭Hadoop
./stop-all.sh

Guess you like

Origin blog.csdn.net/weixin_44039347/article/details/91600622