Common HDFS operations

1. The Shell commands provided by Hadoop accomplish the same task:

1. Create a file txt in the "/home/hadoop/" directory of the local Linux file system, and enter some words at will.
2. View file location locally (ls)
3. Display file contents locally
4. Use the command to upload "txt" in the local file system to the input directory of the current user directory in HDFS.
5. View files in hdfs (-ls)
6. Display the content of the file in hdfs
7. delete the local txt file and view the directory
8. Download the txt to the original local location from hdfs.
9. delete txt from hdfs and view directory   

     

cd /home/hadoop

touch wzh.txt

ls /home/hadoop

cat wzh.txt

./sbin/start-dfs.sh
./bin/hdfs dfs -mkdir -p /user/hadoop
./bin/hdfs dfs -mkdir input
 ./bin/hdfs dfs -put ./wzh.txt input

./bin/hdfs dfs -ls input

./bin/hdfs dfs -ls /input
./bin/hdfs dfs -cat input/wzh.txt

cd hadoop
 rm -r wzh.txt
ls -al

./bin/hdfs dfs -get /input/wzh.txt ~/hadoop

./bin/hdfs dfs -rm input/wzh.txt
./bin/hdfs dfs -ls input

 

 two,

1. Upload any text file to HDFS. If the specified file already exists in HDFS, the user specifies whether to append to the end of the original file or overwrite the original file;
2. Download the specified file from HDFS, if the local file has the same name as the file to be downloaded, the downloaded file will be renamed automatically;
3. Output the contents of the specified file in HDFS to the terminal;
4. Display the read and write permissions, size, creation time, path and other information of the specified file in HDFS;
5. Given a directory in HDFS, output the read and write permissions, size, creation time, path and other information of all files in the directory. If the file is a directory, recursively output all file related information in the directory;
6. Provide the path of a file in HDFS, and perform creation and deletion operations on the file. If the directory where the file is located does not exist, the directory will be created automatically;
7. Provides the path of an HDFS directory, and performs create and delete operations on the directory. When creating a directory, if the directory where the directory file is located does not exist, the corresponding directory is automatically created; when deleting a directory, the user specifies whether to delete the directory when the directory is not empty;
8. Append content to the file specified in HDFS, and append the content specified by the user to the beginning or end of the original file;
9. Delete the specified file in HDFS;
10. Delete the directory specified in HDFS, whether to delete the directory if there is a file in the directory specified by the user;
11. In HDFS, move files from source path to destination path.

hadoop fs -put /home/hadoop/wzh.txt /usr/local/hadoop
hadoop fs -test -e /usr/local/hadoop/wzh.txt



if $(hadoop fs -test -e /home/hadoop/wzh.txt); <br>then $(hadoop fs -copyToLocal /usr/local/hadoop/wzh.txt /home/hadoop/wzh.txt);<br>else $(hadoop fs -copyToLocal /usr/local/hadoop/wzh.txt /home/hadoop/wzh2.txt); fi

hadoop dfs -cat /usr/local/hadoop/wzh.txt

hadoop dfs -ls -h /usr/local/hadoop/wzh.txt

hadoop dfs -ls -R -h /usr/local/hadoop



if $(hadoop dfs -test -d /usr/local/hadoop/test); then $(hadoop dfs -touch /usr/local/hadoop/test/wzh1.txt); else $(hadoop dfs -mkdir -p /usr/local/hadoop/test && hadoop dfs -touch /usr/local/hadoop/test/wzh1.txt); fi

hadoop dfs -rm -r /usr/local/hadoop/test

hadoop dfs -rm /usr/local/hadoop/wzh.txt

hadoop dfs -rmr /usr/local/hadoop

hadoop fs -mv /usr/local/hadoop/wzh.txt /usr/local/hadoop/hadoop_tmp/wzh.txt