Hadoop - familiar with common HDFS operations

1. The Shell commands provided by Hadoop accomplish the same task:

Create a file txt in the "/home/hadoop/" directory of the local Linux file system, and enter some words at will.
View file location locally (ls)

Display file contents locally

cd /usr/local/hadoop
    touch hello.txt
    
    cat hello.txt

Use the command to upload "txt" in the local file system to the input directory of the current user directory in HDFS.

/sbin/start-dfs.sh
    ./bin/hdfs dfs -mkdir -p /user/hadoop
    ./bin/hdfs dfs -mkdir input
    ./bin/hdfs dfs -put ./hello.txt input

View files in hdfs (-ls)
```
./bin/hdfs dfs -ls /input
```
Display the content of the file in hdfs
```
./bin/hdfs dfs -cat input/hello.txt
```
delete the local txt file and view the directory
```
./bin/hdfs dfs -rm -ls input/hello.txt
```
Download the txt to the original local location from hdfs.
```
./bin/hdfs dfs -get input/test.txt ~/hello.txt
```
delete txt from hdfs and view directory
```
./bin/hdfs dfs -rm -ls input/hello.txt
```

two,

Upload any text file to HDFS. If the specified file already exists in HDFS, the user specifies whether to append to the end of the original file or overwrite the original file;
```
if $(hdfs dfs -test -e hello.txt);
then $(hdfs dfs -appendToFile local.txt hello.txt);
else $(hdfs dfs -copyFromLocal -f local.txt hello.txt);
be
```
Download the specified file from HDFS, if the local file has the same name as the file to be downloaded, the downloaded file will be renamed automatically;
```
if $(hdfs dfs -test -e file:
then $(hdfs dfs -copyToLocal hello.txt ./hello2.txt);
else $(hdfs dfs -copyToLocal hello.txt ./hello.txt);
be
```
Output the contents of the specified file in HDFS to the terminal;
```
hdfs dfs -cat hello.txt
```
Display the read and write permissions, size, creation time, path and other information of the specified file in HDFS;
```
hdfs dfs -ls -h hello.txt
```
Given a directory in HDFS, output the read and write permissions, size, creation time, path and other information of all files in the directory. If the file is a directory, recursively output all file related information in the directory;
```
hdfs dfs -ls -R -h /user/hadoop
```
Provide the path of a file in HDFS, and perform creation and deletion operations on the file. If the directory where the file is located does not exist, the directory will be created automatically;
```
if $(hdfs dfs -hello -d dir1/dir2);
then $(hdfs dfs -touchz dir1/dir2/filename);
else $(hdfs dfs -mkdir -p dir1/dir2 && hdfs dfs -touchz dir1/dir2/filename);
be
```
Provides the path of an HDFS directory, and performs create and delete operations on the directory. When creating a directory, if the directory where the directory file is located does not exist, the corresponding directory is automatically created; when deleting a directory, the user specifies whether to delete the directory when the directory is not empty;
```
if $(hdfs dfs -hello -d dir1/dir2);
then $(hdfs dfs -touchz dir1/dir2/filename);
else $(hdfs dfs -mkdir -p dir1/dir2);
be
if$(hdfs dfs -rmdir dir1/dir2);
then $(hdfs dfs -rmdir dir1/dir2)
be
```

Append content to the file specified in HDFS, and append the content specified by the user to the beginning or end of the original file;

Append to end of file: hdfs dfs - appendToFile local.txt hello.txt
Append to the beginning of the file:
(Because there is no direct command to operate, one of the methods is to move to the local operation first, and then upload and cover it):
hdfs dfs -get hello.txt
cat text.txt >> local.txt
hdfs dfs -copyFromLocal -f hello.txt hello.txt

Delete the specified file in HDFS;
```
hdfs dfs -rm hello.txt
```

Delete the directory specified in HDFS, whether to delete the directory if there is a file in the directory specified by the user;

Delete the directory (if the directory is not empty, it will prompt not empty and do not delete it): hdfs dfs -rmdir dir1/ dir2
Force delete a directory: hdfs dfs -rm -R dir1/dir2

In HDFS, move files from source path to destination path.
```
hdfs dfs -mv hello.txt hello2.txt
```

Hadoop - familiar with common HDFS operations

Guess you like