HDFS简单操作与sqoop建表操作

目录

sudo -u hdfs hdfs dfs -ls -h /flume_xdr

条数

sudo -u hdfs hdfs dfs -cat /flume_xdr/* | wc -l

数据

hdfs dfs -cat /flume_5min/2018-10-30/15/5min_20181030_16.1540886400001.csv |head -n 100

 

flume接数

tail -F /etc/flume-ng/conf/nohup.out

yarn logs -applicationId application_1540954664082_0090

 

sqoop建表

sqoop list-databases --connect jdbc:postgresql://172.22.13.222:5432/ --username postgres

sqoop list-tables --connect jdbc:postgresql://172.22.13.222:5432/postgres --username postgres

sqoop export --connect "jdbc:postgresql://172.22.13.222:5432/postgres?useUnicode=true&characterEncoding=utf-8" --username postgres  \

--table xdr_table_102223 -m 3 --export-dir /data --update-mode allowinsert --input-fields-terminated-by ',' --input-null-string "\\\\N" --input-null-non-string "\\\\N" 

ftp操作

ftp localhost

输入账号密码

cd

ls

get

put

参考:

https://blog.csdn.net/sunshingheavy/article/details/53227581