Use sqoop tools to import data from the database to the HDFS and mysql Hbase

Import data using Sqoop

 

Configuring ssh-free dense Login

Input Terminal: sudo Service ssh restart to restart the ssh service.

Input terminal: the sudo Start-Service MySQL open mysql.

Free secret landing configuration: RM -rf ~ / .ssh / enter ssh-keygen knock three Enter

输入ssh-copy-id user@localhost

Enter yes, you need to enter a password to place input Huawei @ 123, after the start hdfs when you do not need to enter a password more than once. 

 

1. introduced into the hdfs

In the terminal type: start-dfs.sh

 Enter jps view the process:

After entering: Start-yarn.shk open yarn

Enter jps View

After entering: Sqoop Import --connect JDBC: MySQL: // localhost: 3306 / lydsj --username the root -P---table tbl_gender --target the dir / MySQL / Output - M. 1 to import data into the database hdfs directory, the process to enter the database password (my 000000). We need to wait for some time.

Use hadoop fs -cat / mysql / output / part-m-00000 results View introduced

 

2. introduced into the hbase

在开启了hdfs和yarn的情况下,输入:zkServer.sh start开启Zookeeper,之后输入start-hbase.sh开启hbase。

输入jps查看进程:

在终端中输入hbase shell

输入:create 'tbl_dim','info'创建一张表

 输入quit退出hbase shell

然后输入命令sqoop import -m 1 --connect jdbc:mysql://localhost:3306/lydsj --username root --password 000000 --table tbl_dimensioncat --hbase-table tbl_dim --columns DIMCATID,DIMENSIONNAME,MEMO --column-family info --hbase-row-key DIMCATID

将数据导入到Hbase中

再在终端输入habse shell然后在shell中输入scan 'tbl_dim'查看表中的内容

结束。

Guess you like

Origin blog.csdn.net/weixin_44961794/article/details/91064953