MongoDB's hive data read mongodb

MongoDB (c) of the hive data read mongodb

Description: users with the table data MongoDB (II) users table

1 jar prepared package

mongo-hadoop-core-2.0.2.jar;
mongo-hadoop-hive-2.0.2.jar;
mongo-java-driver-3.9.1.jar;
mongodb-driver-3.9.1.jar;

2 jar package uploaded to the hdfs

hdfs dfs -mkdir /user/mongo
hdfs dfs -put mongo*.jar /user/mongo
hdfs dfs -ls /user/mongo

3 to complete the hive and mongoDB write data
3.1 into the hive

beeline -u 'jdbc:hive2://localhost:10000' '' ''

3.2 Adding jar package from the hive hdfs

add jar hdfs://sandbox-hdp.hortonworks.com:8020/user/mongo/mongo-hadoop-core-2.0.2.jar;
add jar hdfs://sandbox-hdp.hortonworks.com:8020/user/mongo/mongo-hadoop-hive-2.0.2.jar;
add jar hdfs://sandbox-hdp.hortonworks.com:8020/user/mongo/mongodb-driver-3.9.1.jar;
add jar hdfs://sandbox-hdp.hortonworks.com:8020/user/mongo/mongo-java-driver-3.9.1.jar;

And 3.3 users create the external table is read from the table data users mongoDB

drop table if exists demo.users;
create external table demo.users(object_id STRING,
 user_id STRING,
 locale STRING,
 birthyear INT,
 gender STRING,
 joined_at STRING,
 location STRING,
 time_zone STRING
)
 stored by 'com.mongodb.hadoop.hive.MongoStorageHandler'
 with serdeproperties('mongo.columns.mapping'='{"object_id":"_id","user_id":"user_id","locale":"locale","birthyear":"birthyear","gender":"gender"},"joinedAt":"joinedAt","location":"location","timezone":"timezone"')
 tblproperties('mongo.uri'='mongodb://192.168.30.1:27017/events_db.users');

192.168.30.1:27017 for the ip and port number where I mongoDB virtual machines, remember to modify for your own Oh! ! !

3.4 After successful execution, view the users table in the hive

use demo;
show tables;

Note: Due to an external table, so the data is still stored in mongoDB must guarantee the existence of the users table mongoDB in events_db library, or they will not find the data.

Published 22 original articles · won praise 22 · views 768

Guess you like

Origin blog.csdn.net/weixin_45568892/article/details/105293405