1、搭建的大致流程:
- 上传解压hive包
- 配置环境变量
- 修改配置文件
- 上传jar包替换jar包
- 安装mysql
- 启动Hive
- 验证hive远程模式搭建是否成功
2、简单介绍hive远程模式布局
节点名 | 服务名 |
---|---|
hadoop01(主节点) | mysql服务端 |
hadoop02 | Hive服务端 |
Hadoop03 | hive客户端 |
3、搭建步骤
在从节点上作如下操作
1.安装解压hive
2.配置hive环境
vim /etc/profile
source /etc/profile #编辑完环境必须生效环境,否则无效
3、修改Hive配置文件
在Hadoop02上配置hive服务端,hive-site.xml
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/user/hive/warehouse</value>#hdfs存储目录,和客户端保持一致
</property>
<property>
<name>hive.metastore.local</name> #元数据在本机
<value>true</value>
</property>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://192.168.40.128/hive_remote?createDatabaseIfNotExist=true</value>#msql地址
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>#数据库驱动
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>root</value> #mysql用户名
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>123456</value> #密码
</property>
在hadoop03上配置hive客户端,hive-site.xml
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/data/hive/warehouse</value> #hdfs存储目录,和服务端保持一致
</property>
<property>
<name>hive.metastore.local</name>
<value>false</value> #元数据不在本机
</property>
<property>
<name>hive.metastore.uris</name>
<value>thrift://192.168.40.129:9083</value> #服务端hadoop02通信地址、端口
</property>
4、上传jar包替换jar包
1.把JDBC驱动包放到hive/lib目录下
mysql jar 获取链接:
提取码: mavt
2.替换 jline-2.12.jar 包
在主节点上作如下操作
1、安装MySQL
安装mysql步骤
第一种方式
5、启动服务
首先启动Hive服务端:
hive --service metastore
其次启动Hive客户端:
hive
6 测试hive
我们在Hive服务端输入查找数据库的命令:show databases;
出现下图就代表Hive远程搭建成功。
元数据
mysql> show tables;
+---------------------------+
| Tables_in_hivedb |
+---------------------------+
| BUCKETING_COLS |
| CDS |
| COLUMNS_V2 |
| COMPACTION_QUEUE |
| COMPLETED_TXN_COMPONENTS |
| DATABASE_PARAMS |
| DBS |
| DB_PRIVS |
| DELEGATION_TOKENS |
| FUNCS |
| FUNC_RU |
| GLOBAL_PRIVS |
| HIVE_LOCKS |
| IDXS |
| INDEX_PARAMS |
| MASTER_KEYS |
| NEXT_COMPACTION_QUEUE_ID |
| NEXT_LOCK_ID |
| NEXT_TXN_ID |
| NOTIFICATION_LOG |
| NOTIFICATION_SEQUENCE |
| NUCLEUS_TABLES |
| PARTITIONS |
| PARTITION_EVENTS |
| PARTITION_KEYS |
| PARTITION_KEY_VALS |
| PARTITION_PARAMS |
| PART_COL_PRIVS |
| PART_COL_STATS |
| PART_PRIVS |
| ROLES |
| ROLE_MAP |
| SDS |
| SD_PARAMS |
| SEQUENCE_TABLE |
| SERDES |
| SERDE_PARAMS |
| SKEWED_COL_NAMES |
| SKEWED_COL_VALUE_LOC_MAP |
| SKEWED_STRING_LIST |
| SKEWED_STRING_LIST_VALUES |
| SKEWED_VALUES |
| SORT_COLS |
| TABLE_PARAMS |
| TAB_COL_STATS |
| TBLS |
| TBL_COL_PRIVS |
| TBL_PRIVS |
| TXNS |
| TXN_COMPONENTS |
| TYPES |
| TYPE_FIELDS |
| VERSION |
+---------------------------+
53 rows in set (0.00 sec)
第二种方式
检测是否启动(查看端口号)
netstat-antp | grep 9083
netstat-antp | grep 100000
结束进程
java 使用上述查看端口对应的进程的命令找出pid,然后使用kill -9 pid
在hadoop02上(设置成远程模式之后必须保证先启动metastore服务或者进程)
- 先启动元数据服务
hive --service metastore &
- 启动远程服务(另开一台Hadoop02)
hive --service hiveserver2 &
在hadoop03上
beeline
[root@hadoop03 ~]# beeline
Beeline version 1.2.1 by Apache Hive
beeline> !connect jdbc:hive2://hadoop02:10000
Connecting to jdbc:hive2://hadoop02:10000
Enter username for jdbc:hive2://hadoop02:10000: root #用户名
Enter password for jdbc:hive2://hadoop02:10000: **** #密码
Connected to: Apache Hive (version 1.2.1)
Driver: Hive JDBC (version 1.2.1)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://hadoop02:10000> show tables;
+-----------+--+
| tab_name |
+-----------+--+
| stu2 |
| student |
+-----------+--
登录MySQL使用命令或者可视化工具(Navicat)查看元数据信息
mysql> show tables;
+---------------------------+
| Tables_in_hivedb |
+---------------------------+
| BUCKETING_COLS |
| CDS |
| COLUMNS_V2 |
| COMPACTION_QUEUE |
| COMPLETED_TXN_COMPONENTS |
| DATABASE_PARAMS |
| DBS |
| DB_PRIVS |
| DELEGATION_TOKENS |
| FUNCS |
| FUNC_RU |
| GLOBAL_PRIVS |
| HIVE_LOCKS |
| IDXS |
| INDEX_PARAMS |
| MASTER_KEYS |
| NEXT_COMPACTION_QUEUE_ID |
| NEXT_LOCK_ID |
| NEXT_TXN_ID |
| NOTIFICATION_LOG |
| NOTIFICATION_SEQUENCE |
| NUCLEUS_TABLES |
| PARTITIONS |
| PARTITION_EVENTS |
| PARTITION_KEYS |
| PARTITION_KEY_VALS |
| PARTITION_PARAMS |
| PART_COL_PRIVS |
| PART_COL_STATS |
| PART_PRIVS |
| ROLES |
| ROLE_MAP |
| SDS |
| SD_PARAMS |
| SEQUENCE_TABLE |
| SERDES |
| SERDE_PARAMS |
| SKEWED_COL_NAMES |
| SKEWED_COL_VALUE_LOC_MAP |
| SKEWED_STRING_LIST |
| SKEWED_STRING_LIST_VALUES |
| SKEWED_VALUES |
| SORT_COLS |
| TABLE_PARAMS |
| TAB_COL_STATS |
| TBLS |
| TBL_COL_PRIVS |
| TBL_PRIVS |
| TXNS |
| TXN_COMPONENTS |
| TYPES |
| TYPE_FIELDS |
| VERSION |
+---------------------------+
53 rows in set (0.00 sec)
可视化工具查看结果