配置Hive使用Mysql作为数据源

版权声明:本文为博主原创文章,未经博主允许不得转载。 https://blog.csdn.net/qq_36933797/article/details/79601170

Hive 默认使用 derby 作为映射表(SQL 操作映射为MapReduce Job,将SQL中创建的表映射为 hdfs 的文件/文件夹,字段映射为其中的行),但 derby 的一大缺陷在于它不允许多个客户端同时执行sql操作(可能新版本的hive会有所升级)

使用 mysql 作为 hive 的metastore

使用 hive 连接 mysql 其实意味着使用 JDBC 来连接,不妨再来看一下,hive的体系结构(hive 在hadoop的基础上,既提供了命令行的接口,输入hive进入,也提供了JDBC/ODBC的连接接口,当然还有一套Web GUI):
这里写图片描述
(1)把 mysql 的 jdbc 驱动(mysql-connector-java-5.1.46.jar)放置到 hive 的 lib 目录下;
(2)修改$HIVE_HOME/conf下的 hive-site.xml(由cp hive-default.xml.template hive-site.xml而来)

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?><!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
--><configuration>
  <!-- WARNING!!! This file is auto generated for documentation purposes ONLY! -->
  <!-- WARNING!!! Any changes you make to this file will be ignored by Hive.   -->
  <!-- WARNING!!! You must make your changes in hive-site.xml instead.         -->
  <!-- Hive Execution Parameters -->
    <property>
        <name>javax.jdo.option.ConnectionPassword</name>
        <value>mysql</value>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionURL</name>
        <value>jdbc:mysql://172.16.41.128:3306/hive_metadata?createDatabaseIfNotExsit=true&amp;characterEncoding=UTF-8&amp;verifyServerCertificate=false&amp;useSSL=false</value>
    </property>
                            <!--如果数据库不存在的话,创建数据库名为hive_metadata的数据库>

    <property>
        <name>javax.jdo.option.ConnectionDriverName</name>
        <value>com.mysql.jdbc.Driver</value>
    </property>
                            <!--设置数据库驱动>
    <property>
        <name>javax.jdo.option.ConnectionUserName</name>
        <value>root</value>
    </property>
                            <!--用户名>
    <property>
        <name>javax.jdo.option.ConnectionPassword</name>
        <value>xxxx</value><!--这里填数据库的密码>
    </property>
    <property>
        <name>datanucleus.schema.autoCreateAll</name>
        <value>true</value>
    </property>
    <property>
        <name>hive.metastore.schema.verification</name>
        <value>false</value>
    </property>
</configuration>

初始化schema

bin/schematool -initSchema -dbType mysql

这里写图片描述

(3)远程登录(使用Navicat for Mysql)登录此节点上的mysql(主机名,用户名,密码,如前所设),会在Navicat 中发现创建的数据库hive,其内的表DBS的 DB_LOCATION_URI(也即指向的路径),为hdfs://sunyuqiang.com:9000/user/hive/warehouse(此路径由hive-site.xml中的hive.metastore.warehouse.dir所决定)。
这里写图片描述

猜你喜欢

转载自blog.csdn.net/qq_36933797/article/details/79601170