hive集群安装，连接mysql

版权声明：本文为博主九师兄（QQ群:spark源代码 198279782 欢迎来探讨技术）原创文章，未经博主允许不得转载。 https://blog.csdn.net/qq_21383435/article/details/82631874

1.linux安装mysql，并且生成hive用户，密码为Abc!123D，权限为所有权限，请看（这点很重要）http://blog.csdn.net/qq_21383435/article/details/76573955

2。我的hadoop是2。7，hive下载的是2.2，下载解压hive到/opt/moudles/apache-hive-2.2.0-bin这个目录
3。配置hive环境变量

export HIVE_HOME=/opt/moudles/apache-hive-2.2.0-bin
export PATH=$PATH:$HIVE_HOME/bin
"/etc/profile" 109L, 2732C                              

4。配置hive的环境

vim hive-env.sh
HADOOP_HOME=/opt/moudles/hadoop-2.7.3
export HIVE_CONF_DIR=/opt/moudles/apache-hive-2.2.0-bin/conf

5.配置vim conf/hive-site.xml

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?><configuration>
  <!-- WARNING!!! This file is auto generated for documentation purposes ONLY! -->
  <!-- WARNING!!! Any changes you make to this file will be ignored by Hive.   -->
  <!-- WARNING!!! You must make your changes in hive-site.xml instead.         -->
  <!-- Hive Execution Parameters -->

    <property>
        <name>hive.metastore.local</name>
        <value>false</value>
    </property>
    <!-- 配置元数据不在本地  -->

    <property>
         <name>hive.metastore.warehouse.dir</name>
         <value>hdfs://mycluster/hive/warehouse</value>
     </property>
    <!-- 配置元数据不在本地 在hdfs://mycluster/hive/warehouse这个目录里，因为我是配置的HA,所以这里用的是mycluster，不是的话，需要配置hadoop的NameNode节点所在的主机名 -->    

    <property>
      <name>javax.jdo.option.ConnectionURL</name>
      <value>jdbc:mysql://biluos.com:3306/hive_db?createDatabaseIfNotExist=true</value>
      <description>JDBC connect string for a JDBC metastore</description>
    </property>
    <!-- 配置hive要连接的数据库 这里有时候需要useSSL=true -->

    <property>
      <name>javax.jdo.option.ConnectionDriverName</name>
      <value>com.mysql.jdbc.Driver</value>
      <description>Driver class name for a JDBC metastore</description>
    </property>
    <!-- 配置驱动  -->

    <property>
      <name>javax.jdo.option.ConnectionUserName</name>
      <value>hive</value>
      <description>username to use against metastore database</description>
    </property>
    <!-- 配置访问MySQL的用户名  -->

    <property>
      <name>javax.jdo.option.ConnectionPassword</name>
      <value>Abc!123D</value>
      <description>password to use against metastore database</description>
    </property>
    <!-- 配置访问MySQL的密码  -->

    <property>
      <name>hive.metastore.uris</name>
      <value>thrift://biluos.com:9083</value>
      <description>Thrift URI for the remote metastore. ...</description>
    </property>
    <!-- 配置不知道啥玩意  -->

    <property>
      <name>hive.metastore.schema.verification</name>
      <value>false</value>
    </property>
    <!-- 配置不知道啥玩意，貌似hive格式化的时候有的出错和这个有关  -->


    <property>
      <name>hive.cli.print.current.db</name>
      <value>true</value>
    </property>
    <!-- 配置使用hive命令行的时候显示当前数据库  -->

    <property>
      <name>hive.cli.print.header</name>
      <value>true</value>
    </property>
    <!-- 配置不知道  -->

</configuration>

6。把MySql驱动放到Hive的lib目录下

上面这个带bin的不知道是不是windows使用的，反正没测试，因为报错太多，用空测试吧
7。初始化hive

[root@biluos apache-hive-2.2.0-bin]# schematool -initSchema -dbType mysql --verbose

–verbose可要可不要，要的话会显示详细信息，我在这里卡了很久。但是不知怎么就解决了。主要是出问题

这里重点检查配置文件和hive用户的权限，主要是连接的URL配置。
成功后mysql中可以看到

8。启动Hive
实际使用时，一般通过后台启动metastore和hiveserver实现服务，命令如下：
hive –service metastore &
hive –service hiveserver &
我现在是测试

然后启动hive命令行

这一点如果先执行这个命令会报错如下，需要先执行hive –service metastore

Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

Caused by: java.lang.reflect.InvocationTargetException

Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused (Connection refused)

Caused by: java.net.ConnectException: Connection refused (Connection refused)

9。hive中建立一个数据库，然后建立一个表create table test(id int,name string); 可以看到网页上可以看到

10。好了，安装完成

---

11，下面是集群的另外一种配置，其他不变只需要修改hive-site.xml就好了

<!-- hive表的默认存储路径 -->
        <property>
         <name>hive.metastore.warehouse.dir</name>
         <value>hdfs://mycluster/hive/warehouse</value>
        </property>


        <!-- 指定mysql的连接 -->
        <property>
          <name>javax.jdo.option.ConnectionURL</name>
          <value>jdbc:mysql://biluos.com:3306/hive_db?createDatabaseIfNotExist=true</value>
          <description>JDBC connect string for a JDBC metastore</description>
        </property>


        <!-- 指定驱动类 -->
        <property>
          <name>javax.jdo.option.ConnectionDriverName</name>
          <value>com.mysql.jdbc.Driver</value>
          <description>Driver class name for a JDBC metastore</description>
        </property>


        <!-- 指定用户名 -->
        <property>
          <name>javax.jdo.option.ConnectionUserName</name>
          <value>hive</value>
          <description>username to use against metastore database</description>
        </property>

        <!-- 指定密码 -->
        <property>                                                                                                                                                         
          <name>javax.jdo.option.ConnectionPassword</name>                                                                                                                 
          <value>Abc!123D</value>                                                                                                                                          
          <description>password to use against metastore database</description>                                                                                            
        </property>                                                                                                                                                        


        <!-- thrift://<host_name>:<port> 默认端口是9083 -->                                                                                                                
        <property>                                                                                                                                                         
          <name>hive.metastore.uris</name>                                                                                                                                 
          <value>thrift://biluos.com:9083</value>                                                                                                                          
          <description>Thrift URI for the remote metastore. ...</description>                                                                                              
        </property>  

        <!--指定zookeeper-->                                                                                                                                               
        <property>                                                                                                                                                         
                <name>hbase.zookeeper.quorum</name>                                                                                                                        
                <value>biluos.com,biluos1.com,biluos2.com</value>                                                                                                          
        </property>