Hive cluster construction under CentOS7

1. Download Hive

This example uses apache-hive-2.1.1. Please download hive locally and decompress it as needed. Download link: http://archive.apache.org/dist/hive/
uncompressed path:

[root@hadoop-master apache-hive-2.1.1]# pwd
/usr/local/hive/apache-hive-2.1.1

2. Install mysql database

There are three ways to build Hive
内嵌Derby方式: When using the derby storage method, running hive will generate a derby file and a metastore_db directory in the current directory. The disadvantage of this storage method is that only one hive client can use the database in the same directory at the same time.
本地模式: This storage method needs to run a mysql server locally and configure the following (the following two ways to use mysql, you need to copy the mysql jar package to the $HIVE_HOME/lib directory).
多用户模式: This storage method needs to run a mysql server on the remote server, and needs to start the meta service on the Hive server.
In the final analysis 元数据的存储位置不一样, the three methods are multi-user mode.
For the installation of mysql, please refer to "Installing mysql-5.7.24 under CentOS7"

3. Modify the configuration file

First enter the following directory, edit the hive-site.xmlfile, add a new one if not

[root@hadoop-master conf]# vi /usr/local/hive/apache-hive-2.1.1/conf/hive-site.xml 

The content of hive-site.xml:

<?xml version="1.0" encoding="utf-8"?>

<configuration>

 <!--  在hdfs上文件的位置  -->
  <property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/usr/local/hive/warehouse</value>
  </property>

  <!--数据连接地址  -->
  <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://hadoop-master:3306/hive?createDatabaseIfNotExist=true</value>
    <description>JDBC connect string for a JDBC metastore</description>
  </property>
  <!-- 数据库驱动，这里使用mysql数据库驱动 -->
  <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    <description>Driver class name for a JDBC metastore</description>
  </property>
  <!-- 数据库用户名，根据自己的数据库用户名填写 -->
  <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>root</value>
    <description>username to use against metastore database</description>
  </property>
  <!-- 数据库密码，根据自己的数据库密码填写 -->
  <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>root</value>
    <description>password to use against metastore database</description>
  </property>

 <property>
   <name>hive.metastore.schema.verification</name>
   <value>false</value>
    <description>
    Enforce metastore schema version consistency.
    True: Verify that version information stored in metastore matches with one from Hive jars.  Also disable automatic
          schema migration attempt. Users are required to manully migrate schema after Hive upgrade which ensures
          proper metastore schema migration. (Default)
    False: Warn if the version information stored in metastore doesn't match with one from in Hive jars.
    </description>
 </property>

 <property>
     <name>hive.querylog.location</name>
     <value>/usr/local/hive/tmp</value>
 </property>

 <property>
     <name>hive.exec.local.scratchdir</name>
     <value>/usr/local/hive/tmp</value>
 </property>

 <property>
     <name>hive.downloaded.resources.dir</name>
     <value>/usr/local/hive/tmp</value>
 </property>

 <property>
     <name>datanucleus.schema.autoCreateAll</name>
     <value>true</value>
 </property>
</configuration>

4. Add mysql driver to hive

Add in the following directorymysql-connector-java-5.1.30.jar

[root@hadoop-master lib]# pwd
/usr/local/hive/apache-hive-2.1.1/lib

And chomd 777 mysql-connector-java-5.1.30.jargrant permissions

5. Add the hive command to the environment variable

[root@hadoop-master bin]# vi /etc/profile

Add HIVE_HOME to environment variables

#java环境变量
export JAVA_HOME=/usr/local/jdk/jdk1.8.0_261
export CLASSPATH=.:${
    
    JAVA_HOME}/jre/lib/rt.jar:${
    
    JAVA_HOME}/lib/dt.jar:${
    
    JAVA_HOME}/lib/tools.jar
export PATH=$PATH:${
    
    JAVA_HOME}/bin

#配置Hadoop环境变量
export HADOOP_HOME=/usr/local/hadoop/apps/hadoop-2.7.3
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:

#配置hive环境变量
export HIVE_HOME=/usr/local/hive/apache-hive-2.1.1
export PATH=$PATH:$HIVE_HOME/bin:$HIVE_HOME/sbin:

Perform refresh after adding:

[root@hadoop-master bin]# source /etc/profile

6. Start hive

If startup error:

MetaException(message:Hive metastore database is not initialized. Please use schematool (e.g. ./sch

You can do as follows: https://blog.csdn.net/beidiqiuren/article/details/53056270 You
need to execute the following instructions (executed in the bin directory of hive) to initialize metadata

[root@hadoop-master bin]# ./schematool -initSchema -dbType mysql

Finally start hive

[root@hadoop-master bin]# hive
which: no hbase in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/local/jdk/jdk1.8.0_261/bin:/usr/local/hadoop/apps/hadoop-2.7.3/bin:/usr/local/hadoop/apps/hadoop-2.7.3/sbin::/root/bin:/usr/local/jdk/jdk1.8.0_261/bin:/usr/local/hadoop/apps/hadoop-2.7.3/bin:/usr/local/hadoop/apps/hadoop-2.7.3/sbin::/usr/local/hive/apache-hive-2.1.1/bin:/usr/local/hive/apache-hive-2.1.1/sbin:)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hive/apache-hive-2.1.1/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop/apps/hadoop-2.7.3/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]

Logging initialized using configuration in jar:file:/usr/local/hive/apache-hive-2.1.1/lib/hive-common-2.1.1.jar!/hive-log4j2.properties Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
hive> 

The other machines only need to configure the environment variables of hive.