Big data--Hadoop environment deployment (5) Hive deployment

Hadoop cluster deployment: https://www.cnblogs.com/Studywith/p/16948866.html

1. Three deployment modes of Hive

1. Embedded mode
Using the embedded Derby database to store metadata is the easiest way to deploy Hive. Hive in embedded mode does not support multi-session connections and is not suitable for production environments, but only for test environments.
2. Local mode
Use a local independent database to store metadata, and the independent database here usually uses the MySQL database. Hive deployed in local mode supports metadata sharing and supports multi-session connections.
3. Remote mode
The remote mode, like the local mode, also uses an independent database to store metadata. The difference is that the remote mode uses a remote independent database, while the local mode uses a local independent database. The remote mode is mainly used when there are many Hive clients.

2. Embedded mode

1. Download the installation package + upload + decompress

Official website: https://dlcdn.apache.org/hive/

Downloaded here is apache-hive-2.3.9-bin.tar.gz, uploaded to /export/software/ of node01, decompressed to /export/servers/

tar -zxvf /export/software/apache-hive-2.3.9-bin.tar.gz -C /export/servers/

2. System environment variables

It is convenient to call Hive directly under each folder

vim /etc/profile

HIVE_HOME=/export/servers/apache-hive-2.3.9-bin
PATH=$PATH:$HIVE_HOME/bin
export HIVE_HOME PATH

source /etc/profile

3. Hive configuration file

Reference: https://blog.csdn.net/QYHuiiQ/article/details/124157773

(1)hive-env.sh

cd /export/servers/apache-hive-2.3.9-bin/conf
cp hive-env.sh.template hive-env.sh
vim hive-env.sh

export HIVE_CONF_DIR=/export/servers/apache-hive-2.3.9-bin/conf
export JAVA_HOME=/export/servers/jdk1.8.0_161
export HADOOP_HOME=/export/servers/hadoop-2.7.4
export HIVE_AUX_JARS_PATH=/export/servers/apache-hive-2.3.9-bin/lib

(2)hive-site.xml

cp hive-default.xml.template hive-site.xml
vim hive-site.xml

(命令的意思就是全文查找对应符号,将符号替换为指定符号)
#1.将${system:java.io.tmpdir}替换为我们的tmp目录(iotmp，该目录会自动创建)。直接在当前模式下输入以下命令即可替换，不需要切换为insert模式

:%s#${system:java.io.tmpdir}#/export/servers/apache-hive-2.3.9-bin/iotmp#g

#执行完之后，会提示我们替换了4个地方。
 

#2.再将系统用户名替换为root

:%s/${system:user.name}/root#g

#执行完之后提示我们修改了3个地方。

4. Initialize Derby

Before starting Hive, you need to initialize the Derby database in the Hive installation directory. If the message "schemaTool completed" appears, it proves that the Derby database has been successfully initialized.

cd /export/servers/apache-hive-2.3.9-bin/
bin/schematool -initSchema -dbType derby

5. Start Hadoop

Hadoop must be started before Hive

#node01,02,03分别依次执行(具体见上文Hadoop执行)
zkServer.sh start
zkServer.sh status
hadoop-daemon.sh start journalnode

#node01执行
start-dfs.sh
start-yarn.sh

#完成后分别执行jps查看启动情况

node01

node02

node03

6. Start the Hive client tool

Execute the "hive" command to start the Hive client tool HiveCLI. You can execute the "quit;" command to exit the Hive client tool HiveCLI. At this time, the file derby.log and the folder metastore_db will be generated by default in the Hive installation directory.

The other two are to be updated~