table of Contents:
Similarly Hadoop, Hive also has three different deployment models (this paper, second):
Embedded mode : Hive metadata stored in the embedded Derby database, only allows a session connection
local mode :( to MySQL, for example) in a local MySQL installation, metadata stored in MySQL, supports multiple sessions, and more user connections
remote mode : metadata stored in the database at the distal end
1. Download and install hive
I use hive3.1.2, at the following address:
https://mirrors.tuna.tsinghua.edu.cn/apache/hive/
Unzip and rename the folder to hive
Cd / Usr / Software // Kai压 Tar -zxvf Apache-Hive- 3.1 . 2 - Bin.Tar.Gz
// heavy naming Mv Apache -Hive- 3.1 . 2 -Bin Hive
2. Configure Hive
2.1 Configuration Environment Variables
vi / etc / Profile, add the HIVE_HOME
HADOOP_HOME=/usr/software/hadoop HIVE_HOME=/usr/software/hive JAVA_HOME=/usr/software/java/jdk1.8 JAVA_BIN=/usr/software/java/jdk1.8/bin PATH=$PATH:$JAVA_HOME/bin:$HIVE_HOME/bin:$HADOOP_HOME/bin CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar export JAVA_HOME JAVA_BIN PATH CLASSPATH HIVE_HOME
Execute>. / Etc / profile so that the environment variables to take effect
2.2 modify the configuration file hive-site.xml
Into the hive configuration directory:
cd /usr/software/hive/conf
The hive-default.xml.template rename hive-site.xml
sudo mv hive-default.xml.template hive-site.xml
Create a new configuration file and edit the hive-site.xml
vi hive-site.xml
Write MySQL configuration information (although we have not installed MySQL) in the hive-site.xml:
Note that the third part <value>hadoop</value>of the hadoop user name for MySQL
Similarly, the fourth part <value>123456</value>of the MySQL password is 123456
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration> <property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true&useSSL=false</value> <description>JDBC connect string for a JDBC metastore</description> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.jdbc.Driver</value> <description>Driver class name for a JDBC metastore</description> </property> <property> <name>javax.jdo.option.ConnectionUserName</name> <value>root</value> <description>username to use against metastore database</description> </property> <property> <name>javax.jdo.option.ConnectionPassword</name> <value>admin</value> <description>password to use against metastore database</description> </property> </configuration>
A configuration again, look for the convenience, with separate area above
vi hive-site.xml
1. See hive-site.xml configuration, see the configuration values comprising "system: java.io.tmpdir" configuration item
2 containing the: value of the configuration item "system java.io.tmpdir" address is modified as follows , or else to hive error
<property> <name>hive.exec.scratchdir</name> <value>/tmp/hive</value> <description>HDFS root scratch dir for Hive jobs which gets created with write all (733) permission. For each connecting user, an HDFS scratch dir: ${hive.exec.scratchdir}/<username> is created, with ${hive.scratch.dir.permission}.</description> </property> <property> <name>hive.exec.local.scratchdir</name> <value>/tmp/hive/local</value> <description>Local scratch space for Hive jobs</description> </property> <property> <name>hive.downloaded.resources.dir</name> <value>/tmp/hive/resources</value> <description>Temporary local directory for added resources in the remote file system.</description> </property>
2.3 modify the configuration file hive-env.sh
Increase in hive-env.sh in:
# Hadoop position: Export HADOOP_HOME is = / usr / Software / Hadoop # hive conf directory location: Export HIVE_CONF_DIR = / usr / Software / Hive / conf lib directory location # hive of: Export HIVE_AUX_JARS_PATH = / usr / Software / Hive / lib
3. Download and install MySQL
mysql installation can be found in this article: Linux under jdk, tomcat, mysql installation
In this simple mention
apt-get install mysql-server
After installation, the account is assigned a privilege to allow even outside
mysql> grant all privileges on *.* to 'root'@'%' identified by 'root' with grant option mysql> flush privileges;
Then mysql driver jar package thrown hive / lib directory
sudo cp -rf /home/mysql-connector-java-5.1.46/mysql-connector-java-5.1.46-bin.jar /usr/software/hive/lib
4. Configure Permissions
Create the relevant warehouse on hdfs, and configure the permissions:
If you do not start hadoop, please start hadoop
4.1 Creating a warehouse and configure permissions
hadoop fs -mkdir -p /user/hive/warehouse hadoop fs -mkdir -p /user/hive/tmp hadoop fs -mkdir -p /user/hive/log hadoop fs -chmod -R 777 /user/hive/warehouse hadoop fs -chmod -R 777 /user/hive/tmp hadoop fs -chmod -R 777 /user/hive/log
Note that a direct call hadoop fs command requires a configured environment variables related
4.2 Initialization
cd /usr/local/hive/bin
schematool -initSchema -dbType mysql
We see schemaTool completedthat initialization is successful!
5. Start the hive and testing
Please make sure hadoop running normally! # Start hive (due configured relevant environmental variables directly): Hive # successful start, the startup process after the prompt information, echo: Hive > # indicating that it has successfully started. Also, pay attention to " ; " the end of a command logo!
5.1 Test hive
# Display Database: Hive > Show Databases; the OK default Time taken: 14.107 seconds The, FETCHED:. 1 Row (S) # Create a table: Hive > Create Table test1 (TID int, tname String); the OK Time taken: 5.021 seconds The # Display the tables: Hive > Show the tables; the OK test1 Time taken: 5.077 seconds the, FETCHED: 1 Row (S) # delete just created table test1: Hive > drop the table test1; the OK Time taken: 5.223 seconds the
#Re-create table test1 (for mysql test): Hive > the Create the Table test1 (int tid, tname String); the OK Time taken: 1.322 seconds The # exit shell Hive Hive > Exit;