Introduction
Hive is based on Hadoop data warehousing tools, you can map the structure of the data file to a database table, and provide a simple SQL query capabilities, you can convert SQL statements to run MapReduce tasks. The advantage is the low cost of learning, you can quickly achieve a simple MapReduce statistics by type of SQL statements, without having to develop specialized MapReduce applications, data warehouse is very suitable for statistical analysis.
Hive is built on the basis of static batch Hadoop, Hadoop usually have higher because the delay and the time of filing and scheduling of the job requires a lot of overhead. Therefore, Hive is not suitable for applications that require low latency, it is most suitable for use in a large number of batch jobs based on immutable data, such as network log analysis.
Hive is characterized by: a scalable (dynamically add device on Hadoop cluster), scalable, fault-tolerant, the output format of loose coupling.
Hive metadata stored in a relational database (RDBMS), such as MySQL, Derby in.
There are three modes Hive connected to the data, which is: single-user mode, multi-user mode and a remote service mode. (That is, embedded mode, local mode, remote mode).
Experimental Procedure
1. First, local Linux, the new / data / hive1 directory for storing the desired file.
mkdir -p / data / hive1
the directory to the switching / data / hive1, using wget command to download the required installation package hive hive-1.1.0-cdh5.4.5.tar.gz and mysql-connector-java-5.1.26- bin.jar.
cd /data/hive1
wget http://59.74.172.143:60000/allfiles/hive1/hive-1.1.0-cdh5.4.5.tar.gz
wget http://59.74.172.143:60000/allfiles/hive1/mysql-connector-java-5.1.26-bin.jar
2. The hive-1.1.0-cdh5.4.5.tar.gz in / data / hive1 directory, extract to / apps directory.
tar -xzvf hive-1.1.0-cdh5.4.5.tar.gz -C / apps /
then switch to the next / apps directory, /apps/hive-1.1.0-cdh5.4.5, rename hive.
cd /apps
mv /apps/hive-1.1.0-cdh5.4.5/ /apps/hive
3. Use vim to open the user environment variables.
sudo vim ~ / .bashrc
The Hive bin directory, add the user to your PATH environment variable, and then save and exit.
config #hive
Export HIVE_HOME = / Apps / Hive
Export HIVEHOME the PATH = / bin:
HIVEH OME / bin: the PATH
source your command, the Hive environment variables to take effect.
source ~/.bashrc
4. As Hive required metadata, stored in Mysql. It is necessary to copy mysql-connector-java-5.1.26-bin.jar in / data / hive1 lib directory to the directory of the hive.
cp /data/hive1/mysql-connector-java-5.1.26-bin.jar /apps/hive/lib/
The following configuration Hive, switch to the next / apps / hive / conf directory, and create a configuration file Hive hive-site.xml.
CD / Apps / Hive / the conf
Touch hive-site.xml
using vim open hive-site.xml file.
vim hive-site.xml
and the following configuration items, added to the hive-site.xml file.
<configuration>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExsit=true;characterEncoding=latin1</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>root</value>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>strongs</value>
</property>
</configuration>
Since the Hive metadata stored in the Mysql database, so it is necessary Hive configuration file to specify the information mysql.
javax.jdo.option.ConnectionURL: database link string.
javax.jdo.option.ConnectionDriverName: database connection driver package.
javax.jdo.option.ConnectionUserName: database user name.
javax.jdo.option.ConnectionPassword: password to connect to the database.
Here the database user name and password, you need to set the username and password for the database system itself.
6. In addition, the need to tell the Hive, Hadoop environment configuration. So we need to modify hive-env.sh file.
First, we will hive-env.sh.template rename hive-env.sh .
/apps/hive/conf/hive-env.sh.template /apps/hive/conf/hive-env.sh mv
mv /apps/hive/conf/hive-env.sh.template / Apps / Hive / conf / Hive -env.sh
use vim open hive-env.sh file.
Vim hive-env.sh
added Hadoop path, and the path to the configuration file Hive file.
# Set HADOOP_HOME to point to a specific hadoop install directory
# HADOOP_HOME=${bin}/../../hadoop
HADOOP_HOME=/apps/hadoop
# Hive Configuration Directory can be controlled by:
# export HIVE_CONF_DIR=
export HIVE_CONF_DIR=/apps/hive/conf
7. The next step is to configure Mysql, the metadata storage Hive.
First, we need to ensure that Mysql has been launched. Execute the following command to view the running status of Mysql.
sudo service mysql status
Through the output, it can be seen Mysql not started. Need to do a start command.
sudo service mysql start
if Mysql is not installed you need to perform the installation command. If our environment is Mysql installed, you do not need to perform this step.
sudo apt-get install mysql-server
8. Turn Mysql database.
mysql -u root -p
You are prompted for a password, the password is here strongs
Create a database, encoding format called hive is latin1, for storing metadata.
create database hive CHARACTER SET latin1;
see if the database was created successfully.
show databases;
below, enter exit to exit Mysql.
exit
9. The test is performed. Since the Hive processing of data, depending MapReduce computation model, so it is necessary to ensure that Hadoop-related process has started.
Enter jps, view the process status. If Hadoop-related process is not started, you need to start Hadoop.
/apps/hadoop/sbin/start-all.sh
After starting the Hadoop, the command line interface in the terminal, input can start hive Hive command line.
hive
input HQL statement to query the database to test whether the Hive can be used normally.
show databases;
At this point Hive installed.