hive using simple structures (1)
Tags (separated by spaces): hive, mysql
hive assembly corresponding to the compiler, he does not store data, the metadata stored in mysql, the data is stored in hdfs by hive, you can use the data stored in the sql statement hdfs processed.
Preparation Before Installation
Hadoop to build a good (refer to the previous article)
MySQL-5.6.34-Linux-glibc2.5-x86_64.tar.gz
the Apache-Hive-2.1.1-bin.tar.gz
MySQL installation reference the blog: HTTPS: // www.cnblogs.com/clsn/p/8025324.html
Precautions
My path in the / root / down, you want to modify the following operating according to their own
operating system is centos7
Detailed steps to build
1.mysql installation
1. build mysql, mysql does not need to have installed on each machine, only you need to install the line on namenode (host), if ensure high availability, you can also do a master-slave replication.
2. This compressed package placed in the / root / directory and execute unzip command:
tar xf mysql-5.6.34-linux-glibc2.5-x86_64.tar.gz
mv mysql-5.6.34-linux-glibc2.5-x86_64 mysql3. Create a user mysql landing system but can not manage database storage directory:
cd mysql
useradd -s /sbin/nologin -M mysql4. Authorization for database storage directory:
chown -R mysql.mysql ./mysql/data/5. initialize the database:
./mysql/scripts/mysql_install_db --basedir=/root/mysql --datadir=/root/mysql/data --user=mysql
###备注：这一步可能会报错，是因为没有下载perl 和 perl-devel，用yum进行安装即可6. The startup script into the startup directory, and modify two files path:
cp /root/mysql/support-files/mysql.server /etc/init.d/mysqld
sed -i 's#/usr/local/mysql#/root/mysql#g' /root/mysql/bin/mysqld_safe
sed -i 's#/usr/local/mysql#/root/mysql#g' /etc/init.d/mysql7. Copy the configuration file that comes with mysql as /etc/my.cnf:
cp /root/mysql/support-files/my-default.cnf /etc/my.cnf8. Start mysql, execute the command:
service mysqld start9. Set the root password:
/root/mysql/bin/mysqladmin -u root password 'mysql123456'10. Log mysql, execute the command:
/root/mysql/bin/mysql -uroot -pmysql123456
2.hive installation
1. Extract hive, using the version apache-hive-2.1.1-bin.tar.gz
tar xf apache-hive-2.1.1-bin.tar.gz
mv apache-hive-2.1.1-bin hive2. Add the environment variable in the / etc / profile plus
export HIVE_HOME=/root/hive
export HIVE_CONF_DIR=/root/hive/conf
PATH=$PATH:$HIVE_HOME/bin
#保存退出后，执行 source /etc/profile 让环境变量生效3. Start Hadoop
start-all.sh4. Using Hadoop to create the necessary directories (configuration files to use these directories), and give permission
hadoop fs -mkdir -p /user/hive/warehouse
hadoop fs -chmod -R 777 /user/hive/warehouse
hadoop fs -mkdir -p /tmp/hive
hadoop fs -chmod -R 777 /tmp/hive
cd /root/hive
mkdir tmp
chmod 777 ./tmp5. Modify hive-env.sh
cp hive-env.sh.template hive-env.sh
vi hive-env.sh
#将下面三行写入这个配置文件，注意用自己的路径
HADOOP_HOME=/root/hadoop-2.8.0
export HIVE_CONF_DIR=/root/hive/conf
export HIVE_AUX_JARS_PATH=/root/hive/lib6. Modify hive-site.xml
cd /root/hive/conf
cp hive-default.xml.template hive-site.xml
#下面两个命令把配置文件中的两个变量进行全部替换
#这个/root/hive/tmp就是之前创建的
#这个root是我的用户名
sed -i ‘s#${system:java.io.tmpdir}#/root/hive/tmp#g’ hive-site.xml
sed -i ‘s#${system:user.name}#root#g’ hive-site.xml7. Modify hive-site.xml configuration about the mysql
将javax.jdo.option.ConnectionDriverName的值改成mysql驱动：
com.mysql.jdbc.Drive
将javax.jdo.option.ConnectionURL的值改成mysql连接路径：
jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true
将javax.jdo.option.ConnectionUserName的值改成mysql用户名：root
将javax.jdo.option.ConnectionPassword的值改成mysql密码：mysql1234568. The driver package download mysql
cd /root/hive/lib
wget http://central.maven.org/maven2/mysql/mysql-connector-java/5.1.38/mysql-connector-java-5.1.38.jar9. initialize mysql
cd /root/hive/bin
./schematool -initSchema -dbType mysql10. Run hiveto see if the installation was successful
So far, hive building is completed, the code has been experimental, ease of use, but still we need to make the appropriate changes according to the situation
Novice, errors are inevitable, I implore you to a lot of corrections