Win10+hadoop2.9.1+hive2.2.0 test environment deployment

After deploying and installing hadoop 2.9.1, download hive 2.2.0 and extract it to the installation directory. This article is E:\post\hive\apache-hive-2.3.3-bin

    PS: After downloading, it is found that there is no win-related cmd command after 2.2.0

    PS: The configuration of hadoop is relatively simple, so I won't go into details in this article

• Environment variable

    A total of four environment variables need to be configured (system variables, if you do not configure these four, you will not be able to run hive , which has been pitted here for a long time)

1. HIVE_BIN_PATH：E:\post\hive\apache-hive-2.2.0-bin\bin
2. HIVE_HOME：E:\post\hive\apache-hive-2.2.0-bin
3. HIVE_LIB：E:\post\hive\apache-hive-2.2.0-bin\lib
4. PATH increase: %HIVE_HOME%\bin

• Configuration file

1. Create a new hive_site.xml file in the conf directory of Hive with the following content (to avoid mining pits, post the content of the completed xml file, here is also a pit):

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
    <property>
    <name>javax.jdo.option.ConnectionURL</name>
        <value>jdbc:mysql://IP:3306/hive_metadata?createDatabaseIfNotExist=true</value>
        </property>
    <property>
        <name>javax.jdo.option.ConnectionDriverName</name>
        <value>com.mysql.jdbc.Driver</value>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionUserName</name>
        <value>root</value>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionPassword</name>
        <value>123456</value>
    </property>   
    <property>
        <name>datanucleus.fixedDatastore</name>
        <value>false</value>
    </property>
    <property>
      <name>datanucleus.autoCreateSchema</name>
      <value>true</value>
    </property>
    <property>
      <name>datanucleus.autoCreateTables</name>
      <value>true</value>
    </property>
    <property>
      <name>datanucleus.autoCreateColumns</name>
      <value>true</value>
    </property>
    <property>
        <name>hive.metastore.schema.verification</name>
        <value>false</value>
        <description>
          Enforce metastore schema version consistency.
          True: Verify that version information stored in metastore matches with one from Hive jars.  Also disable automatic
          schema migration attempt. Users are required to manully migrate schema after Hive upgrade which ensures proper metastore schema migration. (Default)
          False: Warn if the version information stored in metastore doesn't match with one from in Hive jars.
         </description>
    </property>
</configuration>

 

      PS: The red boldface needs to be changed to be consistent with my actual situation ; in addition, if other errors are reported, add the corresponding property by Google.

2. A new hive-env.sh file is added under the conf directory of Hive, the content is as follows:

        export HADOOP_HOME=E:\post\hadoop\hadoop-2.9.1

        export HIVE_CONF_DIR=E:\post\hadoop\hadoop-2.9.1\bin\conf

        export HIVE_AUX_JARS_PATH=E:\post\hadoop\hadoop-2.9.1\bin\lib

       PS: For the corresponding hadoop related environment, you need to modify it according to your actual situation

3. Add the mysql-connector-java.jar driver to the lib directory of Hive

  Official website download link: https://dev.mysql.com/downloads/file/?id=476198

After decompression, rename mysql-connector-java-5.1.46.jar to mysql-connector-java.jar and add it to the lib directory of hive. (If the link fails, you can search and download it on Google)

• test

1. Win+R open cmd, enter hive, and successfully enter hive:

               PS: You don't need to run the following command to initialize, I have been pitted by this for a long time

               schematool -initSchema -dbType mysql

               hive --service metastore

        

    2. View hive's mysql meta database table: