IDEA 导入Hive源码并编译

编译源码

• 下载Hive源码:http://archive.cloudera.com/cdh5/cdh/5/hive-1.1.0-cdh5.16.2-src.tar.gz
• 编译Hive源码(切记不要idea里面执行命令)：mvn clean package -DskipTests=true -Phadoop-2

mvn clean package -DskipTests=true -Phadoop-2

[INFO] Reactor Summary:
[INFO]
[INFO] Hive 1.1.0-cdh5.16.2 ............................... SUCCESS [  3.119 s]
[INFO] Hive Classifications ............................... SUCCESS [  2.406 s]
[INFO] Hive Shims Common .................................. SUCCESS [  3.327 s]
[INFO] Hive Shims 0.23 .................................... SUCCESS [  3.494 s]
[INFO] Hive Shims Scheduler ............................... SUCCESS [  2.423 s]
[INFO] Hive Shims ......................................... SUCCESS [  1.463 s]
[INFO] Hive Common ........................................ SUCCESS [  8.382 s]
[INFO] Hive Serde ......................................... SUCCESS [  8.001 s]
[INFO] Hive Metastore ..................................... SUCCESS [ 28.285 s]
[INFO] Hive Ant Utilities ................................. SUCCESS [  1.668 s]
[INFO] Spark Remote Client ................................ SUCCESS [  4.915 s]
[INFO] Hive Query Language ................................ SUCCESS [01:36 min]
[INFO] Hive Service ....................................... SUCCESS [ 22.921 s]
[INFO] Hive Accumulo Handler .............................. SUCCESS [  5.496 s]
[INFO] Hive JDBC .......................................... SUCCESS [  5.797 s]
[INFO] Hive Beeline ....................................... SUCCESS [  3.957 s]
[INFO] Hive CLI ........................................... SUCCESS [  4.060 s]
[INFO] Hive Contrib ....................................... SUCCESS [  4.321 s]
[INFO] Hive HBase Handler ................................. SUCCESS [  5.518 s]
[INFO] Hive HCatalog ...................................... SUCCESS [  1.399 s]
[INFO] Hive HCatalog Core ................................. SUCCESS [  5.933 s]
[INFO] Hive HCatalog Pig Adapter .......................... SUCCESS [  4.632 s]
[INFO] Hive HCatalog Server Extensions .................... SUCCESS [  4.477 s]
[INFO] Hive HCatalog Webhcat Java Client .................. SUCCESS [  4.903 s]
[INFO] Hive HCatalog Webhcat .............................. SUCCESS [  7.452 s]
[INFO] Hive HCatalog Streaming ............................ SUCCESS [  4.306 s]
[INFO] Hive HWI ........................................... SUCCESS [  3.461 s]
[INFO] Hive ODBC .......................................... SUCCESS [  3.061 s]
[INFO] Hive Shims Aggregator .............................. SUCCESS [  0.840 s]
[INFO] Hive TestUtils ..................................... SUCCESS [  1.077 s]
[INFO] Hive Packaging 1.1.0-cdh5.16.2 ..................... SUCCESS [  4.194 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 04:22 min
[INFO] Finished at: 2020-04-12T18:50:46+08:00
[INFO] ------------------------------------------------------------------------

3.IDEA导入源码
本地调试Hive

集群服务器启动$HIVE_HOME/bin/hive --service metastore -p 9083

分别把集群的配置文件加载到hive-cli的resources下

core-site.xml

<property>
        <name>fs.defaultFS</name>
        <value>hdfs://bigdata01:9000</value>
</property>

hdfs-site.xml

 <property>
        <name>dfs.client.use.datanode.hostname</name>
        <value>true</value>
 </property>

mapred-site.xml

mapred-site.xml
<property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
</property>

yarn-site.xml

**yarn-site.xml**
<property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
</property>

hive-site.xml

<property>
        <name>hive.cli.print.current.db</name>
        <value>true</value>
</property>

<property>
        <name>hive.cli.print.header</name>
        <value>true</value>
</property>

<property>
        <name>hive.metastore.uris</name>
        <value>thrift://bigdata01:9083</value>
</property>

在CliDriver类中加入vm参数:-Djline.WindowsTerminal.directConsole=false,Hive是默认Jline输入到控制太只支持Linux和mac,所以需要关闭

运行CliDriver 运行org.apache.hadoop.hive.cli.CliDriver 类

控制台输入

hive (default)> show databases;
show databases;
OK
database_name
company
default
skewtest
hive (default)> Time taken: 0.303 seconds, Fetched: 10 row(s)