hive on spark 与 hive on tez共存,tez-ui配置

1.设置tez-ui

需要先安装Tomcat
1.找到war包
因为我安装的是bin所以在安装包中就有tez-ui-0.9.0.war
自己编译的话也有生成,在这里不做过多赘述。

2.将tez-ui部署在tomcat

#在webapps下创建tez-ui目录
mkdir /usr/local/apache/apache-tomcat-8.5.31/webapps/tez-ui

#进入文件
cd  /usr/local/apache/apache-tomcat-8.5.31/webapps/tez-ui

#解压war包
unzip tez-ui-0.9.0.war

#编辑配置文件
vim config/configs.env

在这里插入图片描述

3.配置timelineserver

目录:/usr/local/hadoop/etc/hadoop

vim yarn-site.xml添加以下内容.然后分发到各个节点。

<!-- conf timeline server -->
   <property>
        <name>yarn.timeline-service.enabled</name>
        <value>true</value>
   </property>
   <property>
        <name>yarn.timeline-service.hostname</name>
        <value>tmaster</value>
   </property>
   <property>
        <name>yarn.timeline-service.http-cross-origin.enabled</name>
        <value>true</value>
   </property>
   <property>
        <name> yarn.resourcemanager.system-metrics-publisher.enabled</name>
        <value>true</value>
   </property>
   <property>
        <name>yarn.timeline-service.generic-application-history.enabled</name>
        <value>true</value>
   </property>
   <property>
        <description>Address for the Timeline server to start the RPC server.</description>
        <name>yarn.timeline-service.address</name>
        <value>tmaster:10201</value>
   </property>
   <property>
        <description>The http address of the Timeline service web application.</description>
        <name>yarn.timeline-service.webapp.address</name>
        <value>tmaster:8188</value>
   </property>
   <property>
        <description>The https address of the Timeline service web application.</description>
        <name>yarn.timeline-service.webapp.https.address</name>
        <value>tmaster:2191</value>
   </property>
   <property>
        <name>yarn.timeline-service.handler-thread-count</name>
        <value>24</value>
   </property>

vim tez-site.xml添加下列几行,分发到各个节点。

<!--Configuring Tez to use YARN Timeline-->
    <property>
        <name>tez.history.logging.service.class</name>
        <value>org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService</value>
    </property>
    <property>
        <name>tez.tez-ui.history-url.base</name>
        <value>http://192.168.205.70:8880/tez-ui/</value>
    </property>

4.启动hadoop

5.启动timelineserver

yarn-daemon.sh start timelineserver

6.启动tomcat

/usr/local/apache/apache-tomcat-8.5.31/bin/startup.sh

访问界面
http://192.168.205.70:8880/tez-ui/

在这里插入图片描述

遇到问题:配置好后 hive on spark出现了问题,用下面命令启动

hive --hiveconf hive.root.logger=warn,console

报出找不到类的异常:
NoClassDefFoundError: org/codehaus/jackson/Versioned和 org/codehaus/jackson/map/ObjectMapper
去阿里云把 jackson-mapper-asl-1.9.13.jar 和jackson-core-asl-1.9.13.jar 下载下来,传到 hive ,spark ,以及hive 里面的spark的jars 里面,就解决了。

启动命令:

sh /usr/local/service/hadoop/sbin/yarn-daemon.sh start timelineserver
sh /usr/local/service/apache-tomcat-8.5.63/bin/startup.sh

停止命令:

sh /usr/local/service/hadoop/sbin/yarn-daemon.sh stop timelineserver
sh /usr/local/service/apache-tomcat-8.5.63/bin/shutdown.sh

猜你喜欢

转载自blog.csdn.net/qq_27474277/article/details/114526178
今日推荐