Storm 集群的入门安装

1、前提

Storm 安装前,确保 zookeeper 的单机或集群版已经配置好,并启动成功。

2、集群说明

zookeeper :

IP地址 名称 说明
192.168.1.102 zookeeper 单机版的(如果是集群版的会更好)

storm :

IP地址 名称 说明
192.168.1.104 nimbus、ui 的服务 主节点
192.168.1.105 supervisor 工作节点1
192.168.1.106 supervisor 工作节点2

3、安装步骤

3.1、下载 storm

清华大学镜像地址: https://mirrors.cnnic.cn/apache/storm/

我下载的是 apache-storm-0.9.2-incubating.tar.gz

3.2、上传、解压(3台)

上传 apache-storm-0.9.2-incubating.tar.gz 到 centos7 的 /mysoft

解压命令:

tar -zxvf  apache-storm-0.9.2-incubating.tar.gz

重命名为 /apache-storm-0.9.2 ,移动 /usr/local/ 目录下:

mv apache-storm-0.9.2-incubating /usr/local/apache-storm-0.9.2

3.3、创建 data 目录(3台)

此目录用于存放 storm 的数据

cd  /usr/local/apache-storm-0.9.2/

/usr/local/apache-storm-0.9.2/data

3.4、配置 /etc/profile(3台)

将 storm_home 配置到 /etc/profile 上。

vi  /etc/profile

内容:

STORM_HOME=/usr/local/apache-storm-0.9.2/

PATH=$PATH:$STORM_HOME/bin

export PATH STORM_HOME

重新加载 profile:

source /etc/profile

3.5、配置 storm(3台)

编辑 conf 目录下的 storm.yaml 配置文件:

vi  /usr/local/apache-storm-0.9.2/conf/storm.yaml 

添加 或修改内容如下:

 storm.zookeeper.servers:
     - "192.168.1.102"


 nimbus.host: "192.168.1.104"
 storm.local.dir: "/usr/local/apache-storm-0.9.2/data"
 ui.port: 18080
 supervisor.slots.ports:
     - 6700
     - 6701
     - 6702
     - 6703

说明 :

(3台) 是指此步骤的操作,在3台服务器上完全相同。

3.6、启动

3.6.1、在 192.168.1.104 启动 nimbus

由于 storm 已经在 /etc/profile 中配置全局命令,所以 启动命令 可以在任何目录下操作

启动 nimbus :

storm nimbus  &

也可以使用下面的命令:
nohup bin/storm nimbus >/dev/null 2>&1 & 

3.6.2、在 192.168.1.105、106 启动 supervisor

启动 nimbus :

storm supervisor  &

3.7、检查是否启动成功

192.168.1.104 查看 nimbus 的启动状态 :

[root@rocketmq-nameserver1 conf]# jps
5425 Jps
5252 nimbus

192.168.1.105、106 上 查看 supervisor 的启动状态:

[root@rocketmq-nameserver3 logs]# jps
4677 supervisor
6795 Jps

3.8、查看日志,检查是否启动成功:

192.168.1.104 查看 nimbus 的日志 :

进入 logs 目录:

[root@rocketmq-nameserver1 apache-storm-0.9.2]# cd logs/
[root@rocketmq-nameserver1 logs]# ls
access.log  metrics.log  nimbus.log  ui.log

查看 nimbus.log 日志(192.168.1.104 启动的是 nimbus )

[root@rocketmq-nameserver1 logs]# tail -300f nimbus.log 
2019-05-19 19:07:28 o.a.z.ZooKeeper [INFO] Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
2019-05-19 19:07:28 o.a.z.ZooKeeper [INFO] Client environment:host.name=rocketmq-nameserver1
2019-05-19 19:07:28 o.a.z.ZooKeeper [INFO] Client environment:java.version=1.8.0_201
2019-05-19 19:07:28 o.a.z.ZooKeeper [INFO] Client environment:java.vendor=Oracle Corporation
2019-05-19 19:07:28 o.a.z.ZooKeeper [INFO] Client environment:java.home=/mysoft/jdk1.8.0_201/jre

.....(省略)

2019-05-19 19:07:29 o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=192.168.1.102:2181 sessionTimeout=20000 watcher=org.apache.curator.ConnectionState@400d912a
2019-05-19 19:07:29 o.a.z.ClientCnxn [INFO] Opening socket connection to server 192.168.1.102/192.168.1.102:2181. Will not attempt to authenticate using SASL (unknown error)
2019-05-19 19:07:30 o.a.z.ClientCnxn [INFO] Socket connection established to 192.168.1.102/192.168.1.102:2181, initiating session
2019-05-19 19:07:30 o.a.z.ClientCnxn [INFO] Session establishment complete on server 192.168.1.102/192.168.1.102:2181, sessionid = 0x16acfa5816d0000, negotiated timeout = 20000
2019-05-19 19:07:30 o.a.c.f.s.ConnectionStateManager [INFO] State change: CONNECTED
2019-05-19 19:07:30 o.a.c.f.s.ConnectionStateManager [WARN] There are no ConnectionStateListeners registered.
2019-05-19 19:07:30 b.s.zookeeper [INFO] Zookeeper state update: :connected:none
2019-05-19 19:07:31 o.a.z.ZooKeeper [INFO] Session: 0x16acfa5816d0000 closed
2019-05-19 19:07:31 o.a.z.ClientCnxn [INFO] EventThread shut down
2019-05-19 19:07:31 o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
2019-05-19 19:07:31 o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=192.168.1.102:2181/storm sessionTimeout=20000 watcher=org.apache.curator.ConnectionState@6ac97b84
2019-05-19 19:07:31 o.a.z.ClientCnxn [INFO] Opening socket connection to server 192.168.1.102/192.168.1.102:2181. Will not attempt to authenticate using SASL (unknown error)
2019-05-19 19:07:31 o.a.z.ClientCnxn [INFO] Socket connection established to 192.168.1.102/192.168.1.102:2181, initiating session
2019-05-19 19:07:31 o.a.z.ClientCnxn [INFO] Session establishment complete on server 192.168.1.102/192.168.1.102:2181, sessionid = 0x16acfa5816d0001, negotiated timeout = 20000
2019-05-19 19:07:31 o.a.c.f.s.ConnectionStateManager [INFO] State change: CONNECTED
2019-05-19 19:07:31 o.a.c.f.s.ConnectionStateManager [WARN] There are no ConnectionStateListeners registered.
2019-05-19 19:07:32 b.s.d.nimbus [INFO] Starting Nimbus server...

192.168.1.105、106 查看 nimbus 的日志 :

[root@rocketmq-nameserver2 apache-storm-0.9.2]# cd logs/
[root@rocketmq-nameserver2 logs]# ls
access.log  metrics.log  supervisor.log

查看 supervisor.log 日志:(192.168.1.105 、106 启动的是 supervisor )

[root@rocketmq-nameserver3 logs]# tail -300f supervisor.log 
2019-05-19 19:08:38 o.a.z.ZooKeeper [INFO] Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
2019-05-19 19:08:38 o.a.z.ZooKeeper [INFO] Client environment:host.name=rocketmq-nameserver3
2019-05-19 19:08:38 o.a.z.ZooKeeper [INFO] Client environment:java.version=1.8.0_201
2019-05-19 19:08:38 o.a.z.ZooKeeper [INFO] Client environment:java.vendor=Oracle Corporation
2019-05-19 19:08:38 o.a.z.ZooKeeper [INFO] Client environment:java.home=/mysoft/jdk1.8.0_201/jre

.....(省略)

2019-05-19 19:08:39 o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=192.168.1.102:2181 sessionTimeout=20000 watcher=org.apache.curator.ConnectionState@6579c3d9
2019-05-19 19:08:39 o.a.z.ClientCnxn [INFO] Opening socket connection to server 192.168.1.102/192.168.1.102:2181. Will not attempt to authenticate using SASL (unknown error)
2019-05-19 19:08:39 o.a.z.ClientCnxn [INFO] Socket connection established to 192.168.1.102/192.168.1.102:2181, initiating session
2019-05-19 19:08:40 o.a.z.ClientCnxn [INFO] Session establishment complete on server 192.168.1.102/192.168.1.102:2181, sessionid = 0x16acfa5816d0002, negotiated timeout = 20000
2019-05-19 19:08:40 o.a.c.f.s.ConnectionStateManager [INFO] State change: CONNECTED
2019-05-19 19:08:40 o.a.c.f.s.ConnectionStateManager [WARN] There are no ConnectionStateListeners registered.
2019-05-19 19:08:40 b.s.zookeeper [INFO] Zookeeper state update: :connected:none
2019-05-19 19:08:41 o.a.z.ZooKeeper [INFO] Session: 0x16acfa5816d0002 closed
2019-05-19 19:08:41 o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
2019-05-19 19:08:41 o.a.z.ClientCnxn [INFO] EventThread shut down
2019-05-19 19:08:41 o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=192.168.1.102:2181/storm sessionTimeout=20000 watcher=org.apache.curator.ConnectionState@59712875
2019-05-19 19:08:41 o.a.z.ClientCnxn [INFO] Opening socket connection to server 192.168.1.102/192.168.1.102:2181. Will not attempt to authenticate using SASL (unknown error)
2019-05-19 19:08:41 o.a.z.ClientCnxn [INFO] Socket connection established to 192.168.1.102/192.168.1.102:2181, initiating session
2019-05-19 19:08:41 o.a.z.ClientCnxn [INFO] Session establishment complete on server 192.168.1.102/192.168.1.102:2181, sessionid = 0x16acfa5816d0003, negotiated timeout = 20000
2019-05-19 19:08:41 o.a.c.f.s.ConnectionStateManager [INFO] State change: CONNECTED
2019-05-19 19:08:41 o.a.c.f.s.ConnectionStateManager [WARN] There are no ConnectionStateListeners registered.
2019-05-19 19:08:41 b.s.d.supervisor [INFO] Starting supervisor with id 0ee53c84-1eb4-4a4d-b9be-f7f72ba8e9b2 at host rocketmq-nameserver2

3.9、storm UI

如果前面的步骤检查没有问题了,启动 storm ui

3.9.1、启动 storm ui

storm ui  &

也可以使用下面的命令:
nohup bin/storm supervisor >/dev/null 2>&1 &

3.9.2、查看 storm ui 的日志

[root@rocketmq-nameserver1 apache-storm-0.9.2]# cd logs/
[root@rocketmq-nameserver1 logs]# ls
access.log  metrics.log  nimbus.log  ui.log

查看日志:

[root@rocketmq-nameserver1 logs]# tail -500f ui.log 
2019-05-19 19:18:26 o.m.log [INFO] Logging to Logger[org.mortbay.log] via org.mortbay.log.Slf4jLog
2019-05-19 19:18:26 o.m.log [INFO] jetty-6.1.26
2019-05-19 19:18:26 o.m.log [INFO] Started [email protected]:18080
2019-05-19 19:18:28 b.s.thrift [INFO] Connecting to Nimbus at 192.168.1.104:6627
2019-05-19 19:18:28 b.s.thrift [INFO] Connecting to Nimbus at 192.168.1.104:6627

从日志中可以看出 UI 的端口号是 18080

3.9.3、登录 storm ui

地址:http://192.168.1.104:18080

在这里插入图片描述

猜你喜欢

转载自blog.csdn.net/xiaojin21cen/article/details/90349111