Storm运行模式:
- 本地模式(Local Mode): 即Topology(相当于一个任务,后续会详细讲解) 运行在本地机器的单一JVM上,这个模式主要用来开发、调试。
- 远程模式(Remote Mode):在这个模式,我们把我们的Topology提交到集群,在这个模式中,Storm的所有组件都是线程安全的,因为它们都会运行在不同的Jvm或物理机器上,这个模式就是正式的生产模式。
本地模式:
LocalCluster cluster = new LocalCluster(); cluster.submitTopology("Getting-Started-Toplogie", conf, builder.createTopology()); Thread.sleep(1000); cluster.shutdown();
远程模式:
StormSubmitter.submitTopology("Count-Word-Topology-With-Refresh-Cache", conf, builder.createTopology());
执行命令:
storm jar storm.jar storm.starter.WordCountTopology WordCount -c nimbus.host=localhost
如果job中需要引用三方jar:
1.将第三方jar放到storm lib 目录
2.通过maven打包,将第三方jar解压打包到一个jar中。 (不解压不行)
pom
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.cyyun</groupId> <artifactId>stormtest</artifactId> <version>0.0.1-SNAPSHOT</version> <profiles> <profile> <id>test</id> <properties> <package.finalName>stormtest</package.finalName> </properties> <activation> <activeByDefault>true</activeByDefault> </activation> </profile> </profiles> <repositories> <!-- Repository where we can found the storm dependencies --> <repository> <id>clojars.org</id> <url>http://clojars.org/repo</url> </repository> </repositories> <dependencies> <!-- <dependency> <groupId>org.apache.storm</groupId> <artifactId>storm-core</artifactId> <version>0.9.1-incubating</version> </dependency> --> <!--md5--> <dependency> <groupId>fast-md5</groupId> <artifactId>fast-md5</artifactId> <version>2.5</version> </dependency> </dependencies> <build> <plugins> <plugin> <artifactId>maven-assembly-plugin</artifactId> <version>2.4</version> <configuration> <appendAssemblyId>false</appendAssemblyId> <finalName>${package.finalName}</finalName> <descriptorRefs> <descriptorRef>jar-with-dependencies</descriptorRef> </descriptorRefs> <archive> <manifest> <mainClass>${package.mainClass}</mainClass> </manifest> </archive> </configuration> <executions> <execution> <id>make-assembly</id> <phase>package</phase> <goals> <goal>assembly</goal> </goals> </execution> </executions> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <version>2.3.2</version> <configuration> <encoding>UTF-8</encoding> <compilerArguments> <verbose /> <bootclasspath>${java.home}/lib/rt.jar;${java.home}/lib/jce.jar</bootclasspath> </compilerArguments> </configuration> </plugin> </plugins> <resources> <resource> <directory>${project.basedir}/src/main/resources</directory> <filtering>true</filtering> <excludes> <exclude>com/**/*.*</exclude> </excludes> </resource> <resource> <directory>${project.basedir}/src/main/resources</directory> <filtering>false</filtering> <includes> <include>com/**/*.*</include> </includes> </resource> </resources> </build> </project>