Flink随笔一 书写wordCount

版本：Flink：1.7.0

编码语言：java \ scala

java:1.8

scala:2.11.7

需要配置mavn：<flink.version>1.7.0</flink.version>

<!--使用java语言编写的时候用到的-->
<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-java</artifactId>
    <version>${flink.version}</version>
</dependency>

<!--使用scala语言编写的时候用到的-->
<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-scala_2.11</artifactId>
    <version>${flink.version}</version>
</dependency>

import org.apache.flink.api.scala._

object WordCountTest {

  def main(args: Array[String]): Unit = {

    val env = ExecutionEnvironment.getExecutionEnvironment
    val text = env.fromElements(
      "Who's there?",
      "I think I hear them. Stand, ho! Who's there?")

    val counts = text.flatMap {_.toLowerCase.split("\\W+") filter {_.nonEmpty}}
      .map {(_, 1)}
      .groupBy(0)
      .sum(1)

    counts.print()
    //env.execute()
  }
}

  运行最终结果

mavn配置：

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.sns.data.flink.test</groupId>
    <artifactId>flink_test</artifactId>
    <version>1.0-SNAPSHOT</version>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>6</source>
                    <target>6</target>
                </configuration>
            </plugin>
        </plugins>
    </build>

    <properties>
        <flink.version>1.7.0</flink.version>
        <hadoop.version>2.6.0</hadoop.version>
        <log4j.version>1.2.15</log4j.version>
    </properties>

    <dependencies>

        <!--使用java语言编写的时候用到的-->
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-java</artifactId>
            <version>${flink.version}</version>
        </dependency>

        <!--使用scala语言编写的时候用到的-->
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-scala_2.11</artifactId>
            <version>${flink.version}</version>
        </dependency>

        <!--使用flink需要引入的jar包-->
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-streaming-java_2.11</artifactId>
            <version>${flink.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-clients_2.11</artifactId>
            <version>${flink.version}</version>
        </dependency>

        <!--<dependency>-->
            <!--<groupId>org.apache.flink</groupId>-->
            <!--<artifactId>flink-connector-wikiedits_2.11</artifactId>-->
            <!--<version>${flink.version}</version>-->
        <!--</dependency>-->

        <!--flink和kafka相连接 -->
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-connector-kafka-0.8_2.11</artifactId>
            <version>${flink.version}</version>
        </dependency>

        <dependency>
            <groupId>log4j</groupId>
            <artifactId>log4j</artifactId>
            <version>${log4j.version}</version>
        </dependency>

    </dependencies>

</project>

运行过程中出现的问题：

一、

1）.代码中需要加上import org.apache.flink.api.scala._

2）.使用下面mavn配置

最早使用的mavn报错(scala版本不一致也会导致)

二、

错误导致原因：缺少log4jjar包

添加mavn依赖：<log4j.version>1.2.15</log4j.version>

  <dependency>
       <groupId>log4j</groupId>
       <artifactId>log4j</artifactId>
       <version>${log4j.version}</version>
  </dependency>