使用idea 创建flink项目并测试WindowWordCount

原创已于 2025-09-12 16:00:41 修改 · 210 阅读

1 ·

本内容遵循CC 4.0 BY-SA版权协议

标签

#intellij-idea #flink #java

于 2023-10-20 14:29:40 首次发布

flink 专栏收录该内容

2 篇文章

订阅专栏

使用idea 创建flink项目并测试WindowWordCount

环境

flink 1.17.1

jdk：1.8

scala 2.13.11

创建项目

使用idea来直接创建项目如下：

设置Archetype 内容如上

org.apache.flink

flink-quickstart-java

version 为flink的版本

配置pom

基本设置

<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-streaming-java</artifactId>
    <version>${flink.version}</version>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-clients</artifactId>
    <version>${flink.version}</version>
    <scope>provided</scope>
</dependency>

flink核心设置

<!-- flink核心API -->
<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-java</artifactId>
    <version>${flink.version}</version>
</dependency>
<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-table-api-scala-bridge_${scala.binary.version}</artifactId>
    <version>${flink.version}</version>
</dependency>

<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-table-planner_${scala.binary.version}</artifactId>
    <version>${flink.version}</version>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-json</artifactId>
    <version>${flink.version}</version>
</dependency>
<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-scala_${scala.binary.version}</artifactId>
    <version>${flink.version}</version>
</dependency>
<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-table-api-scala_${scala.binary.version}</artifactId>
    <version>${flink.version}</version>
</dependency>

测试使用flnk开发一个 WindowWordCount 代码如下

package org.example;
import org.apache.flink.api.common.functions.FlatMapFunction;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.streaming.api.datastream.DataStream;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.windowing.assigners.TumblingEventTimeWindows;
import org.apache.flink.streaming.api.windowing.assigners.TumblingProcessingTimeWindows;
import org.apache.flink.streaming.api.windowing.time.Time;
import org.apache.flink.util.Collector;
/**
nc -lk 9999
ceshi nhiaho  zhong
ceshi nhiaho  zhong
cehi  zhang san
cehi  zhang san
**/
public class WindowWordCount {
    public static void main(String[] args) throws  Exception{
        StreamExecutionEnvironment env =StreamExecutionEnvironment.getExecutionEnvironment();
        DataStream<Tuple2<String,Integer>> dataStream=env
                .socketTextStream("172.31.7.10",9999)
                .flatMap(new Splitter())
                .keyBy(v->v.f0)
                .window(TumblingProcessingTimeWindows.of(Time.seconds(5)))
                .sum(1);
        dataStream.print();
        env.execute("Window WordNount");

    }
    public static class Splitter implements FlatMapFunction<String,Tuple2<String,Integer>>
    {

        @Override
        public void flatMap(String s, Collector<Tuple2<String, Integer>> outcol) throws Exception {
            for(String word:s.split(" "))
            {
                outcol.collect(new Tuple2<String,Integer>(word,1));
            }

        }
    }
}

在socket 机器上输入