Flink tutorial (16) ReducingState use case of Keyed State state management to find the maximum value

Series of articles

Flink tutorial (13) Keyed State state management ValueState using temperature difference alarm
Flink tutorial (14) Keyed State state management MapState use case
Flink tutorial (15) Keyed State state management ListState using ValueState to implement
Flink tutorial (16) Keyed State state The ReducingState use case of management to find the maximum value
Flink tutorial (17) The AggregatingState use case of Keyed State state management to find the average value

One, the method of ReducingState

• ReducingState is used in conjunction with ReduceFunction
• get() Get the value of the state
• The add(IN value) method adds an element and triggers the reduceFunction to calculate once
  

Second, the Descriptor of ReducingState

The ReducingState descriptor is different from the previous ValueState and ListState, it has to be used in conjunction with a ReduceFunction.

So you have to implement a ReduceFunction first, such as the one below to calculate the highest temperature.

public static class MyMaxTemp implements ReduceFunction<SensorRecord> {
    
    

    @Override
    public SensorRecord reduce(SensorRecord value1, SensorRecord value2) throws Exception {
    
    
        return value1.getRecord() >= value2.getRecord() ? value1 : value2;
    }
}

The second parameter of ReducingStateDescriptor is passed new SensorRecordUtils.MyMaxTemp().
In the future, every time a piece of data comes in, the reduce() method of this MyMaxTemp will be executed.

//用ReducingStateDescriptor定义描述器
    ReducingStateDescriptor reducingStateDescriptor = new ReducingStateDescriptor(
            "max-temp-state",//id
            new SensorRecordUtils.MyMaxTemp(),//ReduceFunction
            SensorRecord.class);//状态里的值的类型

//获取ReducingState
reducingState = getRuntimeContext().getReducingState(reducingStateDescriptor);

Three, the main body of the program

public class Test05_ReduceState {
    
    

    public static void main(String[] args) throws Exception {
    
    

        StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();

        //方便测试，设置为1
        env.setParallelism(1);

        DataStreamSource<String> source = env.socketTextStream(BaseConstant.URL, BaseConstant.PORT);

        /*
        设置watermark和指定时间属性
         */
        SingleOutputStreamOperator<SensorRecord> dataStream = source
                .map(new SensorRecordUtils.BeanMap());

        dataStream
                .keyBy(SensorRecord::getId)
                .process(new MyKeyedProcessFunction())
                .print();

        env.execute();
    }
}

Four, KeyedProcessFunction processing class

public static class MyKeyedProcessFunction extends KeyedProcessFunction<String, SensorRecord, SensorRecord> {
    
    

    private transient ReducingState reducingState;

    @Override
    public void open(Configuration parameters) throws Exception {
    
    
        super.open(parameters);

        //用ReducingStateDescriptor定义描述器
        ReducingStateDescriptor reducingStateDescriptor = new ReducingStateDescriptor(
                "max-temp-state",//id
                new SensorRecordUtils.MyMaxTemp(),//ReduceFunction
                SensorRecord.class);//状态里的值的类型

        //获取ReducingState
        reducingState = getRuntimeContext().getReducingState(reducingStateDescriptor);
    }

    @Override
    public void processElement(SensorRecord value, Context ctx, Collector<SensorRecord> out) throws Exception {
    
    

        reducingState.add(value);

        out.collect((SensorRecord) reducingState.get());
    }
}

operation result: