ListState ValueState implementation of Keyed State state management
Series of articles
Flink tutorial (13) Keyed State state management ValueState using temperature difference alarm
Flink tutorial (14) Keyed State state management MapState use case
Flink tutorial (15) Keyed State state management ListState using ValueState to implement
Flink tutorial (16) Keyed State state The ReducingState use case of management to find the maximum value
Flink tutorial (17) The AggregatingState use case of Keyed State state management to find the average value
One, ListState method
- get() method to get the value
- add(IN value), addAll(List values) method to update the value
- update(List values) replace the original List with the new List
- clear() Clear the List, the List still exists, but there are no elements
Two, code case
1. Demand
Record each person’s historical operation information, such as input
1,login
1,buy something
1,logout
2,login
2,click book
1,login
To output
(1,[login])
(1,[login, buy something])
(1,[login, buy something, logout])
(2,[login])
(2,[login, click book])
(1,[login, buy something, logout, login])
2. Subject
public class Test03_ListState {
public static void main(String[] args) throws Exception {
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.setParallelism(1);
DataStreamSource<String> source = env.socketTextStream(BaseConstant.URL, BaseConstant.PORT);
SingleOutputStreamOperator<Tuple2<String, String>> dataStream = source
.map(new MapFunction<String, Tuple2<String, String>>() {
@Override
public Tuple2<String, String> map(String value) throws Exception {
String[] split = value.split(",");
if (split != null && split.length == 2) {
return Tuple2.of(split[0], split[1]);
}
return null;
}
});
dataStream
.keyBy(t -> t.f0)
.process(new MyKeyedProcessFunction())
.print();
env.execute();
}
}
3. Processing method
public static class MyKeyedProcessFunction extends KeyedProcessFunction<String, Tuple2<String, String>, Tuple2<String, List<String>>> {
//之前的操作记录
private transient ListState<String> listState;
@Override
public void open(Configuration parameters) throws Exception {
super.open(parameters);
ListStateDescriptor<String> recentOperatorsDescriptor = new ListStateDescriptor<String>(
"recent-operator",
String.class);
listState = getRuntimeContext().getListState(recentOperatorsDescriptor);
}
@Override
public void processElement(Tuple2<String, String> value, Context ctx, Collector<Tuple2<String, List<String>>> out) throws Exception {
String action = value.f1;
listState.add(action);
Iterable<String> iterable = listState.get();
ArrayList<String> events = new ArrayList<>();
for (String actionName : iterable) {
events.add(actionName);
}
out.collect(Tuple2.of(value.f0, events));
listState.update(events);
}
}
4. Run results
5. Implement ListState with ValueState
Using ValueState to implement ListState can implement ListState, as long as ValueState is loaded with List
public static class MyKeyedProcessFunction extends KeyedProcessFunction<String, Tuple2<String, String>, Tuple2<String, List<String>>> {
//之前的操作记录
private transient ValueState<List<String>> listState;
@Override
public void open(Configuration parameters) throws Exception {
super.open(parameters);
ValueStateDescriptor<List<String>> recentOperatorsDescriptor = new ValueStateDescriptor<List<String>>(
"recent-operator",
TypeInformation.of(new TypeHint<List<String>>(){
}));
listState = getRuntimeContext().getState(recentOperatorsDescriptor);
}
@Override
public void processElement(Tuple2<String, String> value, Context ctx, Collector<Tuple2<String, List<String>>> out) throws Exception {
String action = value.f1;
List<String> lst = listState.value();
if (lst == null){
lst = new ArrayList<>();
}
lst.add(action);
listState.update(lst);
out.collect(Tuple2.of(value.f0, listState.value()));
}
}