Preface
I believe everyone is familiar with deduplication of simple list arrays, such as the following code
int[] arrays = {
1, 2, 2, 2, 3, 3, 3};
Arrays.stream(arrays).distinct().forEach(item -> {
System.out.println("item ->" + item);
});
Then let's discuss, for the array saved as an object in the list, what are the good ideas for deduplication based on a certain field of the internal object?
Given a simple Student object
public class Student {
String id;
String index;
String name;
}
针对该Student对象,以下是我想到的三种方法去重方法
Method 1: List.contains + Stream
/**
* @author: 代码丰
* @Date: 2022/10/24 13:57
* @Description:
*/
public class DeRepeatFromTwoListTest {
public static void main(String[] args) {
//测试1:去重两个列表的重复值 填充参数
List<Student2> list1 = new ArrayList<>();
List<Student2> list2 = new ArrayList<>();
for(int i = 1;i<=5;i++){
Student2 Student2 = new Student2();
Student2.setId(String.valueOf(i));
Student2.setIndex(String.valueOf(i));
Student2.setName("name"+String.valueOf(i));
list1.add(Student2);
}
for(int i = 1;i<=3;i++){
Student2 Student2 = new Student2();
Student2.setId(String.valueOf(i));
Student2.setIndex(String.valueOf(i));
Student2.setName("name"+String.valueOf(i));
list2.add(Student2);
}
// 基本思路:
// 1、将【数组流】转换为【字段流】
// 2、流重新恢复数组
// 3、然后再使用List.contains方法去过滤
List<Student2> resultList = list1.stream()
.filter(item -> !(list2.stream().map(e -> e.getId()).collect(Collectors.toList()).contains(item.getId()))
).collect(Collectors.toList());
System.out.println(resultList);
}
}
Method 2: Set non-repeatable characteristics + Stream flow
/**
* @author: 代码丰
* @Date: 2022/10/24 17:24
* @Description:
*/
public class DeRepeatFromThreeListTest {
public static void main(String[] args) {
//测试2:去重一个列表的重复值
List<Student> list = new ArrayList<>();
for (int i = 1; i <= 5; i++) {
Student student = new Student();
student.setId("1");
student.setIndex("1" + String.valueOf(i));
student.setName("name" + String.valueOf(i));
list.add(student);
}
//基本思路:利用set不可重复key特性
List<Student> after = list.stream().collect(Collectors.collectingAndThen(Collectors.toCollection(() -> new TreeSet<>(Comparator.comparing(Student::getId))), ArrayList::new));
}
}
Method 3: concurrentHashMap’s putIfAbsent + Stream
/**
* 去重工具类
* @author 代码丰
*/
@Slf4j
public class CustomizeDistinctUtil {
//基本思路
//1.利用 ConcurrentHashMap 的 putIfAbsent(假如map中key对应的value不存在,放value进入map 假如map中key对应的value存在,返回key对应的value)
//2. 构造 Predicate 返回值
// 不存在时,putIfAbsent 得到null,== null比较后 会返回true
//3. filter true的得到保留 false的直接过滤
//4. 效果为只有不存在的才会保留,存在的都得到了过滤,即实现去重
public static <T> Predicate<T> distinctByKey(Function<? super T, Object> keyExtractor) {
Map<Object, Boolean> seen = new ConcurrentHashMap<>();
return t -> seen.putIfAbsent(keyExtractor.apply(t), Boolean.TRUE) == null;
}
}
/**
* @author: 代码丰
* @Date: 2022/10/24 17:24
* @Description:
*/
public class DeRepeatFromOneListTest {
public static void main(String[] args) {
//测试3:去重一个列表的重复值
List<Student> list = new ArrayList<>();
for(int i = 1;i<=5;i++){
Student student = new Student();
student.setId("1");
student.setIndex("1"+String.valueOf(i));
student.setName("name"+String.valueOf(i));
list.add(student);
}
List<Student> after= list.stream().filter(CustomizeDistinctUtil.distinctByKey(Student::getId)).collect(Collectors.toList());
}
}
Tail
Have you all learned a lesson?