table of Contents
Preface
In many cases, we need to de-duplicate the data in the List. Below we will introduce five ways to remove duplicate data in the List.
1. Use LinkedHashSet to delete duplicate data in arraylist
LinkedHashSet is the best way to delete duplicate data in an ArrayList. LinkedHashSet accomplishes two things internally:
1. Delete duplicate data
2. Keep the order of the data added to it
Use LinkedHashSet to delete duplicates in the arraylist. In the given example, numbersList is an arraylist containing integers, some of which are repeated numbers.
For example 1, 3 and 5. We add the list to the LinkedHashSet, and then return the content to the list. The result arraylist has no repeated integers.
import java.util.ArrayList;
import java.util.Arrays;
import java.util.LinkedHashSet;
public class ArrayListExample
{
public static void main(String[] args)
{
ArrayList<Integer> numbersList = new ArrayList<>(Arrays.asList(1, 1, 2, 3, 3, 3, 4, 5, 6, 6, 6, 7, 8));
System.out.println(numbersList);
LinkedHashSet<Integer> hashSet = new LinkedHashSet<>(numbersList);
ArrayList<Integer> listWithoutDuplicates = new ArrayList<>(hashSet);
System.out.println(listWithoutDuplicates);
}
}
Output result
[1, 1, 2, 3, 3, 3, 4, 5, 6, 6, 6, 7, 8]
[1, 2, 3, 4, 5, 6, 7, 8]
2. Use java8 new feature stream for List deduplication
To remove duplicates from the arraylist, we can also use java 8 stream api. Use Steam's distinct() method to return a stream composed of different data, which is compared through the object's equals() method.
Collect all area data List using Collectors.toList().
Used to remove duplicate items from the arraylist in java without using Set.
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.stream.Collectors;
public class ArrayListExample
{
public static void main(String[] args)
{
ArrayList<Integer> numbersList = new ArrayList<>(Arrays.asList(1, 1, 2, 3, 3, 3, 4, 5, 6, 6, 6, 7, 8));
System.out.println(numbersList);
List<Integer> listWithoutDuplicates = numbersList.stream().distinct().collect(Collectors.toList());
System.out.println(listWithoutDuplicates);
}
}
Output result
[1, 1, 2, 3, 3, 3, 4, 5, 6, 6, 6, 7, 8]
[1, 2, 3, 4, 5, 6, 7, 8]
3. Use HashSet
Use HashSet's feature that duplicate data cannot be added. Since HashSet cannot guarantee the order of addition, it can only be used as a judgment condition to guarantee the order:
private static void removeDuplicate(List<String> list) {
HashSet<String> set = new HashSet<String>(list.size());
List<String> result = new ArrayList<String>(list.size());
for (String str : list) {
if (set.add(str)) {
result.add(str);
}
}
list.clear();
list.addAll(result);
}
4. Use the contains method of List
Use the contains method of List to loop through, reorder, add data only once to avoid duplication:
private static void removeDuplicate(List<String> list) {
List<String> result = new ArrayList<String>(list.size());
for (String str : list) {
if (!result.contains(str)) {
result.add(str);
}
}
list.clear();
list.addAll(result);
}
5. Double for loop to remove duplicates
for (int i = 0; i < list.size(); i++) {
for (int j = 0; j < list.size(); j++) {
if(i!=j&&list.get(i)==list.get(j)) {
list.remove(list.get(j));
}
}