I'm trying to remove 140,000 objects from an ArrayList of size 7,140,000. I expected this would take seconds (if that), but instead Java is taking multiple seconds per thousand objects. Here is my code:
for (int i = list.size(); i > P; i--)
{
int size = list.size();
int index = (int) (Math.random() * size);
list.remove(index);
}
Note: P is a constant that I previously set to 7,000,000.
The goal of the loop is to randomly remove objects from list until its size is 7,000,000.
Is Java taking such a long time because I'm starting off with over 7 million objects? I've never noticed this efficiency problem with removing from ArrayLists in the past. If it helps, I use the DrJava Beta IDE.
An ArrayList is backed by an array, so modifications need to really move items aside, and in some cases even create a whole new array.
Some possible solutions:
Consider using LinkedList or skip-list implementation instead. Do note that here, to remove an item it still takes O(N) (or O(logN) in skip-list), because it has to find it. However, you can traverse the items with a chance based on how many items you've removed.
You could randomly take items from the input into a new ArrayList till you get the number of items you wish. You have to know which items you added though, so traverse in a linear way, and have the random chooser to have a chance of how many steps to go, based on how many items you've moved.
Easiest solution: shuffle the entire input array, and then choose the first M items.
Here's a possible code for solution #3:
public static List<String> pickNRandom(List<String> lst, int m) {
Collections.shuffle(lst);
return lst.subList(0, n);
}
The disadvantage here is that it ruins the order of the items. You can overcome this by creating a copy of the list as the input, but this will take more memory (temporarily) ...