I am seeing some strange behavior in a Java program, and I'm wondering if the behavior is expected, and if it's documented anywhere.
I am placing some WeakReference objects into a collection. (Yes, I know I should use WeakHashMap -- it has the same odd behavior, and that's not what this question is about.)
In some circumstances, the object referenced by the last WeakReference placed into the collection does not get garbage collected when I expect it to.
Below, there is a collection of unit tests that show the behavior I'm seeing. All of these tests pass as written, and there are comments where the odd behavior is seen. (Tested using Oracle JDK 1.8 and OpenJDK 11.)
In the first test, I am inserting into the collection a WeakReference to an object that is returned from function call:
List<WeakReference<Person>> refs = Lists.newArrayList();
refs.add(new WeakReference(getPerson("abc")));
The objects that are referenced all get garbage collected as expected.
In the second test, I have create a scoped variable to hold the function's returned object, create a WeakReference to it, and insert that into the collection. The variable then goes out of scope, which seems like it should remove any reference. In all but the last case, this is true: the objects they reference get garbage collected. But the last one hangs around.
List<WeakReference<Person>> refs = Lists.newArrayList();
{
Person person = getPerson("abc");
refs.add(new WeakReference(person));
}
In the third test, I add an additional temporary scope, and explicitly use an additional scoped variable that doesn't get added to the collection. All of the items with references in the collection get garbage collected properly.
List<WeakReference<Person>> refs = Lists.newArrayList();
{
Person person = getPerson("abc");
refs.add(new WeakReference(person));
}
...
{
Person person = null;
}
And in the fourth test, since I was curious if the behavior were related to the variables all having the same name -- were they somehow being interpreted as the same variable? -- I used different names for all of the temporary variables. All of the items with references in the collection get garbage collected as expected.
List<WeakReference<Person>> refs = Lists.newArrayList();
{
Person person1 = getPerson("abc");
refs.add(new WeakReference(person1));
}
...
{
Person person4 = null;
}
The only explanation I can come up with is that somehow the JRE is maintaining a reference to the last object being created, even though it goes out of scope. But I haven't seen any documentation that describes it.
Updated 1: a new test/workaround:
If I explicitly set the scoped variable to null before it goes out of scope, the objects get garbage collected as I would expect.
List<WeakReference<Person>> refs = Lists.newArrayList();
{
Person person = getPerson("abc");
refs.add(new WeakReference(person));
person = null;
}
Updated 2: Another new test:
The new, extraneous object doesn't need to be of the same type. This works fine.
List<WeakReference<Person>> refs = Lists.newArrayList();
{
Person person = getPerson("abc");
refs.add(new WeakReference(person));
}
...
{
String unused = "unused string";
}
import com.google.common.base.MoreObjects;
import com.google.common.collect.Lists;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.lang.ref.WeakReference;
import java.util.Iterator;
import java.util.List;
import java.util.Objects;
import static org.testng.Assert.assertEquals;
public class WeakReferenceCollectionTest {
private static final Logger logger = LoggerFactory.getLogger(WeakReferenceCollectionTest.class);
static class Person {
private String name;
public Person() {
}
public String getName() {
return name != null ? name : "<null>";
}
public Person setName(String name) {
this.name = name;
return this;
}
@Override
public String toString() {
return MoreObjects.toStringHelper(this)
.add("name", name)
.toString();
}
@Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
final Person person = (Person) o;
return Objects.equals(name, person.name);
}
@Override
public int hashCode() {
return Objects.hash(name);
}
}
@Test
public void collectionWorksAsExpected() throws InterruptedException {
List<WeakReference<Person>> refs = Lists.newArrayList();
refs.add(new WeakReference(getPerson("abc")));
refs.add(new WeakReference(getPerson("bcd")));
refs.add(new WeakReference(getPerson("cde")));
assertEquals(refs.size(), 3);
System.gc();
Thread.sleep(1000);
evictDeadRefs(refs);
assertEquals(refs.size(), 0);
refs.add(new WeakReference(getPerson("def")));
refs.add(new WeakReference(getPerson("efg")));
refs.add(new WeakReference(getPerson("fgh")));
assertEquals(refs.size(), 3);
System.gc();
Thread.sleep(1000);
evictDeadRefs(refs);
assertEquals(refs.size(), 0);
}
@Test
public void collectionWithScopesWorksDifferently() throws InterruptedException {
List<WeakReference<Person>> refs = Lists.newArrayList();
{
Person person = getPerson("abc");
refs.add(new WeakReference(person));
}
{
Person person = getPerson("bcd");
refs.add(new WeakReference(person));
}
{
Person person = getPerson("cde");
refs.add(new WeakReference(person));
}
assertEquals(refs.size(), 3);
System.gc();
Thread.sleep(1000);
evictDeadRefs(refs);
assertEquals(refs.size(), 1); // last one never goes away
assertEquals(refs.get(0).get().getName(), "cde");
{
Person person = getPerson("def");
refs.add(new WeakReference(person));
}
{
Person person = getPerson("efg");
refs.add(new WeakReference(person));
}
{
Person person = getPerson("fgh");
refs.add(new WeakReference(person));
}
assertEquals(refs.size(), 4); // previous last one is still in there
System.gc();
Thread.sleep(1000);
evictDeadRefs(refs);
assertEquals(refs.size(), 1); // last one never goes away
assertEquals(refs.get(0).get().getName(), "fgh");
}
@Test
public void collectionWithScopesAndNewVariableSetToNull() throws InterruptedException {
List<WeakReference<Person>> refs = Lists.newArrayList();
{
Person person = getPerson("abc");
refs.add(new WeakReference(person));
}
{
Person person = getPerson("bcd");
refs.add(new WeakReference(person));
}
{
Person person = getPerson("cde");
refs.add(new WeakReference(person));
}
{
Person person = null;
}
assertEquals(refs.size(), 3);
System.gc();
Thread.sleep(1000);
evictDeadRefs(refs);
assertEquals(refs.size(), 0);
}
@Test
public void collectionWithScopesAndDifferentVariableNames() throws InterruptedException {
List<WeakReference<Person>> refs = Lists.newArrayList();
{
Person person1 = getPerson("abc");
refs.add(new WeakReference(person1));
}
{
Person person2 = getPerson("bcd");
refs.add(new WeakReference(person2));
}
{
Person person3 = getPerson("cde");
refs.add(new WeakReference(person3));
}
{
Person person4 = null;
}
assertEquals(refs.size(), 3);
System.gc();
Thread.sleep(1000);
evictDeadRefs(refs);
assertEquals(refs.size(), 0);
}
@Test
public void collectionWithScopesAndExplicitlySetToNull() throws InterruptedException {
List<WeakReference<Person>> refs = Lists.newArrayList();
{
Person person = getPerson("abc");
refs.add(new WeakReference(person));
person = null;
}
{
Person person = getPerson("bcd");
refs.add(new WeakReference(person));
person = null;
}
{
Person person = getPerson("cde");
refs.add(new WeakReference(person));
person = null;
}
assertEquals(refs.size(), 3);
System.gc();
Thread.sleep(1000);
evictDeadRefs(refs);
assertEquals(refs.size(), 0);
}
@Test
public void createUnrelatedVariable() throws InterruptedException {
List<WeakReference<Person>> refs = Lists.newArrayList();
{
Person person = getPerson("abc");
refs.add(new WeakReference(person));
}
{
Person person = getPerson("bcd");
refs.add(new WeakReference(person));
}
{
Person person = getPerson("cde");
refs.add(new WeakReference(person));
}
{
String unused = "unused string";
}
assertEquals(refs.size(), 3);
System.gc();
Thread.sleep(1000);
evictDeadRefs(refs);
assertEquals(refs.size(), 0);
}
private void evictDeadRefs(List<WeakReference<Person>> refs) {
final Iterator<WeakReference<Person>> it = refs.iterator();
while (it.hasNext()) {
final WeakReference<Person> ref = it.next();
if (ref.get() == null) {
logger.debug("evictDeadRefs(): removing ref");
it.remove();
} else {
logger.debug("evictDeadRefs(): ref is not null: " + ref.get());
}
}
}
private Person getPerson(String s) {
return new Person().setName(s);
}
}
I think you are seeing some interactions with how the Java code is compiled to byte code. Two important things to note:
- The Garbage collector does not guarantee when, or even if, an object will be collected. The guarantee is on which objects won't be.
- Byte code does not have "local variables". Instead it has a local stack, with many stack frames. A local variable is translated into a specific location in a stack frame.
Because of #1, the scoping curly braces of Java are not required to be implemented as a new stack frame. Instead, the java compiler can create one stack frame for the whole method, and use it in a way that is consistent with the scoping rules. This means, in the second test, the local variable person is represented by a stack frame index that lives until the end of the method, preventing garbage collection.
Because of #2, and because local variables must be initialized before they are used, the java compiler can reuse one index of a stack frame to represent multiple local variables, so long as no two of them are ever in scope at the same time. Thus, all of your "different" person local variables in tests 3 and 4 end up being the same location on the stack.
TL;DR: Don't expect garbage collection to be consistent. When an object is collected can be affected both by which JVM GC you are using, and by the implementation specific details of your Java compiler.