1. weakref non-permanent object references

weakref object module supports weak references. A reference to the normal increase in the number of references to objects, and to avoid it being garbage collected. But the results are not always as expected in it, such as may sometimes appear a circular reference, or sometimes you may want to delete the cache object memory when needed. Weak reference (weak reference) is an unavoidable object is automatically cleaned object handle.

1.1 references

Weak reference to the object to be managed by a ref class. To get the original object, you can call the reference object.

import weakref

class ExpensiveObject:

    def __del__(self):
        print('(Deleting {})'.format(self))

obj = ExpensiveObject()
r = weakref.ref(obj)

print('obj:', obj)
print('ref:', r)
print('r():', r())

print('deleting obj')
del obj
print('r():', r())

Here, because the obj has been deleted before the second call references, so the ref to return None.

1.2 reference to callback

ref constructor accepts an optional callback function, the function will be called when the object is deleted references.

import weakref

class ExpensiveObject:

    def __del__(self):
        print('(Deleting {})'.format(self))

def callback(reference):
    """Invoked when referenced object is deleted"""
    print('callback({!r})'.format(reference))

obj = ExpensiveObject()
r = weakref.ref(obj, callback)

print('obj:', obj)
print('ref:', r)
print('r():', r())

print('deleting obj')
del obj
print('r():', r())

When referring to has been "dead" and no longer references the original object, the callback will accept the reference object as a parameter. One use of this feature is to remove weak reference objects from the cache.

1.3 The final object

To complete a more robust resource management to clean up weak references, can be used to finalize the callback associated with the object. finalize instance will remain (until the associated objects are removed), even if the application is not final object references.

import weakref

class ExpensiveObject:

    def __del__(self):
        print('(Deleting {})'.format(self))

def on_finalize(*args):
    print('on_finalize({!r})'.format(args))

obj = ExpensiveObject()
weakref.finalize(obj, on_finalize, 'extra argument')

del obj

finalize the parameters include the object to be tracked, callable object to be invoked when recycled garbage, and all incoming position of the callable or named parameters.

Finalize this instance has a writable property atexit, used to control whether to call this callback (if not already call) when the program exits.

import sys
import weakref

class ExpensiveObject:

    def __del__(self):
        print('(Deleting {})'.format(self))

def on_finalize(*args):
    print('on_finalize({!r})'.format(args))

obj = ExpensiveObject()
f = weakref.finalize(obj, on_finalize, 'extra argument')
f.atexit = bool(int(sys.argv[1]))

The default setting is to call this callback. The atexit disable this behavior set to false.

If you provide the tracked object instance to finalize a reference, which would lead to a reference is retained, so the object will never be garbage.

import gc
import weakref

class ExpensiveObject:

    def __del__(self):
        print('(Deleting {})'.format(self))

def on_finalize(*args):
    print('on_finalize({!r})'.format(args))

obj = ExpensiveObject()
obj_id = id(obj)

f = weakref.finalize(obj, on_finalize, obj)
f.atexit = False

del obj

for o in gc.get_objects():
    if id(o) == obj_id:
        print('found uncollected object in gc')

As shown above, although obj explicit references have been removed, but the object remains visible on the garbage collector by f.

Using the tracked object as a binding method may be appropriately avoided callable objects finalized.

import gc
import weakref

class ExpensiveObject:

    def __del__(self):
        print('(Deleting {})'.format(self))

    def do_finalize(self):
        print('do_finalize')

obj = ExpensiveObject()
obj_id = id(obj)

f = weakref.finalize(obj, obj.do_finalize)
f.atexit = False

del obj

for o in gc.get_objects():
    if id(o) == obj_id:
        print('found uncollected object in gc')

Because callable to finalize provide obj is an instance of a binding approach, the final method to retain a reference to obj, it can not be removed and recycled garbage.

1.4 Proxy

Sometimes using a proxy weak references more convenient. Use a proxy can be used like the original objects, and does not require access to the object before the first call to the agent. This shows that an agent can be delivered to a library, but the library does not know that it is receiving a reference instead of a real object.

import weakref

class ExpensiveObject:

    def __init__(self, name):
        self.name = name

    def __del__(self):
        print('(Deleting {})'.format(self))

obj = ExpensiveObject('My Object')
r = weakref.ref(obj)
p = weakref.proxy(obj)

print('via obj:', obj.name)
print('via ref:', r().name)
print('via proxy:', p.name)
del obj
print('via proxy:', p.name)

If the reference object is deleted after access to the proxy, it will have a ReferenceError exception.

1.5 Cache Objects

ref proxy class and are considered "bottom" of. Although they maintain a weak reference to a single object it is useful, and also supports garbage collection for circular references, but WeakKeyDictionary and WeakValueDictionary classes provide a more suitable API to create multiple objects cache.

WeakValueDictionary class uses a weak reference value it contains, the other code is no longer true when using these values, is allowed garbage collection. Use garbage collector called explicitly, the following shows the difference between the use of conventional dictionaries and WeakValueDictionary complete memory processing.

import gc
from pprint import pprint
import weakref

gc.set_debug(gc.DEBUG_UNCOLLECTABLE)

class ExpensiveObject:

    def __init__(self, name):
        self.name = name

    def __repr__(self):
        return 'ExpensiveObject({})'.format(self.name)

    def __del__(self):
        print('    (Deleting {})'.format(self))

def demo(cache_factory):
    # hold objects so any weak references
    # are not removed immediately
    all_refs = {}
    # create the cache using the factory
    print('CACHE TYPE:', cache_factory)
    cache = cache_factory()
    for name in ['one', 'two', 'three']:
        o = ExpensiveObject(name)
        cache[name] = o
        all_refs[name] = o
        del o  # decref

    print('  all_refs =', end=' ')
    pprint(all_refs)
    print('\n  Before, cache contains:', list(cache.keys()))
    for name, value in cache.items():
        print('    {} = {}'.format(name, value))
        del value  # decref

    # remove all references to the objects except the cache
    print('\n  Cleanup:')
    del all_refs
    gc.collect()

    print('\n  After, cache contains:', list(cache.keys()))
    for name, value in cache.items():
        print('    {} = {}'.format(name, value))
    print('  demo returning')
    return

demo(dict)
print()

demo(weakref.WeakValueDictionary)

If the loop variable indicates the value of the cache, then the loop variable must be explicitly cleared, so that the object of reducing the number of references. Otherwise, the garbage collector does not remove these objects, they will still remain in the cache. Similarly, all_refs variable to store references to prevent their premature garbage collection.

WeakKeyDictionary work is similar, but uses the dictionary keys weak references instead of a weak reference value.

Python3 standard library: non-permanent weakref object references