finalizers heavily stress GC

Issue #2590 resolved
Philip Jenvey
created an issue

@Alex Gaynor's script demonstrates app level __del__s:

import random
import sys

class A(object):
    def __del__(self):
        pass


class B(object):
    pass

cls = {"A": A, "B": B}[sys.argv[1]]

while True:
    # Gibberish to make sure the JIT doesn't optimize the allocation away
    [cls()] * random.randint(1, 1)

"A" shooting RSS up to 1-2GB pretty quickly vs "B" holding steady =~ 20mb.

massif+pypy2-5.6 on "A": https://pastebin.mozilla.org/9025519

Comments (6)

  1. Philip Jenvey reporter

    Thanks for bisecting it down @Petre Vijiac

    5.7.13 looks even worse

    @Armin Rigo - Seems the likeliest culprit between 2.6.1 and 4.0 is the gc-more-incremental branch. Did it make external_malloc too clever (or not clever enough) with finalizers (alloc_young=False)?

    Also in between 4 and 5.7 gc-del-3 landed (in 5.3), it added a comment suggesting external_malloc shouldn't even have an alloc_young=False any longer:

    https://bitbucket.org/pypy/pypy/src/fcd9df9/rpython/memory/gc/incminimark.py?at=default&fileviewer=file-view-default#incminimark.py-880

  2. Armin Rigo

    Fixed by the gc-del-limit-growth branch in 44577e4653fa. We need to prevent the GC letting the heap grow more and more in case this heap contains a lot of objects linked from dying objects with finalizers. It is hopefully enough to simply not count such objects towards the next limit.

  3. Log in to comment