memory still leaking

[Former user]

memory grows: py tt.py repeat=1000 leak ('leak' will do the gc.colect/print)

[Mike Bayer]

i have no idea and i think this probably cannot be fixed at this point, unless you want to try digging deep with pdb. I can get all dictionaries to be empty if I just add import sqlalchemy.util as sautil; sautil.ArgSingleton.instances.clear() to the end of clear_mappers()...additionally im printing out the caches on the attribute_manager...empty. yet GC is not getting anything. If there are unreachable cycles, GC should be cleaning those out.

[Former user]

(original author: sdobrev) well, this (ArgSingleton.clear) seems to fix it. if u think all these clearings are not appropriate to live in clear_mappers(), then make another function, or even registry, with more globalistic name and put all them in - and announce it, so whoever adds yet another cache, it's .clear() goes called/registered there.

eventualy u may add a test case running something simple 1000 times, and checking the memusage (maybe grep Vm /proc/$pid/status).

here a func to get mem-status:

def memusage(): import os pid = os.getpid() m = '' for l in file( '/proc/%(pid)s/status' % locals() ): l = l.strip() for k in 'VmPeak VmRSS VmData'.split(): if l.startswith(k): m += '; '+l if m: print m

now how to check if memusage grows... maybe do 1000 runs, and if last 10 memstatuses are different and growing, consider it leaking?

[Mike Bayer]

i feel its leaking because i can see Mappers and BoundMetaDatas adding up in the GC queue and not being cleared out.

[Former user]

(original author: sdobrev) hmmm if u dont call gc.collect, the memory does not grow, or grows a little once per 100 runs or something like that. something fishy there...

IMO it is ok enough as it is now.

[Mike Bayer]

this code:

gc.set_debug( gc.DEBUG_UNCOLLECTABLE | gc.DEBUG_SAVEALL | gc.DEBUG_INSTANCES | gc.DEBUG_STATS )

prevents gc.collect() from cleaning circular references. comment it out and everything clears after each run.

[Mike Bayer]

Removing milestone: 0.4.0 (automated comment)