[Cross posted from https://github.com/joblib/joblib/issues/557]
EDIT: Changed description and example after further digging
I have observed a memory leak when pickling R objects created in python through rpy2. Here is some code that will reproduce the leak (WARNING: Running this might eat up your RAM):
import pickle import tempfile import rpy2.robjects as R def test_R_object_dump(N=1000): """Test creating big R matrices in a loop and dumping them to a temp file""" for i in range(N): print(i, end=", ") with tempfile.TemporaryFile() as f: a = R.r.matrix(1., 5000, 5000) pickle.dump(a, f)
I've observed the leak in Linux (Ubuntu 16.04), Mac OSX seems to not have the issue.
Upon further digging, I believe this might be related to Issue
#321 but on the pickling (rather than unpickling) side of things.