Segfault when running unit test `robjects.tests.testRobjects MappingTestCase.testMapperPy2R_functio`

Issue #471 resolved
Laurent Gautier
created an issue

Running the test results in a segfault:

python -m rpy2.robjects.tests.testRobjects MappingTestCase.testMapperPy2R_function 

Comments (6)

  1. Laurent Gautier reporter

    From a quick look at the trace when catching the segfault in gdb, this is caused by trying to decrease the reference count of a Python object wrapper in an R external pointer (this is something done when having Python functions wrapped as R functions) :

        at ./rpy/rinterface/rexternalptr.c:41
    41      Py_DECREF(pyo);
    

    Changing the decrement to a more permissive Py_XDECREF(pyo) does not solve the issue.

    Interestingly:

    • running the problematic code in a script (not through unit testing) does not trigger the issue. The following is working just fine... unless an attempt to delete the object is made:
    from rpy2 import robjects
    func = lambda x: x
    rob = robjects.default_converter.py2ro(func)
    
    # segfault only if the deletion below is present
    del(rob)
    
    • the segfault with unittests is only happening when Python is exiting (end of the process).
  2. Laurent Gautier reporter

    Tracking this was a bit involved, but I think that I have it.

    Python is doing some cleanup when its process is ending and Python objects are collected in bulk even if their reference count is not at zero and this is happening before the embedded R's own cleanup. Whenever having a Python object that this wrapping an R object that is an external pointer to a Python object (which happens when calling rpy2.rinterface.rternalize()), by the time R is cleaning up and rpy2's custom finalizers for such objects are called the Python objects are no longer valid (or the Python interpreter is no longer functional, I am not sure which one of the two is the trigger) and trying to decrement the reference count for the Python objects pointed to by R's external pointer triggers the segfault.

    Forcing R's cleanup to happen before Python is terminating is preventing the segfault from happening:

    from rpy2 import robjects
    
    func = lambda x: x
    rob = robjects.default_converter.py2ro(func)
    
    # segfault only if the deletion below is present
    del(rob)
    
    # ...but the following prevents the segfault from occurring
    import atexit
    atexit.register(robjects.rinterface.endr, 0)
    
  3. Log in to comment