Can't import _posixsubprocess (subprocess32 package) (missing _PyImport_Release/AcquireLock)

Issue #2451 resolved
Frank Riley
created an issue

Running pypy-5.6-linux_x86_64-portable in a virtualenv:

>>>> import _posixsubprocess
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: unable to load extension module '/data/wombat/home/fhr/silkthread/env27-pypy/site-packages/_posixsubprocess.pypy-41.so': /data/wombat/home/fhr/silkthread/env27-pypy/site-packages/_posixsubprocess.pypy-41.so: undefined symbol: _PyImport_ReleaseLock

Comments (7)

  1. Armin Rigo

    The module _posixsubprocess that you're trying to import was created by cpyext. By default, there is a built-in one that gets imported successfully. Can you investigate what you did that caused this _posixsubprocess.pypy-41.so to be created?

  2. squeaky

    This was opened as well against portable-pypy. I closed the issue in portable-pypy.

    It looks like this module was created by installing the package subprocess32 (https://github.com/google/python-subprocess32) which backports some Python 3 subprocess changes.

    The import fails with undefined symbol: _PyImport_ReleaseLock which looks like relying on a private cext method to me and it's possibly not available under cpyext (didn't verify that).

    In https://github.com/google/python-subprocess32/blob/master/_posixsubprocess.c#L836 you can see following code:

    #define MIN_PY_VERSION_WITH_PYIMPORT_ACQUIRELOCK 0x02060300
    #if (PY_VERSION_HEX < MIN_PY_VERSION_WITH_PYIMPORT_ACQUIRELOCK)
    static PyObject* imp_module;
    
    static void
    _PyImport_AcquireLock(void)
    {
        PyObject *result;
        result = PyObject_CallMethod(imp_module, "acquire_lock", NULL);
        if (result == NULL) {
            fprintf(stderr, "imp.acquire_lock() failed.\n");
            return;
        }
        Py_DECREF(result);
    }
    
    static int
    _PyImport_ReleaseLock(void)
    {
        PyObject *result;
        result = PyObject_CallMethod(imp_module, "release_lock", NULL);
        if (result == NULL) {
            fprintf(stderr, "imp.release_lock() failed.\n");
            return -1;
        }
        Py_DECREF(result);
        return 0;
    }
    #endif /* Python <= 2.5 */
    

    So basically it provides implementation calling out to imp module on earlier CPython2 and uses private APIs on later CPython2.

    Not sure if PyPy cpyext should support these APIs but since this package is quite popular it might be worth adding them. Otherwise the bug should be opened against subprocess32 module.

  3. Log in to comment