deadlock when releasing sharedpointers, that share memory with PyObject*

Issue #81 resolved
M. Gronle
created an issue

This issue has been posted on the itom-discussion mailing list:

Hello,

when I was working with the SerialIO plugin, I noticed random timeouts that did not happen a while ago.

I simplified the getVal and setVal function extremely to trace down the reason:

ito::RetVal SerialIO::getVal(QSharedPointer<char> data, QSharedPointer<int> length, ItomSharedSemaphore *waitCond) { ItomSharedSemaphoreLocker locker(waitCond); ito::RetVal retValue; if (waitCond) { waitCond->returnValue =3D retValue; waitCond->release(); } return retValue; }

ito::RetVal SerialIO::setVal(const char data, const int datalength, ItomSharedSemaphore waitCond) { ItomSharedSemaphoreLocker locker(waitCond); ito::RetVal retval(ito::retOk);

if (waitCond)
{
    waitCond->returnValue =3D retval;
    waitCond->release();
}
return retval;

}

The timeouts are still happening with this python code in Itom:

ser =3D dataIO("SerialIO", 88, 115200,"\n",8,1,0,16,0,0) buf =3D bytearray(50)

for i in range(1,512): ser.setVal('bla') ser.getVal(buf)

After a while I always get:

            RuntimeError: Error invoking function setVal with error message: timeout while calling 'setVal'

Since I did not have this problem a while ago I found out that it started after a specific commit:

  • Hash: 556def47065735e0c1c7ec1b2e43c95acabdf623

  • Message: fix in QSharedPointer deleter method which are responsible to also decrement any PyObject elements. Sometimes, it is not verified which thread (Python, GUI...) is finally releasing a QSharedPointer, which for instance wraps a dataObject, char-buffer... whose base is another PyObject element. Once the pointer is released, a specific deleter-method is called which calls PyDECREF... of the PyObject base pointer. However if PyDECREF is called, the Python interpreter lock (GIL) must be hold by the caller, else a crash might occur. This is now ensured. The solution is only safe for Python >=3D 3.4, since other versions cannot tell if the GIL is currently hold by the caller or not, which can lead to a deadlock.

  • Date: 08.08.2018 18:19:37

My python version is 3.5.2 so it should work.

Comments (1)

  1. M. Gronle reporter

    fixes issue #81: when a QSharedPointer is build to share memory with a PyObject, like bytearray or bytes, this PyObject has to be decremented once the QSharedPointer is deleted. This is done in a deleter-method. To decref the PyObject* the GIL has to be acquired. This caused a dead-lock in some cases. This is fixed now.

    → <<cset b1f60295f386>>

  2. Log in to comment