Odd crash when deallocating OC_PythonArray

Issue #124 new
Kentzo
created an issue

I have an odd crash that only happens occasionally on exit.

Unfortunately I cannot post the code to reproduce it right now, but what I do is basically store a python list in NSUserDefaults by using the setObject_forKey_ method like this:

self._user_defaults = objc.lookUpClass('NSUserDefaults').standardUserDefaults()

# ...

self._user_defaults.setObject_forKey_([str(v) for v in value], name)

I only have pyobjc-core installed.

Here is the crash report.

Process:               Python [2033]
Path:                  /usr/local/Cellar/python3/3.4.2/Frameworks/Python.framework/Versions/3.4/Resources/Python.app/Contents/MacOS/Python
Identifier:            Python
Version:               3.4.2 (3.4.2)
Code Type:             X86-64 (Native)
Parent Process:        zsh [722]
Responsible:           Terminal [570]
User ID:               501

Date/Time:             2015-07-16 18:36:45.573 -0400
OS Version:            Mac OS X 10.10.4 (14E46)
Report Version:        11
Anonymous UUID:        319B57F5-91FC-214A-1DF0-430F5EACDFA4

Sleep/Wake UUID:       B243AE87-DEA6-4EDB-B021-6D08CA8E8FCA

Time Awake Since Boot: 1100000 seconds
Time Since Wake:       360000 seconds

Crashed Thread:        2  Dispatch queue: com.apple.root.default-qos

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000

Application Specific Information:
abort() called
/SourceCache/libpthread/libpthread-105.10.1/src/pthread.c:pthread_exit:1316: pthread_exit() may only be called against threads created via pthread_create()

Thread 0:: Dispatch queue: com.apple.main-thread
0   org.python.python               0x000000010ac49963 tupledealloc + 106
1   org.python.python               0x000000010ac1be36 code_dealloc + 93
2   org.python.python               0x000000010ac298c8 func_dealloc + 91
3   org.python.python               0x000000010ac35e71 insertdict + 405
4   org.python.python               0x000000010ac3ef44 _PyModule_ClearDict + 393
5   org.python.python               0x000000010acb3902 PyImport_Cleanup + 834
6   org.python.python               0x000000010acbe524 Py_Finalize + 95
7   org.python.python               0x000000010acd2615 Py_Main + 3393
8   org.python.python               0x000000010abf6e27 0x10abf5000 + 7719
9   libdyld.dylib                   0x00007fff932075c9 start + 1

Thread 1:: Dispatch queue: com.apple.libdispatch-manager
0   libsystem_kernel.dylib          0x00007fff85be6232 kevent64 + 10
1   libdispatch.dylib               0x00007fff93225a6a _dispatch_mgr_thread + 52

Thread 2 Crashed:: Dispatch queue: com.apple.root.default-qos
0   libsystem_kernel.dylib          0x00007fff85be4c82 __kill + 10
1   ???                             0x0000000000000001 0 + 1
2   libsystem_c.dylib               0x00007fff92373b53 abort + 129
3   libsystem_pthread.dylib         0x00007fff89762cef __pthread_abort + 49
4   libsystem_pthread.dylib         0x00007fff89762da3 __pthread_abort_reason + 180
5   libsystem_pthread.dylib         0x00007fff897603dc pthread_exit + 63
6   org.python.python               0x000000010acd00f5 PyThread_exit_thread + 21
7   org.python.python               0x000000010ac9aced PyEval_RestoreThread + 75
8   org.python.python               0x000000010acbd531 PyGILState_Ensure + 89
9   _objc.so                        0x000000010c092da3 -[OC_PythonArray release] + 19
10  com.apple.CoreFoundation        0x00007fff93ae2db0 CFRelease + 304
11  com.apple.CoreFoundation        0x00007fff93c8e4b6 __67-[CFPrefsPlistSource sendFullyPreparedMessage:settingValue:forKey:]_block_invoke_2 + 166
12  libxpc.dylib                    0x00007fff8a8dd52c _xpc_connection_reply_callout + 47
13  libxpc.dylib                    0x00007fff8a8dd4b8 _xpc_connection_call_reply + 36
14  libdispatch.dylib               0x00007fff93222c13 _dispatch_client_callout + 8
15  libdispatch.dylib               0x00007fff9322588f _dispatch_root_queue_drain + 935
16  libdispatch.dylib               0x00007fff93233fe4 _dispatch_worker_thread3 + 91
17  libsystem_pthread.dylib         0x00007fff8975f637 _pthread_wqthread + 729
18  libsystem_pthread.dylib         0x00007fff8975d40d start_wqthread + 13

Thread 3:
0   libsystem_kernel.dylib          0x00007fff85be594a __workq_kernreturn + 10
1   libsystem_pthread.dylib         0x00007fff8975d40d start_wqthread + 13

Comments (12)

  1. Kentzo reporter

    I updated the code to the following:

    self._user_defaults = objc.lookUpClass('NSUserDefaults').standardUserDefaults()
    self._nsarray_class = objc.lookUpClass('NSArray')
    
    # ...
    
    with objc.autorelease_pool():
        self._user_defaults.setObject_forKey_(self._nsarray_class.arrayWithArray_([str(v) for v in value]), name)
    

    But it still crashes, with slightly different exception:

    Thread 2 Crashed:: Dispatch queue: com.apple.root.default-qos
    0   libsystem_kernel.dylib          0x00007fff85be4c82 __kill + 10
    1   ???                             0x00007ff14c587be1 0 + 140674344713185
    2   libsystem_c.dylib               0x00007fff92373b53 abort + 129
    3   libsystem_pthread.dylib         0x00007fff89762cef __pthread_abort + 49
    4   libsystem_pthread.dylib         0x00007fff89762da3 __pthread_abort_reason + 180
    5   libsystem_pthread.dylib         0x00007fff897603dc pthread_exit + 63
    6   org.python.python               0x00000001001690f5 PyThread_exit_thread + 21
    7   org.python.python               0x0000000100133ced PyEval_RestoreThread + 75
    8   org.python.python               0x0000000100156531 PyGILState_Ensure + 89
    9   _objc.so                        0x0000000101535913 -[OC_PythonUnicode release] + 19
    10  com.apple.CoreFoundation        0x00007fff93ae2db0 CFRelease + 304
    11  com.apple.CoreFoundation        0x00007fff93b03bbd -[__NSArrayI dealloc] + 125
    12  libobjc.A.dylib                 0x00007fff8958289c objc_object::sidetable_release(bool) + 236
    13  com.apple.CoreFoundation        0x00007fff93ae2db0 CFRelease + 304
    14  com.apple.CoreFoundation        0x00007fff93c8e4b6 __67-[CFPrefsPlistSource sendFullyPreparedMessage:settingValue:forKey:]_block_invoke_2 + 166
    15  libxpc.dylib                    0x00007fff8a8dd52c _xpc_connection_reply_callout + 47
    16  libxpc.dylib                    0x00007fff8a8dd4b8 _xpc_connection_call_reply + 36
    17  libdispatch.dylib               0x00007fff93222c13 _dispatch_client_callout + 8
    18  libdispatch.dylib               0x00007fff9322588f _dispatch_root_queue_drain + 935
    19  libdispatch.dylib               0x00007fff93233fe4 _dispatch_worker_thread3 + 91
    20  libsystem_pthread.dylib         0x00007fff8975f637 _pthread_wqthread + 729
    21  libsystem_pthread.dylib         0x00007fff8975d40d start_wqthread + 13
    

    I'm going to try to use NSString instead of OC_PythonUnicode.

  2. Kentzo reporter

    The final versions seems to work, but since bug is occasional it's hard to be sure:

    self._user_defaults = objc.lookUpClass('NSUserDefaults').standardUserDefaults()
    self._nsarray_class = objc.lookUpClass('NSArray')
    
    # ...
    
    self._user_defaults.setObject_forKey_(self._nsarray_class.arrayWithArray_([self._nsstring_class.stringWithString_(str(v)) for v in value]), self._nsstring_class.stringWithString_(name))
    
  3. Ronald Oussoren repo owner

    Not sure what's going on here. The first crash log seems to indicate that this happens during/after interpreter shutdown (see PyEval_RestoreThread in Python/ceval.c).

    What I think happens is that the .app calls Py_Finalize and after that either the main thread or some other thread cleans up Cocoa and that calls back into Python again. That fails because the Python interpreter is already gone.

    BTW. Which version of PyObjC do you use? And how do you build the application bundle? Using py2app or something else?

    I have to think about this a bit more, but think I'll work around this in py2app by not calling Py_Finalize. That's not ideal (because the usual orderly shutdown for Python objects is not run), but better than the alternative.

  4. Ronald Oussoren repo owner

    Pyqtdeploy does the same as py2app: it calls py_finalize at the end, and the orderly shutdown of cocoa can happen after that call.

    I'm not sure if there is a way to avoid crashes here. For Python 3 it might be possible to invalidate all proxy objects, but that will likely have other unwanted side effects (such as silently erasing a user setting instead of crashing).

    -- On the road, hence brief.

  5. Ronald Oussoren repo owner

    I think I can fix this particular crash on the PyObjC side by adding more code to -[OC_Python* release] and dealloc, those basically need to check if the interpreter is initialised before calling into Python. That will have a small performance impact, but that cannot be helped.

    I don't know yet how to fix this in general though. There is a race condition between shutdown of Python and cleanup of Cocoa. I can work around this in the management of reference counts, but not when Cocoa tries to call into Python code that does something more substantial.

    I have a (partial) workaround in changeset 68804363ebf4 (not yet pushed to the repository)

  6. Ronald Oussoren repo owner

    It already does, but not before all other threads have died. What seems to happen in the first crash dump is that Py_Finalize is working on cleaning up the interpreter (and has already marked it as being no longer valid), while another thread (Thread 2) is still doing some work with a OC_PythonArray object (reducing the refcount).

    The only way you can be sure to not have crashes in this case is to remove the call to Py_Finalize in pyqtdeploy_start.cpp. That has an unwanted side effect though: the interpreter is not cleanly shut down and that means the destructors of objects aren't run. One possible result of that is that open files aren't closed cleanly, in particularly buffers aren't flushed (more so in Python 3 than in Python 2, in the latter the buffers are in stdio and that will still clean up after itself).

  7. Log in to comment