cffi doesn't work inside a macOS app bundle signed with 'runtime' option without entitlement

Create issue
Issue #391 on hold
Glyph created an issue

per, Apple is encouraging all developers to get their apps "notarized", which comes along with new code signing requirements.

One of these requirements is the new 'runtime' code signing option, which is intended to preserve signature integrity by preventing dynamic loading of malicious code.

I am guessing something is broken with libffi, because when I try to retrieve an https URL with treq inside an application code-signed with this option, I get a traceback as soon as it starts trying to talk to cffi-wrapped APIs:

2018-10-28T11:45:56-0700 [stderr#error]     treq.get("").addCallback(print, "hooray")
2018-10-28T11:45:56-0700 [stderr#error]   File "treq/api.pyc", line 24, in get
2018-10-28T11:45:56-0700 [stderr#error]   File "treq/client.pyc", line 112, in get
2018-10-28T11:45:56-0700 [stderr#error]   File "treq/client.pyc", line 239, in request
2018-10-28T11:45:56-0700 [stderr#error]   File "twisted/web/client.pyc", line 1975, in request
2018-10-28T11:45:56-0700 [stderr#error]   File "twisted/web/client.pyc", line 2043, in request
2018-10-28T11:45:56-0700 [stderr#error]   File "twisted/web/client.pyc", line 1850, in request
2018-10-28T11:45:56-0700 [stderr#error]   File "twisted/web/client.pyc", line 1651, in request
2018-10-28T11:45:56-0700 [stderr#error]   File "twisted/web/client.pyc", line 1635, in _getEndpoint
2018-10-28T11:45:56-0700 [stderr#error]   File "twisted/web/client.pyc", line 1510, in endpointForURI
2018-10-28T11:45:56-0700 [stderr#error]   File "twisted/web/client.pyc", line 945, in creatorForNetloc
2018-10-28T11:45:56-0700 [stderr#error]   File "twisted/internet/_sslverify.pyc", line 1304, in optionsForClientTLS
2018-10-28T11:45:56-0700 [stderr#error]   File "twisted/internet/_sslverify.pyc", line 1664, in getContext
2018-10-28T11:45:56-0700 [stderr#error]   File "twisted/internet/_sslverify.pyc", line 1695, in _makeContext
2018-10-28T11:45:56-0700 [stderr#error]   File "OpenSSL/SSL.pyc", line 1108, in set_verify
2018-10-28T11:45:56-0700 [stderr#error]   File "OpenSSL/SSL.pyc", line 333, in __init__
2018-10-28T11:45:56-0700 [stderr#error] SystemError: <built-in method callback of CompiledFFI object at 0x10824e3e8> returned NULL without setting an error

Comments (21)

  1. Armin Rigo

    You'll need to provide a pull request, because I have limited motivation to try to understand Apple-specific messes without being able to test them myself.

  2. Armin Rigo

    No :-) Thanks anyway. In fact I'm with Fijal for the next 6 weeks, and he has got a Mac. At some point we (all three of us) could look at this.

  3. Maciej Fijalkowski

    Hey Glyph. Would you mind providing some background how can we create and sign such an app written in python? I'm sure it's findable somewhere in the docs, but I'm not finding it right now

  4. Glyph reporter

    Sorry for the long latency on a minimal reproducer here. Setting up a new py2app project that is minimal in a relevant way is a bit of a pain; I'd be happy to invite you to the (private) project I was testing with, but that pulls in all kind of gunk you don't want to look at with PyGame etc.

    I'm honestly not sure if there's anything for you to fix here at this point, except to use the MAP_JIT flag when available, which might be entirely libffi's problem anyway.

    However, this hasn't fallen off my radar entirely; if I do have time I will get back to you with a reproducer. Perhaps we can talk about it at PyCon.

  5. Armin Rigo

    Note that I've read somewhere that MAP_JIT suffers from the same issue than libffi's ffi_closure_alloc() functions, documented at :

    Note also that a cffi fix for the latter issue was
    attempted—see the ffi_closure_alloc branch—
    but was not merged because it creates 
    potential memory corruption with fork().

    It means that if you fork() and then the child continues to use any callback made in the parent, then you crash (in ways that are probably exploitable by a malicious actor). That's one of my favorites long-standing issues with the whole approach: it's all supposed to be a security hardening, but in one case, namely fork(), it actually opens the door widely to a whole new category of easy attacks.

  6. Armin Rigo

    To clarify: cffi already doesn't use libffi's page-mapping protocol, because it is on many platforms open to the fork() bug. It would be easy to just add MAP_JIT in the code inside cffi that calls mmap(), but doing so opens the code to the very same fork() bug. So I think there is no solution to this problem.

    I'm thinking about closing this issue and writing another line in the warning in the documentation, about needing

  7. Glyph reporter

    Hmm. This is unfortunate.

    On the one hand, Apple has been telegraphing for years that fork() is bad and they want to abandon it; their docs uniformly recommend NSTask, which, at a lower level, corresponds to posix_spawn.

    On the other, Apple has also hinted that while allow-jit is supported-ish (still not on iOS of course, but at least on macOS) allow-unsigned-executable-memory is a legacy catch-all entitlement that is present only for the duration of the migration to the more secure, more explicit allow-jit.

    I can see that some legacy unix applications are going to want to keep using fork() more or less forever, but if a more modern application wants to adopt posix_spawn() and not care about fork() bugs, is there some way for cffi to use a run-time switch to use MAP_JIT?

  8. Armin Rigo

    Basically, no. I refuse to add to cffi a feature that is billed as "more secure" when it is the exact opposite. You'll have to use custom-built versions of cffi if you really want to do that.

    Now of course, the real fix would be to stop using ffi.callback() and rely on the newer ffi.def_extern() mechanism. I'm talking about the OpenSSL/ in your traceback.

  9. Glyph reporter

    Aah, so if pyOpenSSL were to stop using these old-style callbacks then we would need neither the entitlement nor MAP_JIT? Sorry, I misunderstood the root cause here.

    That does indeed sound better.

  10. Armin Rigo

    I've got a potential idea to fix the exploitable issue added by MAP_JIT. All I need is to convince myself. The idea would be to keep a copy of the memory inside a MAP_JIT allocation, but not use it, just keep it around. Then we register a pthread_atfork() handler for the children, and in the child of a fork, we re-allocate all the MAP_JIT pages at the same place with the same flags, and re-initialize their content from the copy. If done correctly, this should give the same result as if the MAP_JIT pages survived a fork.

    The same idea may work on SELinux. If we were to use libffi's approach, then after fork() the memory would still be mmaped, but this memory is MAP_SHARED. The two processes can continue running more cffi code and each try to create some more cdata callback objects. They would likely be placed in free slots inside the same mmaped page... but it's the same actual MAP_SHARED page, so they would randomly overwrite each other. Same if the cdata callback objects are freed in one process, freeing and possibly overwriting bits of the page that are still in use in the other process. Here again, the potential fix idea would be to register a pthread_atfork() handler for the children, and re-allocate the pages as needed, still with the MAP_SHARED flag but not actually shared with the fork's parent; and re-initialize the content of these pages from a separate copy.

  11. Armin Rigo

    Still "on hold", and will likely remain on hold until someone who cares about these platforms contributes a fix. The condition for acceptance is that this fix must either work in the presence of os.fork(), or clearly raise a Python-level non-fatal exception in a fork child if and when we try to use the now-freed callback objects. It must NOT crash in an exploitable way if we try to do that.

  12. Log in to comment