Python 3 has a stable ABI. https://www.python.org/dev/peps/pep-0384/. It could be interesting for cffi to have a mode to take advantage of it. Then one could distribute a single cffi module for several Python 3 versions.
Did you try? Because of the potential mess involved, I won't go there without someone checking the basics first. I'm afraid we're depending on specific CPython versions at some places, because of an unstability of details from the official API; these details are probably not part of the stable ABI at all, but I don't know (search for VERSION in misc_thread_common.h).
Sorry, I just realized another interpretation of what you said: maybe it is not about _cffi_backend, but about the C extension modules produced by set_source() / compile(). Maybe you're saying that it would be useful if these modules were using only the limited API.
This is easy because these modules use only a very limited subset of the API anyway, at least for the non-embedding mode. I've fixed the only place where they use a "non-limited" API in 1629330be7e6.
So it seems to be done, but distutils still produces names like foo.cpython-35m-x86_64-linux-gnu.so. I guess we can manually rename them, but isn't there some more official way?
I see that distutils is half-deprecated, but I can't depend on another third-party library inside CFFI. If there is no standard solution, then I guess I can just document the presence of the Py_LIMITED_API flag. That means it is up to individual CFFI module authors to use other tools to compile the CFFI-produced C files. I'd still like to know if just renaming the final .so is enough or not, and what it should be renamed to.
In:fromdistutils.sysconfigimportget_config_varIn:get_config_var('EXT_SUFFIX')Out:'.cpython-35m-darwin.so'distutils.command.build_ext:defget_ext_filename(self,ext_name):r"""Convert the name of an extension (eg. "foo.bar") into the name of the file from which it will be loaded (eg. "foo/bar.so", or "foo\bar.pyd"). """fromdistutils.sysconfigimportget_config_varext_path=ext_name.split('.')ext_suffix=get_config_var('EXT_SUFFIX')returnos.path.join(*ext_path)+ext_suffix
Is there any documentation or standard example of compiling a CPython extension module that contains #define Py_LIMITED_API 1 into the expected file name? Or are you saying that there is nothing better than: "we need to patch distutils by following the instructions given above"?
Haven't seen any. At least you can pass defines to distutils without patching. Also, py hex version -DPy_LIMITED_API=0x03030000 is better than 1, you get the limited API from that version 3.3 onward, it has more functions.
Unfortunately the PEP itself says that no support will be added to distutils for compiling these extensions "due to the distutils code freeze". I doubt that there is a better way than patching / extending distutils by overriding the build_ext command, but, changing command classes is the way to extend distutils.
I'm -0.5 about adding a feature to CFFI that has nothing to do with CFFI. If there is no officially-approved way neither in distutils nor in another widely-used .so-maker for CPython, then I think I'll not do more than emit the #define, and mention in the documentation that people can look around for 3rd-party ways to compile this C code.
Thank you for taking such a thorough look into the issue. Unfortunately as far as I can tell despite being a feature since Python 3.2 Py_LIMITED_API is a bit of an uncharted territory, but with any luck it can become less so, and reduce the burden of distributing cffi extensions for Windows. It is completely satisfying for cffi to stick to generating the code, and for some other software to worry about the matter of compiling and naming the artifact.