# pypy / pypy / doc / ctypes-implementation.rst

 David Malcolm 1e46012 2011-03-14 ModCloth 6971524 2013-02-22 David Malcolm 1e46012 2011-03-14 lac b3a9ff6 2011-04-29 David Malcolm 1e46012 2011-03-14 Maciej Fijalkows… d526611 2013-02-06 David Malcolm 1e46012 2011-03-14 Armin Rigo a53ca6d 2011-05-07 David Malcolm 1e46012 2011-03-14 Armin Rigo a53ca6d 2011-05-07   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 ============================= PyPy's ctypes implementation ============================= Summary ======== Terminology: * application level code - code written in full Python * interpreter level code - code written in RPython, compiled to something else, say C, part of the interpreter. PyPy's ctypes implementation in its current state proves the feasibility of implementing a module with the same interface and behavior for PyPy as ctypes for CPython. PyPy's implementation internally uses libffi_ like CPython's ctypes. In our implementation as much as possible of the code is written in full Python, not RPython. In CPython's situation, the equivalent would be to write as little as possible code in C. We essentially favored rapid experimentation over worrying about speed for this first trial implementation. This allowed to provide a working implementation with a large part of ctypes features in 2 months real time. We reused the ctypes package version 1.0.2 as-is from CPython. We implemented _ctypes which is a C module in CPython mostly in pure Python based on a lower-level layer extension module _rawffi. .. _libffi: http://sources.redhat.com/libffi/ Low-level part: _rawffi ============================ This PyPy extension module (pypy/module/_rawffi) exposes a simple interface to create C objects (arrays and structures) and calling functions in dynamic libraries through libffi. Freeing objects in most cases and making sure that objects referring to each other are kept alive is responsibility of the higher levels. This module uses bindings to libffi which are defined in rpython/rlib/libffi.py. We tried to keep this module as small as possible. It is conceivable that other implementations (e.g. Jython) could use our ctypes implementation by writing their version of _rawffi. High-level parts ================= The reused ctypes package lives in lib_pypy/ctypes. _ctypes implementing the same interface as _ctypes in CPython is in lib_pypy/_ctypes. Discussion and limitations ============================= Reimplementing ctypes features was in general possible. PyPy supports pluggable garbage collectors, some of them are moving collectors, this means that the strategy of passing direct references inside Python objects to an external library is not feasible (unless the GCs support pinning, which is not the case right now). The consequence of this is that sometimes copying instead of sharing is required, this may result in some semantics differences. C objects created with _rawffi itself are allocated outside of the GC heap, such that they can be passed to external functions without worries. Porting the implementation to interpreter-level should likely improve its speed. Furthermore the current layering and the current _rawffi interface require more object allocations and copying than strictly necessary; this too could be improved. Here is a list of the limitations and missing features of the current implementation: * No support for PyXxx functions from libpython, for obvious reasons. * We copy Python strings instead of having pointers to raw buffers * Features we did not get to implement: - custom alignment and bit-fields - resizing (resize() function) - non-native byte-order objects - callbacks accepting by-value structures - slight semantic differences that ctypes makes between its primitive types and user subclasses of its primitive types Running application examples ============================== pyglet_ is known to run. We also had some success with pygame-ctypes (which is no longer maintained) and with a snapshot of the experimental pysqlite-ctypes. We will only describe how to run the pyglet examples. pyglet ------- We tried pyglet checking it out from its repository at revision 1984. From pyglet, the following examples are known to work: - opengl.py - multiple_windows.py - events.py - html_label.py - timer.py - window_platform_event.py - fixed_resolution.py The pypy-c translated to run the ctypes tests can be used to run the pyglet examples as well. They can be run like e.g.:: $cd pyglet/$ PYTHONPATH=. ../ctypes-stable/pypy/goal/pypy-c examples/opengl.py they usually should be terminated with ctrl-c. Refer to the their doc strings for details about how they should behave. The following examples don't work for reasons independent from ctypes: - image_convert.py needs PIL - image_display.py needs PIL - astraea/astraea.py needs PIL We did not try the following examples: - media_player.py needs avbin or at least a proper sound card setup for .wav files - video.py needs avbin - soundscape needs avbin .. _pyglet: http://pyglet.org/ ctypes configure ================= We also released ctypes-configure, which is an experimental package trying to approach the portability issues of ctypes-based code. idea ---- One of ctypes problems is that ctypes programs are usually not very platform-independent. We created ctypes_configure, which invokes c compiler (via distutils) for various platform-dependent details like exact sizes of types (for example size_t), #defines, exact outline of structures etc. It replaces in this regard code generator (h2py). installation ------------ easy_install ctypes_configure usage ----- ctypes_configure/doc/sample.py_ explains in details how to use it. .. include:: _ref.txt 
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.