pypy / pypy / doc / faq.rst

 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 Carl Friedrich B… de55c45 2011-04-30 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 Armin Rigo b2c92d4 2011-12-11 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 Armin Rigo b2c92d4 2011-12-11 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 Dario Bertini 74b4a53 2011-04-25 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 holger krekel 8dfcde3 2011-05-09 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 Dario Bertini 5d78885 2011-04-25 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 David Malcolm 1e46012 2011-03-14 Armin Rigo 8eee94a 2011-09-02 Armin Rigo e9cb09c 2011-09-02 Armin Rigo 8eee94a 2011-09-02 Armin Rigo b8398cc 2011-09-02 David Malcolm 1e46012 2011-03-14 lac d38e306 2011-04-27 Carl Friedrich B… 541308f 2011-04-27 lac d38e306 2011-04-27  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 ========================== Frequently Asked Questions ========================== .. contents:: General ======= ------------- What is PyPy? ------------- PyPy is both: - a reimplementation of Python in Python, and - a framework for implementing interpreters and virtual machines for programming languages, especially dynamic languages. PyPy tries to find new answers about ease of creation, flexibility, maintainability and speed trade-offs for language implementations. For further details see our goal and architecture document_ . .. _goal and architecture document: architecture.html .. _drop in replacement: ------------------------------------------ Is PyPy a drop in replacement for CPython? ------------------------------------------ Almost! The mostly likely stumbling block for any given project is support for extension modules_. PyPy supports a continually growing number of extension modules, but so far mostly only those found in the standard library. The language features (including builtin types and functions) are very complete and well tested, so if your project does not use many extension modules there is a good chance that it will work with PyPy. We list the differences we know about in cpython differences_. -------------------------------------------- Do CPython Extension modules work with PyPy? -------------------------------------------- We have experimental support for CPython extension modules, so they run with minor changes. This has been a part of PyPy since the 1.4 release, but support is still in beta phase. CPython extension modules in PyPy are often much slower than in CPython due to the need to emulate refcounting. It is often faster to take out your CPython extension and replace it with a pure python version that the JIT can see. We fully support ctypes-based extensions. For information on which third party extensions work (or do not work) with PyPy see the compatibility wiki_. .. _extension modules: cpython_differences.html#extension-modules .. _cpython differences: cpython_differences.html .. _compatibility wiki: https://bitbucket.org/pypy/compatibility/wiki/Home --------------------------------- On which platforms does PyPy run? --------------------------------- PyPy is regularly and extensively tested on Linux machines and on Mac OS X and mostly works under Windows too (but is tested there less extensively). PyPy needs a CPython running on the target platform to bootstrap, as cross compilation is not really meant to work yet. At the moment you need CPython 2.5 - 2.7 for the translation process. PyPy's JIT requires an x86 or x86_64 CPU. ------------------------------------------------ Which Python version (2.x?) does PyPy implement? ------------------------------------------------ PyPy currently aims to be fully compatible with Python 2.7. That means that it contains the standard library of Python 2.7 and that it supports 2.7 features (such as set comprehensions). .. _threading: ------------------------------------------------- Does PyPy have a GIL? Why? ------------------------------------------------- Yes, PyPy has a GIL. Removing the GIL is very hard. The first problem is that our garbage collectors are not re-entrant. ------------------------------------------ How do I write extension modules for PyPy? ------------------------------------------ See Writing extension modules for PyPy__. .. __: extending.html ----------------- How fast is PyPy? ----------------- This really depends on your code. For pure Python algorithmic code, it is very fast. For more typical Python programs we generally are 3 times the speed of Cpython 2.6 . You might be interested in our benchmarking site_ and our jit documentation_. Note that the JIT has a very high warm-up cost, meaning that the programs are slow at the beginning. If you want to compare the timings with CPython, even relatively simple programs need to run *at least* one second, preferrably at least a few seconds. Large, complicated programs need even more time to warm-up the JIT. .. _benchmarking site: http://speed.pypy.org .. _jit documentation: jit/index.html --------------------------------------------------------------- Couldn't the JIT dump and reload already-compiled machine code? --------------------------------------------------------------- No, we found no way of doing that. The JIT generates machine code containing a large number of constant addresses --- constant at the time the machine code is written. The vast majority is probably not at all constants that you find in the executable, with a nice link name. E.g. the addresses of Python classes are used all the time, but Python classes don't come statically from the executable; they are created anew every time you restart your program. This makes saving and reloading machine code completely impossible without some very advanced way of mapping addresses in the old (now-dead) process to addresses in the new process, including checking that all the previous assumptions about the (now-dead) object are still true about the new object. .. _prolog and javascript: -------------------------------------------------------------------------- Can I use PyPy's translation toolchain for other languages besides Python? -------------------------------------------------------------------------- Yes. The toolsuite that translates the PyPy interpreter is quite general and can be used to create optimized versions of interpreters for any language, not just Python. Of course, these interpreters can make use of the same features that PyPy brings to Python: translation to various languages, stackless features, garbage collection, implementation of various things like arbitrarily long integers, etc. Currently, we have preliminary versions of a JavaScript interpreter (Leonardo Santagada as his Summer of PyPy project), a Prolog interpreter_ (Carl Friedrich Bolz as his Bachelor thesis), and a SmallTalk interpreter_ (produced during a sprint). On the PyPy bitbucket page_ there is also a Scheme and an Io implementation; both of these are unfinished at the moment. .. _Prolog interpreter: https://bitbucket.org/cfbolz/pyrolog/ .. _SmallTalk interpreter: http://dx.doi.org/10.1007/978-3-540-89275-5_7 .. _PyPy bitbucket page: https://bitbucket.org/pypy/ Development =========== ----------------------------------------------------------- How do I get into PyPy development? Can I come to sprints? ----------------------------------------------------------- Certainly you can come to sprints! We always welcome newcomers and try to help them as much as possible to get started with the project. We provide tutorials and pair them with experienced PyPy developers. Newcomers should have some Python experience and read some of the PyPy documentation before coming to a sprint. Coming to a sprint is usually the best way to get into PyPy development. If you get stuck or need advice, contact us_. IRC is the most immediate way to get feedback (at least during some parts of the day; most PyPy developers are in Europe) and the mailing list_ is better for long discussions. .. _contact us: index.html .. _mailing list: http://python.org/mailman/listinfo/pypy-dev ------------------------------------------------------------- OSError: ... cannot restore segment prot after reloc... Help? ------------------------------------------------------------- On Linux, if SELinux is enabled, you may get errors along the lines of "OSError: externmod.so: cannot restore segment prot after reloc: Permission denied." This is caused by a slight abuse of the C compiler during configuration, and can be disabled by running the following command with root privileges:: # setenforce 0 This will disable SELinux's protection and allow PyPy to configure correctly. Be sure to enable it again if you need it! The PyPy translation tool chain =============================== --------------------------------------------- Can PyPy compile normal Python programs to C? --------------------------------------------- No, PyPy is not a Python compiler. In Python, it is mostly impossible to *prove* anything about the types that a program will manipulate by doing a static analysis. It should be clear if you are familiar with Python, but if in doubt see [BRETT]_. If you want a fast Python program, please use our JIT_ instead. .. _JIT: jit/index.html .. [BRETT] Brett Cannon, Localized Type Inference of Atomic Types in Python, http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.90.3231 .. _PyPy's RPython: ------------------------------ What is this RPython language? ------------------------------ RPython is a restricted subset of the Python language. It is used for implementing dynamic language interpreters within the PyPy toolchain. The restrictions ensure that type inference (and so, ultimately, translation to other languages) of RPython programs is possible. The property of "being RPython" always applies to a full program, not to single functions or modules (the translation toolchain does a full program analysis). The translation toolchain follows all calls recursively and discovers what belongs to the program and what does not. RPython program restrictions mostly limit the ability to mix types in arbitrary ways. RPython does not allow the binding of two different types in the same variable. In this respect (and in some others) it feels a bit like Java. Other features not allowed in RPython are the use of special methods (__xxx__) except __init__ and __del__, and the use of reflection capabilities (e.g. __dict__). You cannot use most existing standard library modules from RPython. The exceptions are some functions in os, math and time that have native support. To read more about the RPython limitations read the RPython description_. .. _RPython description: coding-guide.html#restricted-python --------------------------------------------------------------- Does RPython have anything to do with Zope's Restricted Python? --------------------------------------------------------------- No. Zope's RestrictedPython_ aims to provide a sandboxed execution environment for CPython. PyPy's RPython_ is the implementation language for dynamic language interpreters. However, PyPy also provides a robust sandboxed Python Interpreter_. .. _sandboxed Python Interpreter: sandbox.html .. _Zope's RestrictedPython: http://pypi.python.org/pypi/RestrictedPython ------------------------------------------------------ What's the "NOT_RPYTHON" I see in some docstrings? ------------------------------------------------------ If you put "NOT_RPYTHON" into the docstring of a function and that function is found while trying to translate an RPython program, the translation process stops and reports this as an error. You can therefore mark functions as "NOT_RPYTHON" to make sure that they are never analyzed. ------------------------------------------------------------------- Couldn't we simply take a Python syntax tree and turn it into Lisp? ------------------------------------------------------------------- It's not necessarily nonsense, but it's not really The PyPy Way. It's pretty hard, without some kind of type inference, to translate this Python:: a + b into anything significantly more efficient than this Common Lisp:: (py:add a b) And making type inference possible is what RPython is all about. You could make #'py:add a generic function and see if a given CLOS implementation is fast enough to give a useful speed (but I think the coercion rules would probably drive you insane first). -- mwh -------------------------------------------- Do I have to rewrite my programs in RPython? -------------------------------------------- No. And you shouldn't try. PyPy always runs your code in its own interpreter, which is a full and compliant Python 2.7 interpreter. RPython is only the language in which parts of PyPy itself are written and extension modules for it. Not only is it not necessary for you to rewrite your code in RPython, it probably won't give you any speed improvements if you try. --------------------------------------------------- Which backends are there for the RPython toolchain? --------------------------------------------------- Currently, there are backends for C_, the CLI_, and the JVM_. All of these can translate the entire PyPy interpreter. To learn more about backends take a look at the translation document_. .. _C: translation.html#the-c-back-end .. _CLI: cli-backend.html .. _JVM: translation.html#genjvm .. _translation document: translation.html ---------------------- How do I compile PyPy? ---------------------- See the getting-started_ guide. .. _getting-started: getting-started-python.html .. _how do I compile my own interpreters: ------------------------------------- How do I compile my own interpreters? ------------------------------------- Begin by reading Andrew Brown's tutorial_ . .. _Andrew Brown's tutorial: http://morepypy.blogspot.com/2011/04/tutorial-writing-interpreter-with-pypy.html --------------------------------------------------------- Can RPython modules for PyPy be translated independently? --------------------------------------------------------- No, you have to rebuild the entire interpreter. This means two things: * It is imperative to use test-driven development. You have to test exhaustively your module in pure Python, before even attempting to translate it. Once you translate it, you should have only a few typing issues left to fix, but otherwise the result should work out of the box. * Second, and perhaps most important: do you have a really good reason for writing the module in RPython in the first place? Nowadays you should really look at alternatives, like writing it in pure Python, using ctypes if it needs to call C code. Other alternatives are being developed too (as of summer 2011), like a Cython binding. In this context it is not that important to be able to translate RPython modules independently of translating the complete interpreter. (It could be done given enough efforts, but it's a really serious undertaking. Consider it as quite unlikely for now.) ---------------------------------------------------------- Why does PyPy draw a Mandelbrot fractal while translating? ---------------------------------------------------------- Because it's fun. .. include:: _ref.txt 
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.