RPython modules no longer have __file__, but their functions have func_code.co_filename

Issue #1885 resolved
Ned Batchelder created an issue

In PyPy 2.3, modules like _structseq had a __file__ attribute with build-system file paths in them, like '/Users/kostia/programming/pypy/lib_pypy/_structseq.pyc'. This was weird:

$ pypy2.3
Python 2.7.6 (32f35069a16d, Jun 06 2014, 20:12:47)
[PyPy 2.3.1 with GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.2.79)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>> import _structseq
>>>> _structseq.__file__
'/Users/kostia/programming/pypy/lib_pypy/_structseq.pyc'

In PyPy 2.4, that attribute is gone, but you can still find paths like that in func_code.co_filename:

$ pypy2.4
Python 2.7.8 (f5dcc2477b97, Sep 19 2014, 18:09:54)
[PyPy 2.4.0 with GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.38)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>> import _structseq
>>>> _structseq.__file__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute '__file__'
>>>> _structseq.structseq_new.func_code.co_filename
'/Users/kostia/programming/pypy/lib_pypy/_structseq.py'

This mattered to coverage.py, because it was excluding the PyPy stdlib modules based on where _structseq was from. Without a __file__ attribute, coverage wasn't excluding the stdlib modules, and _structseq would be measured, but would then fail during reporting when coverage.py couldn't read the co_filename path to produce the report.

Coverage.py is now fixed with this: https://bitbucket.org/ned/coveragepy/commits/789cc15f04ccb46b53e96e5ffd3debbfbc1defaa , but Alex asked that I write a bug, and I do what Alex asks me to! :)

Comments (9)

  1. Armin Rigo

    The same issue can be seen in all our functions that are (1) translated inside PyPy, but (2) actually written as pure Python functions. For example, map: it says it is a built-in function, but it has a map.func_code which is a regular code object, including a co_filename attribute that records the original file location during the build.

    What do we want to do against it? Patch all co_filename of all code objects as they are translated together with PyPy? Replace them with some dummy value like <builtin>? I'm not sure it would have helped coverage.py, as the reporter would then complain about not finding the file called <builtin>, but it's the best I can come up with...

  2. Ned Batchelder reporter

    As I said, coverage.py is all set now, but if "<builtin>" were in the co_filename, it would recognize that as not reportable, and would deal with it gracefully.

    In my eyes, the current behavior is not a problem in and of itself. It's odd, but PyPy is different than CPython.

  3. mattip

    we should not be coding the filename of the original file location during build. <builtin> would be better, best would be to somehow patch the runtime location of lib_pypy into the co_filename, IMO

  4. Armin Rigo

    (It's not only about lib_pypy, it's about the pypy/module/*/app_*.py modules, whose sources are not included in a standard binary distribution.)

  5. Armin Rigo

    Fixed in f3d8768ebb1c: all functions that are pre-translated inside a PyPy should now have a func_code.co_filename of the form: <builtin>/lastdirname/filename.py, with the exact string <builtin> first, and still some directory info following it (the last directory component and the base filename), just in case we need to ever figure out where a function really comes from.

  6. Abdeali Kothari

    Hi, I had the same issue with coverage. Any estimated date when the next release will occur with this fixed for pypy3 ?

  7. Log in to comment