egg __spec__ is `None` if queried while importing using `importlib.util.find_spec`

Create issue
Issue #3034 resolved
Anthony Sottile created an issue

this is a bit of a rabbit hole, apologies – originally from https://github.com/pallets/flask/issues/3278

An egg which queries itself using importlib.util.find_spec is unable to retrieve its own information. (This works fine for normal modules it appears)

Here’s my minimal setup, shoved into one script:

import os.path
import tempfile
import subprocess
import sys

with tempfile.TemporaryDirectory() as t:
    with open(os.path.join(t, 'setup.py'), 'w') as setup:
        setup.write(
            'from setuptools import setup\n'
            'setup(packages=["site_egg"], zip_safe=True)'
        )
    os.mkdir(os.path.join(t, 'site_egg'))
    with open(os.path.join(t, 'site_egg/__init__.py'), 'w') as f:
        f.write(
            'import importlib.util\n'
            'assert importlib.util.find_spec(__name__) is not None\n'
        )
    open(os.path.join(t, 'site_egg/__init__.py'), 'a').close()

    subprocess.check_call(
        (sys.executable, 'setup.py', 'bdist_egg'),
        stdout=subprocess.DEVNULL,
        cwd=t,
    )

    egg, = os.listdir(os.path.join(t, 'dist'))
    sys.path.insert(0, os.path.join(t, 'dist', egg))

    import site_egg

in cpython I get the following output:

$ python3.6 t.py
$

However with pypy3 I get:

$ pypy3 t.py
Traceback (most recent call last):
  File "t.py", line 29, in <module>
    import site_egg
  File "/tmp/tmp1c9axaoe/dist/UNKNOWN-0.0.0-py3.6.egg/site_egg/__init__.py", line 2, in <module>
  File "/tmp/y/venvpp/lib-python/3/importlib/util.py", line 102, in find_spec
    raise ValueError('{}.__spec__ is None'.format(name))
ValueError: site_egg.__spec__ is None

$ pypy3 --version --version
Python 3.6.1 (784b254d6699, Apr 14 2019, 10:22:42)
[PyPy 7.1.1-beta0 with GCC 6.2.0 20160901]

Comments (3)

  1. Armin Rigo

    It seems to be a problem with the zip importer. Reduced steps:

    1. make foo.py with the content: print(__spec__)

    2. zip foo.zip foo.py and remove the original foo.py

    3. PYTHONPATH=foo.zip pypy3 -c "import foo"

    This prints None with pypy3, but on CPython 3.6 I get ModuleSpec(name='foo', loader=<zipimporter object "/tmp/z/foo.zip">, origin='/tmp/z/foo.zip/foo.py').

  2. Armin Rigo

    The issue might also come from the fact that CPython's Python/import.c code calls the Python function importlib._bootstrap_external._fix_up_module() at some point, but I found no equivalent code inside pypy.

  3. Log in to comment