Issue #131 resolved

ImportMismatchError with namespace packages

Jason R. Coombs
created an issue

Similar to issue #116, I'm getting an an ImportMismatchError when running with doctest-modules, but only for namespace packages. I suspect this issue is going to be harder to resolve, as namespace packages are still a hack in Python.

It appears the problem lies in the check for uniqueness of modules, but for namespace packages, all of the namespace packages will resolve to the same name. For example, if one has a project 'myns.projA' and another 'myns.projB', the module 'myns' is actually represented in multiple locations.

Maybe it's not too hard to detect a namespace package and not perform the uniqueness test on it?

Here's how to reproduce in Python 2.6 on Ubuntu with py==1.3.4 and distribute 0.6.14.

easy_install path.py (required to run namespace.py)

wget http://bitbucket.org/jaraco/jaraco.develop/raw/tip/jaraco/develop/namespace.py

Run namespace.py. This creates two namespace packages, myns.projA and myns.projB.

pushd myns.projA; python setup.py develop; popd

pushd myns.projB; python setup.py develop

py.test --doctest-modules

You should get output similar to the following:

{{{

!shell

jaraco@livid:~/myns.projB$ py.test --doctest-modules ============================= test session starts ============================== platform linux2 -- Python 2.6.5 -- pytest-1.3.4 test path 1: /home/jaraco/myns.projB

myns/init.py F myns/projB/init.py . setup.py . test/init.py . test/test_basic.py ..

=================================== FAILURES ===================================


self = <CallInfo when='call' exception: ('myns', '/home/jaraco/myns.projA/myns/init.py', local('/home/jaraco/myns.projB/myns/init.py'))> func = <function <lambda> at 0x27eb0c8>, when = 'call'

def __init__(self, func, when):
    self.when = when
    try:
      self.result = func()

/usr/local/lib/python2.6/dist-packages/py-1.3.4-py2.6.egg/py/_plugin/pytest_runner.py:104:


return CallInfo(lambda: ihook(item=item), when=when)

/usr/local/lib/python2.6/dist-packages/py-1.3.4-py2.6.egg/py/_plugin/pytest_runner.py:97:


def call_matching_hooks(**kwargs):
    plugins = self.node.config._getmatchingplugins(self.node.fspath)
  return hookmethod.pcall(plugins, **kwargs)

/usr/local/lib/python2.6/dist-packages/py-1.3.4-py2.6.egg/py/_test/collect.py:21:


self = <HookCaller 'pytest_runtest_call'> plugins = [<py._test.pluginmanager.PluginManager object at 0x245af50>, <module 'py._plugin.pytest_default' from '/usr/local/lib/..._plugin.pytest_mark' from '/usr/local/lib/python2.6/dist-packages/py-1.3.4-py2.6.egg/py/_plugin/pytest_mark.pyc'>;, ...]

def pcall(self, plugins, **kwargs):
    methods = self.hookrelay._registry.listattr(self.name, plugins=plugins)
    mc = MultiCall(methods, kwargs, firstresult=self.firstresult)
  return self.hookrelay._performcall(self.name, mc)

/usr/local/lib/python2.6/dist-packages/py-1.3.4-py2.6.egg/py/_test/pluginmanager.py:354:


self = <py._test.pluginmanager.HookRelay instance at 0x2414878> name = 'pytest_runtest_call' multicall = <MultiCall 0 results, 0 meths, kwargs={'item': <DoctestModule 'DoctestModule:init.py'>, 'multicall': <MultiCall 0 results, 0 meths, kwargs={...}>}>

def _performcall(self, name, multicall):
  return multicall.execute()

/usr/local/lib/python2.6/dist-packages/py-1.3.4-py2.6.egg/py/_test/pluginmanager.py:335:


self = <MultiCall 0 results, 0 meths, kwargs={'item': <DoctestModule 'DoctestModule:init.py'>, 'multicall': <MultiCall 0 results, 0 meths, kwargs={...}>}>

def execute(self):
    while self.methods:
        method = self.methods.pop()
        kwargs = self.getkwargs(method)
      res = method(**kwargs)

/usr/local/lib/python2.6/dist-packages/py-1.3.4-py2.6.egg/py/_test/pluginmanager.py:244:


item = <DoctestModule 'DoctestModule:init.py'>

def pytest_runtest_call(item):
    if not item._deprecated_testexecution():
      item.runtest()

/usr/local/lib/python2.6/dist-packages/py-1.3.4-py2.6.egg/py/_plugin/pytest_runner.py:58:


self = <DoctestModule 'DoctestModule:init.py'>

def runtest(self):
  module = self.fspath.pyimport()

/usr/local/lib/python2.6/dist-packages/py-1.3.4-py2.6.egg/py/_plugin/pytest_doctest.py:98:


self = local('/home/jaraco/myns.projB/myns/init.py'), modname = 'myns' ensuresyspath = True

def pyimport(self, modname=None, ensuresyspath=True):
    """ return path as an imported python module.
            if modname is None, look for the containing package
            and construct an according module name.
            The module will be put/looked up in sys.modules.
        """
    if not self.check():
        raise py.error.ENOENT(self)
    #print "trying to import", self
    pkgpath = None
    if modname is None:
        pkgpath = self.pypkgpath()
        if pkgpath is not None:
            if ensuresyspath:
                self._prependsyspath(pkgpath.dirpath())
            pkg = __import__(pkgpath.basename, None, None, [])
            names = self.new(ext='').relto(pkgpath.dirpath())
            names = names.split(self.sep)
            if names and names[-1] == "__init__":
                names.pop()
            modname = ".".join(names)
        else:
            # no package scope, still make it possible
            if ensuresyspath:
                self._prependsyspath(self.dirpath())
            modname = self.purebasename
        mod = __import__(modname, None, None, ['__doc__'])
        modfile = mod.__file__
        if modfile[-4:] in ('.pyc', '.pyo'):
            modfile = modfile[:-1]
        elif modfile.endswith('$py.class'):
            modfile = modfile[:-9] + '.py'
        if modfile.endswith("__init__.py"):
            if self.basename != "__init__.py":
                modfile = modfile[:-12]
        if not self.samefile(modfile):
          raise self.ImportMismatchError(modname, modfile, self)

E ImportMismatchError: ('myns', '/home/jaraco/myns.projA/myns/init.py', local('/home/jaraco/myns.projB/myns/init.py'))

/usr/local/lib/python2.6/dist-packages/py-1.3.4-py2.6.egg/py/_path/local.py:540: ImportMismatchError ====================== 1 failed, 5 passed in 0.08 seconds ======================

}}}

Comments (3)

  1. holger krekel repo owner
    • changed status to open

    thanks for the good error report! I am not totally sure how to best fix it. For now i just skip the importmismatch check if we are importing an "init.py" file. Could you install (best in a fresh virtual environment)

    pip install -U pytest -i http://pypi.testrun.org

    to see if it fixes your problem? You might still get problems with importing "setup.py" - in my test dir i created a conftest.py with this content:

    option_doctestmodules=True
    collect_ignore=["myns.projA/setup.py", "myns.projB/setup.py"]
    

    so that i can just type "py.test".

    Works for you as well?

    Holger

  2. Log in to comment