warning about "module was already imported from"

Tarek Ziadé avatarTarek Ziadé created an issue

Number #36 on setuptools tracker. Needs to be fixed in 0.6 to avoid spurious warnings when using Disttribute

Comments (17)

  1. Reinout van Rees

    Similar problem with buildout.

    And it breaks one of my scripts when run inside buildout, as the script by necessity does a "python setup.py --name" to get at the name as defined in a setup.py in some package. The output of --name now includes those warnings....

  2. Tarek Ziadé
    • changed status to open
    • changed version to 0.6.3
    • changed milestone to 0.6.4

    Can I have your ouputs guys ? This warning is supposed not to pop any warning anymore when importing module/package names that are part of Distribute.

    Also, check if this is not related to an environment where you are using Setuptools, and where its was not properly patched because of the zc.buildout bootstrapping that hardcodes setuptools dependency.

  3. Reinout van Rees
    /home/zope/.buildout/eggs/setuptools-0.6c9-py2.5.egg/setuptools/command/sdist.py:4: UserWarning: Module pkg_resources was already imported from /home/zope/.buildout/eggs/setuptools-0.6c9-py2.5.egg/pkg_resources.py, but /home/zope/localpythonbuildout/python-2.5/lib/python2.5/site-packages/distribute-0.6.3-py2.5.egg is being added to sys.path
        import os, re, sys, pkg_resources
    /home/zope/.buildout/eggs/setuptools-0.6c9-py2.5.egg/setuptools/command/sdist.py:4: UserWarning: Module setuptools was already imported from /home/zope/.buildout/eggs/setuptools-0.6c9-py2.5.egg/setuptools/__init__.py, but /home/zope/localpythonbuildout/python-2.5/lib/python2.5/site-packages/distribute-0.6.3-py2.5.egg is being added to sys.path
        import os, re, sys, pkg_resources
    /home/zope/.buildout/eggs/setuptools-0.6c9-py2.5.egg/setuptools/command/sdist.py:4: UserWarning: Module site was already imported from /home/zope/localpythonbuildout/python-2.5/lib/python2.5/site.pyc, but /home/zope/localpythonbuildout/python-2.5/lib/python2.5/site-packages/distribute-0.6.3-py2.5.egg is being added to sys.path
        import os, re, sys, pkg_resources 
    

    I easy_installed distribute in the python2.5 I used.

    The buildout uses the regular (non-distribute) bootstrap, so there is still a setuptools egg in the path of the generated files:

    #!/usr/local/bin/python2.5
    
    import sys
    sys.path[0:0] = [
      '/home/zope/.buildout/eggs/zest.releaser-2.6-py2.5.egg',
      '/home/zope/.buildout/eggs/pyflakes-0.3.0-py2.5.egg',
      '/home/zope/.buildout/eggs/pep8-0.3.1-py2.5.egg',
      '/home/zope/.buildout/eggs/tha.coverage-0.1.1-py2.5.egg',
      '/srv/buildbot.thehealthagency.com/src',
      '/home/zope/.buildout/eggs/eazysvn-1.11.0-py2.5.egg',
      '/home/zope/.buildout/eggs/docutils-0.5-py2.5.egg',
      '/home/zope/.buildout/eggs/tha.dependencygraph-0.3-py2.5.egg',
      '/home/zope/.buildout/eggs/setuptools-0.6c9-py2.5.egg',
      '/home/zope/.buildout/eggs/z3c.coverage-1.1.3-py2.5.egg',
      '/home/zope/.buildout/eggs/tl.eggdeps-0.4-py2.5.egg',
      ]
    
    import buildbotmaster.documentation
    
    if __name__ == '__main__':
        buildbotmaster.documentation.main()
    

    I tried using the bootstrap you provided, but that seemed to hose my global python's setuptools installation. The bootstrap cannot write to the system directory, so you have to sudo, which lead to other problems. But I could not reliably debug that one as I had to focus on other bugs at that time. If I can reproduce, I'll open a separate bug for that.

    I've had the problem on two different machines: something automatically tried to exchange setuptools for distribute but failed halfway. I had to copy the old setuptools egg back in to be able to easy_install distribute... easy_install complained about something like "python -c xxxxxx not valid".

  4. Tarek Ziadé

    I've figured out that the buildout class in zc.buildout is rewriting sys.path even with our special bootstrap.py version because it doesn't detect that Distribute provides the packages it requires.

    I am currently working on a patch in zc.buildout to resolve this without making any Distribute-harcoded code. This is the patch I have done and seems to work fine together with http://bitbucket.org/tarek/buildout-distribute/ :

    ===================================================================
    --- src/zc/buildout/buildout.py (révision 104774)
    +++ src/zc/buildout/buildout.py (copie de travail)
    @@ -807,7 +807,7 @@
             upgraded = []
             for project in 'zc.buildout', 'setuptools':
                 req = pkg_resources.Requirement.parse(project)
    -            if ws.find(req) != pkg_resources.working_set.find(req):
    +            if ws.find(req).location != pkg_resources.working_set.find(req).location:
                     upgraded.append(ws.find(req))
     
             if not upgraded:
    

    But I need to do more tests, I'll try to finish this this week.

    If you try to apply manually this patch in the zc.buildout that gets in your eggs/ and if you run the script located in http://bitbucket.org/tarek/buildout-distribute/ it should work. Let me know if you try please, that could help providing a new zc.buildout release that works with Distribute, with a minimal impact.

  5. Reinout van Rees

    I'm trying out the bootstrap + buildout patch right now.

    But the problem remains with other buildouts: you don't control all buildouts you have. So buildout ought to work without warnings when you only exchange your global setuptools for distribute. Otherwise people get scared.

  6. Reinout van Rees

    The new bootstrap script plus the buildout patch (neat!) plus a stated dependency somewhere on distribute fixes the issue for that particular buildout.

    So: the problem is fixable for everything under your control but still needs fixing for a global distribute installation (if that is possible).

  7. Tarek Ziadé

    What do you mean by "stated dependency" ?

    the problem is fixable for everything under your control but still needs fixing for a global distribute installation (if that is possible).

    Unfortunately we can't avoid warnings on Setuptools side.

  8. Reinout van Rees

    Stated dependency: if I use the new bootstrap plus buildout but nothing actually depends on distribute (but something *does* depend on setuptools), distribute is not in the sys.path. And I get the warnings again.

    And something else is wrong, too: once distribute has changed the setuptools egg in my .buildout/eggs directory, every other buildout (which is unmodified or which has to distribute dependency) gets the warnings. (Probably as it falls back

    So something in the distribute replacement stuff isn't complete. If the distribute egg is not on your sys.path, the patched setuptools is ignored or not found and it -as far as I can see- falls back to the global setuptools. And if you haven't replaced that one with distribute, you again get the warnings.

    Which would mean "fix the warnings" or "wait with using distribute in buildouts until everyone has switched". And I'd like to switch earlier...

  9. Lennart Regebro

    Could having a virtualenv with --no-site-packages be enough? Then installing distribute there would override the setuptools, right? Not a perfect solution, but maybe it's a workaround.

  10. Reinout van Rees

    In the following, by "unpatched setuptools" I mean a setuptools egg that hasn't been replaced/mocked/whatever by distribute.

    I found out the one situation in which I get setuptools warnings when using buildout.

    I have an unpatched global setuptools and a patched setuptools in buildout's egg cache, but I don't use distribute myself. In that case, a setuptools import falls back on the global setuptools, but it does find some distribute here and there and issues warnings.

    When it goes right:

    a) If I have an unpatched global setuptools and a patched setuptools in the buildout egg cache and I also require distribute somewhere in my buildout, everything works just fine. The global setuptools is never seen and distribute correctly replaces setuptools at runtime.

    b) If I have a patched global setuptools and a patched setuptools in the buildout egg cache, everything goes OK even if I don't depend on distribute in my buildout.

    c) If I don't use a buildout egg cache and if my egg there has never been patched before (by experimenting) and left in place.

    For it to go wrong, you need

    - an unpatched global setuptools

    - a buildout egg cache that has a setuptools patched by some other buildout (or by yourself in case you've later removed a distribute dependency)

    - a buildout that effectively uses the patched setuptools egg, but doesn't depend on distribute.

    I really hope it still makes sense :-)

  11. Tarek Ziadé

    Stated dependency: if I use the new bootstrap plus buildout but nothing actually depends on distribute (but something *does* depend on setuptools), distribute is not in the sys.path. And I get the warnings again.

    No because the custom boostrap installs Distribute in your buildout.

    I really hope it still makes sense :-)

    yes that makes sense but unfortunately we can't handle these cases because you will always find combinations where a real setuptools might be located in the path. So I don't think we can get totally rid of those warnings.

  12. Log in to comment
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.