Issue #142 resolved

easy_install will install a package that is already there

Zooko O'Whielacronx
created an issue

(This was originally http://bugs.python.org/setuptools/issue17 .)

It seems like distribute has a change of behavior wrt to setuptools-0.6c10 which might be a regression. agronholm on #distutils tried the minimal test case from http://bugs.python.org/setuptools/msg325 with distribute-0.6.10 and distribute-0.6.10 failed: {{{

python test2.py develop Yes, pkg_resources says that pywin32 is already installed. running develop running egg_info creating mindeponpywin32.egg-info writing requirements to mindeponpywin32.egg-info\requires.txt writing mindeponpywin32.egg-info\PKG-INFO writing top-level names to mindeponpywin32.egg-info\top_level.txt writing dependency_links to mindeponpywin32.egg-info\dependency_links.txt writing manifest file 'mindeponpywin32.egg-info\SOURCES.txt' reading manifest file 'mindeponpywin32.egg-info\SOURCES.txt' writing manifest file 'mindeponpywin32.egg-info\SOURCES.txt' running build_ext Creating c:\python26\lib\site-packages\mindeponpywin32.egg-link (link to .) Adding mindeponpywin32 0.0.0 to easy-install.pth file

Installed c:\users\demi-god Processing dependencies for mindeponpywin32==0.0.0 Searching for pywin32 Reading http://pypi.python.org/simple/pywin32/ Reading http://sf.net/projects/pywin32 Reading http://sourceforge.net/project/showfiles.php?group_id=78018 No local packages or download links found for pywin32 error: Could not find suitable distribution for Requirement.parse('pywin32') }}} (I've also already reported this on http://bitbucket.org/tarek/distribute/issue/141/easy_install-pywin32-binary-succeeds-and-then-says-it-cant-find .) Then Zart on #distutils tried a different experiment using virtualenv which shows that distribute re-installs pywin32 while setuptools uses the existing pywin32: {{{ D:>ver

Microsoft Windows [Version 6.1.7600]

D:>mkdir test

D:>cd test D:\test>\Shared\Egg\virtualenv-1.4.6\virtualenv.py VENV-S New python executable in VENV-S\Scripts\python.exe Installing setuptools.................done.

D:\test>\Shared\Egg\virtualenv-1.4.6\virtualenv.py --distribute VENV-D New python executable in VENV-D\Scripts\python.exe Installing distribute................................................................................................................................................................................

D:\test>type setup.py from setuptools import setup setup(name='foo', version='1.0', install_requires=['pywin32'])

D:\test>VENV-S\Scripts\python.exe setup.py develop running develop running egg_info creating foo.egg-info writing requirements to foo.egg-info\requires.txt writing foo.egg-info\PKG-INFO writing top-level names to foo.egg-info\top_level.txt writing dependency_links to foo.egg-info\dependency_links.txt writing manifest file 'foo.egg-info\SOURCES.txt' reading manifest file 'foo.egg-info\SOURCES.txt' writing manifest file 'foo.egg-info\SOURCES.txt' running build_ext Creating d:\test\venv-s\lib\site-packages\foo.egg-link (link to .) Adding foo 1.0 to easy-install.pth file

Installed d:\test Processing dependencies for foo==1.0 Searching for pywin32==214 Best match: pywin32 214 Adding pywin32 214 to easy-install.pth file

Using c:\program files (x86)\python26\lib\site-packages Finished processing dependencies for foo==1.0

D:\test>VENV-D\Scripts\python.exe setup.py develop running develop running egg_info creating foo.egg-info writing requirements to foo.egg-info\requires.txt writing foo.egg-info\PKG-INFO writing top-level names to foo.egg-info\top_level.txt writing dependency_links to foo.egg-info\dependency_links.txt writing manifest file 'foo.egg-info\SOURCES.txt' reading manifest file 'foo.egg-info\SOURCES.txt' writing manifest file 'foo.egg-info\SOURCES.txt' running build_ext Creating d:\test\venv-d\lib\site-packages\foo.egg-link (link to .) Adding foo 1.0 to easy-install.pth file

Installed d:\test Processing dependencies for foo==1.0 Searching for pywin32==214 Best match: pywin32 214 Processing pywin32-214.win32-py2.6.exe pywintypes: module references file win32evtlogutil: module references file win32serviceutil: module references file win32timezone: module references file Demos.win32rcparser_demo: module references file Demos.security.sspi.validate_password: module references file isapi.install: module references file isapi.samples.advanced: module references file isapi.samples.test: module references file isapi.test.setup: module references file pywin.framework.scriptutils: module references file pywin.framework.startup: module references path pywin.scintilla.config: module references path pywin.scintilla.control: module references file pywin.scintilla.IDLEenvironment: module references file scripts.regsetup: module references file scripts.setup_d: module references file test.testall: module references file test.test_clipboard: module references file test.test_win32api: module references file test.test_win32rcparser: module references file D:\test\VENV-D\lib\site-packages\distribute-0.6.10-py2.6.egg\setuptools\command\bdist_egg.py:422: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal symbols = dict.fromkeys(iter_symbols(code)) test.test_win32trace: module references file win32com.init: module references path win32com.server.register: module references file win32com.server.register: module references path win32com.test.GenTestScripts: module references path win32com.test.pippo_server: module references file win32com.test.testall: module references file win32com.test.testAXScript: module references path win32com.test.testDictionary: module references file win32com.test.testPippo: module references file win32com.test.testPyComTest: module references file win32com.test.testServers: module references file win32com.test.util: module references file win32comext.adsi.init: module references path win32comext.authorization.init: module references path win32comext.axcontrol.init: module references path win32comext.axdebug.codecontainer: module references file win32comext.axdebug.debugger: module references file win32comext.axdebug.init: module references path win32comext.axscript.init: module references path win32comext.bits.init: module references path win32comext.directsound.init: module references path win32comext.directsound.test.ds_test: module references file win32comext.internet.init: module references path win32comext.mapi.init: module references path win32comext.shell.init: module references path win32comext.shell.demos.servers.empty_volume_cache: module references file win32comext.shell.demos.servers.folder_view: module references file win32comext.taskscheduler.init: module references path creating 'c:\users\zart\appdata\local\temp\1\easy_install-myvgyz\pywin32-214-py2.6-win32.egg' and adding 'c:\users\zart\appdata\local\temp\1\easy_install-myvgyz\pywin32-214-py2.6-win32.egg.tmp' to it creating d:\test\venv-d\lib\site-packages\pywin32-214-py2.6-win32.egg Extracting pywin32-214-py2.6-win32.egg to d:\test\venv-d\lib\site-packages Adding pywin32 214 to easy-install.pth file Installing pywin32_postinstall.py script to D:\test\VENV-D\Scripts Installing pywin32_postinstall.pyc script to D:\test\VENV-D\Scripts

Installed d:\test\venv-d\lib\site-packages\pywin32-214-py2.6-win32.egg Finished processing dependencies for foo==1.0

D:\test> }}}

Comments (13)

  1. Zooko O'Whielacronx reporter

    I suspect that this behavior -- reinstalling a package that is already there -- is the underlying cause of the failure to install in http://bitbucket.org/tarek/distribute/issue/141/easy_install-pywin32-binary-succeeds-and-then-says-it-cant-find . Also some people (my fellow developers on the Tahoe-LAFS project) get really annoyed if setuptools reinstalls a package that is already there, so it would be good if we could make it so that distribute never reinstalls a package that is already there.

  2. Zooko O'Whielacronx reporter

    Zart was using setuptools-0.6c11 in the venv-s part of his experiment.

    He compared the easy-install.pth's:

    <Zart> setuptools:
    <Zart> d:\test
    <Zart> c:\program files (x86)\python26\lib\site-packages
    <Zart> distribute:
    <Zart> d:\test
    <Zart> ./pywin32-214-py2.6-win32.egg
    

    Oh wait! Where did the venv-d version find "pywin32-214.win32-py2.6.exe"? The answer is that Zart had that file stashed on his system and had a pydistutils.cfg file with a find_links pointing to it. He removed that file and reran the experiment and this time the venv-s version succeeded and the venv-d version failed, just like in agronholm's experiment. So this appears to be a regression in distribute-0.6.10 vs. setuptools-0.6c10.

  3. Zooko O'Whielacronx reporter

    This prevents installation in some cases. I can see this problem happening on one of my servers. If I have Distribute 0.6.10 installed (this is Ubuntu Lucid) then when I try to install my SG package, which depends on the "tornado" package, Distribute detects the existence of tornado v0.2, and then attempts to download tornado == 0.2 to install it. This fails as tornado v0.2 isn't downloadable, so the attempt to install my package ends with:

    Searching for tornado==0.2
    Reading http://pypi.python.org/simple/tornado/
    Couldn't find index page for 'tornado' (maybe misspelled?)
    Scanning index of all packages (this may take a while)
    Reading http://pypi.python.org/simple/
    No local packages or download links found for tornado==0.2
    error: Could not find suitable distribution for Requirement.parse('tornado==0.2')
    

    If I manually remove Distribute (with /bin/rm -rf ) and then manually install setuptools-0.6c11 from http://pypi.python.org/pypi/setuptools by downloading the .tar.gz and running "sudo python setup.py install", then my package successfully installs. The part of the install output having to do with its dependency on tornado looks like this:

    Searching for tornado==0.2
    Best match: tornado 0.2
    Adding tornado 0.2 to easy-install.pth file
    

    In the short term, I guess we're going to be doing some manual work-arounds on our Ubuntu Lucid servers, or delay upgrading to Ubuntu Lucid. (This problem doesn't occur on Ubuntu Karmic because Karmic provides setuptools when you install the "python-setuptools" package but Lucid provides Distribute when you install the "python-setuptools" package. So from Ubuntu's perspective this is a regression in the "python-setuptools" package.)

    In the medium term, let's set up a unit test framework for the Distribute project so that once we fix this bug we'll have a regression test proving that this bug didn't reappear in a subsequent release. I have experience setting up Buildbot and Hudson and will volunteer some of my time to set that up (I prefer Buildbot) if the people who have commit access to the Distribute repository will commit to taking the tests seriously and not committing things that fail test cases.

  4. Anonymous

    I tried on a totally fresh vm with ubuntu lucid. I have no PYTHONPATH set, no config file, nothing. After the fresh install, I only did the following:

    sudo apt-get install python-virtualenv python-twisted virtualenv tmp source tmp/bin/activate easy_install twisted # download twisted ....

    If I install manually virtualenv 1.4.9 instead of using ubuntu virtualenv (1.4.5), and I make sure to install setuptools instead of distribute, then it works as expected (i.e. creating a virtualenv and easy_installing twisted just register from the ubuntu twisted, nothing is downloaded).

    This works for any package I have tried, not just twisted, so this has nothing to do with twisted

  5. Log in to comment