Issue #264 open

easy_install or ' develop' with setup.cfg ignores dependency_links and allow_hosts

Michael Pedersen
created an issue

This issue is actually somewhat complex, and decidedly different behavior from the way that setuptools works. While it affects the TurboGears tools, I believe it will affect others at some point too.

We provide two packages for use with TurboGears: TurboGears2 and tg.devtools. We're only now deprecating support for Python 2.4 (we had to wait until CentOS stopped supplying it by default).

This means that we have to have special, old versions of the packages that we use. In order to accomplish this, we specified the dependency_links in our, and the allow_hosts directive in our setup.cfg. This means that when someone does "easy_install tg.devtools", everything just works: Our egg index is used, and only that index. People got the versions of the packages that they needed, and it worked.

Now, virtualenv has switched to using distribute by default. As a result, we are getting more and more people who have their virtualenv ignoring the directives we supply, resulting in them getting versions of the packages that are known to fail.

To make things even more confusing for people, if they do "easy_install tg.devtools", and hit Control-C immediately after that package is installed (before any others are downloaded), and then re-run the same command, then these directives will be honored and used.

Finally, when using "python develop", these directives are also ignored. The same problems occur there.

Comments (2)

  1. Jason R. Coombs
    • changed status to open

    I just ran easy_install tg.devtools using a virtualenv with distribute and setuptools, and both gave basically the same behavior. Both seemed to honor find_links and neither seemed to honor setup.cfg (specifically allow_hosts). Here's the output from the virtualenv with setuptools:

    PS C:\Users\jaraco\envs> virtualenv tgtest
    New python executable in tgtest\Scripts\python.exe
    Installing setuptools................done.
    Installing pip...................done.
    PS C:\Users\jaraco\envs> .\tgtest\Scripts\easy_install tg.devtools
    Searching for tg.devtools
    Best match: tg.devtools 2.1.4
    Processing tg.devtools-2.1.4.tar.gz
    Running tg.devtools-2.1.4\ -q bdist_egg --dist-dir c:\users\jaraco\appdata\local\temp\easy_install-tlyqa3\tg.devtools-2.1.4\egg-dist-tmp-vskd05
    Adding tg.devtools 2.1.4 to easy-install.pth file
    Installed c:\users\jaraco\envs\tgtest\lib\site-packages\tg.devtools-2.1.4-py2.7.egg
    Processing dependencies for tg.devtools
    Searching for TurboGears2>=2.1.4
    Best match: TurboGears2 2.1.4
    Processing TurboGears2-2.1.4.tar.gz
    Running TurboGears2-2.1.4\ -q bdist_egg --dist-dir c:\users\jaraco\appdata\local\temp\easy_install-uoed9b\TurboGears2-2.1.4\egg-dist-tmp-wemczl
    Adding turbogears2 2.1.4 to easy-install.pth file
    Installed c:\users\jaraco\envs\tgtest\lib\site-packages\turbogears2-2.1.4-py2.7.egg
    Searching for Babel
    Best match: Babel 0.9.6
    Processing Babel-0.9.6-py2.7.egg
    creating c:\users\jaraco\envs\tgtest\lib\site-packages\Babel-0.9.6-py2.7.egg
    Extracting Babel-0.9.6-py2.7.egg to c:\users\jaraco\envs\tgtest\lib\site-packages
    Adding Babel 0.9.6 to easy-install.pth file
    Installing script to C:\Users\jaraco\envs\tgtest\Scripts
    Installing pybabel.exe script to C:\Users\jaraco\envs\tgtest\Scripts
    Installing pybabel.exe.manifest script to C:\Users\jaraco\envs\tgtest\Scripts
    Installed c:\users\jaraco\envs\tgtest\lib\site-packages\babel-0.9.6-py2.7.egg
    Searching for WebError>=0.10.1
    Best match: WebError 0.10.3
    Processing WebError-0.10.3.tar.gz
    Running WebError-0.10.3\ -q bdist_egg --dist-dir c:\users\jaraco\appdata\local\temp\easy_install-yhdu5i\WebError-0.10.3\egg-dist-tmp-r3m86q
    warning: no files found matching 'eval_template.html' under directory 'weberror'
    Adding weberror 0.10.3 to easy-install.pth file
    Installed c:\users\jaraco\envs\tgtest\lib\site-packages\weberror-0.10.3-py2.7.egg
    Searching for WebFlash>=0.1a8
    Best match: WebFlash 0.1a9
    Processing WebFlash-0.1a9.tar.gz
    Running WebFlash-0.1a9\ -q bdist_egg --dist-dir c:\users\jaraco\appdata\local\temp\easy_install-go3cb1\WebFlash-0.1a9\egg-dist-tmp-uqk2fj
    Adding webflash 0.1a9 to easy-install.pth file
    Installed c:\users\jaraco\envs\tgtest\lib\site-packages\webflash-0.1a9-py2.7.egg
    Searching for Pylons>=0.9.7
    Best match: Pylons 1.0.1rc1
    Processing Pylons-1.0.1rc1.tar.gz
    Running Pylons-1.0.1rc1\ -q bdist_egg --dist-dir c:\users\jaraco\appdata\local\temp\easy_install-r_nzwv\Pylons-1.0.1rc1\egg-dist-tmp-wrrg9n
    warning: no previously-included files matching '*' found under directory 'tests\test_units\session'
    warning: no previously-included files matching '.DS_Store' found anywhere in distribution
    warning: no previously-included files matching '*.hgignore' found anywhere in distribution
    warning: no previously-included files matching '*.hgtags' found anywhere in distribution
    Adding pylons 1.0.1rc1 to easy-install.pth file
    Installed c:\users\jaraco\envs\tgtest\lib\site-packages\pylons-1.0.1rc1-py2.7.egg
    Searching for WebOb==1.0.8
    Best match: WebOb 1.0.8
    Running WebOb-1.0.8\ -q bdist_egg --dist-dir c:\users\jaraco\appdata\local\temp\easy_install-1jfw2q\WebOb-1.0.8\egg-dist-tmp-0kuaoe
    no previously-included directories found matching '*.pyc'
    no previously-included directories found matching '*.pyo'
    Adding webob 1.0.8 to easy-install.pth file
    Installed c:\users\jaraco\envs\tgtest\lib\site-packages\webob-1.0.8-py2.7.egg
    Searching for Paste>=1.7.1
    Best match: Paste
    Processing Paste-
    Running Paste-\ -q bdist_egg --dist-dir c:\users\jaraco\appdata\local\temp\easy_install-mswldk\Paste-\egg-dist-tmp-rtaxdy
    warning: no previously-included files matching '*' found under directory 'docs\_build\_sources'
    Adding paste to easy-install.pth file
    Installed c:\users\jaraco\envs\tgtest\lib\site-packages\paste-
    Searching for Pygments
    Best match: Pygments 1.5
    Processing Pygments-1.5-py2.7.egg
    creating c:\users\jaraco\envs\tgtest\lib\site-packages\Pygments-1.5-py2.7.egg
    Extracting Pygments-1.5-py2.7.egg to c:\users\jaraco\envs\tgtest\lib\site-packages
    Adding Pygments 1.5 to easy-install.pth file
    Installing script to C:\Users\jaraco\envs\tgtest\Scripts
    Installing pygmentize.exe script to C:\Users\jaraco\envs\tgtest\Scripts
    Installing pygmentize.exe.manifest script to C:\Users\jaraco\envs\tgtest\Scripts
    Installed c:\users\jaraco\envs\tgtest\lib\site-packages\pygments-1.5-py2.7.egg
    Searching for Tempita
    Best match: Tempita 0.5.1
    Processing Tempita-0.5.1.tar.gz
    Running Tempita-0.5.1\ -q bdist_egg --dist-dir c:\users\jaraco\appdata\local\temp\easy_install-mpjbxs\Tempita-0.5.1\egg-dist-tmp-9rrzrk
    C:\Python\Lib\distutils\ UserWarning: Unknown distribution option: 'use_2to3'
    Adding tempita 0.5.1 to easy-install.pth file
    Installed c:\users\jaraco\envs\tgtest\lib\site-packages\tempita-0.5.1-py2.7.egg
    error: Installed distribution webob 1.0.8 conflicts with requirement WebOb>=1.1.1

    Can you provide an example where the install succeeds with setuptools but fails with distribute? Do you know why I'm not able to reproduce what you say happens with setuptools (your index is used and only your index)?

    I agree that the behavior you expect is desirable. If this behavior is already in setuptools, we should provide a similar implementation. If not, we will have to come up with one from scratch (and be more cautious about the implications).

  2. Michael Pedersen reporter

    Unfortunately, not anymore. I would *swear* that this worked when I tested it back in July, 2011. I know I tested it and re-tested, since we had so much controversy over it within TG. However, a couple of months ago, it was found not to be working, and I was not able to reproduce it at that time.

    All I can figure is that some unique confluence of circumstances that i was not aware of were making it work.

    As for how to fix this, I really don't know. The complexity of this problem grows extremely quickly. What happens if a user depends on TurboGears2, but then has their own dependency_links with packages that provide version conflicts? While this case sounds outrageous, I've seen signs that this is going to start happening sometime soon. Some of our packages that we rely on have introduced new versions that are incompatible, and some users are starting to look at them.

    We're working on fixing it, but we're not there yet. I don't have any answers, only difficult questions.

  3. Log in to comment