Issue #369 on hold

Handling 404 on find-url

Guy Rozendorn
created an issue

easy_install tries to download packages from their download url if it finds one, and if that download url returns 404 it fails, although it should try to download the package from pypi too.

We had one example for this, pypng, but we reported the bad link to the maintainer and he fixed it already. however, this problem stil exists in easy_install

Comments (5)

  1. Jason R. Coombs

    It's not obvious to my why the existing behavior isn't adequate. Perhaps it's desirable if the package maintainer specifies a download_url that only the download_url is tried. Since there is no specification for these behaviors, I'm more inclined to assume that the existing behavior is by design and only change it if there's an obvious shortcoming.

    In this case, it's not clear cut, and changing it may affect users that depend on the existing behavior. For example, maybe the download_url has been used to supersede dists previously-uploaded to PyPI. Implementing the changes proposed would expose those older versions when the download_url is unreachable (or otherwise returns a 404).

    At the very least, there should be a discussion in the pypa-dev list. If there's a consensus that PyPI downloads should be used as a fallback, then we can surely proceed with implementing this proposal.

    Also to consider, we do have a (somewhat insubstantial) goal to unify the behavior of pip and easy_install. How does pip behave in this case? If pip falls back to PyPI downloads, that provides a strong argument in favor of the proposal.

  2. Jason R. Coombs

    To be clear, I'm not blocking this proposal. It seems like a reasonable behavior, and may be worthwhile pursuing. I just want to be sure not to accept it without due consideration.

  3. Guy Rozendorn reporter

    pip behaves as we expect; if pip finds a broken link is falls back to the downloadable items on pypi

    for example: pypng==0.0.14 (not the latest version) contains a broken download-url, here are the outputs of how pip and easy_install behave in this case:

    ➜  ~  easy_install -v pypng==0.0.14
    Searching for pypng==0.0.14
    Reading http://pypi.python.org/simple/pypng/
    Reading http://code.google.com/p/pypng/
    Found link: http://pypng.googlecode.com/files/pypng-0.0.3.tar.gz
    Reading https://github.com/drj11/pypng
    Found link: https://github.com/drj11/pypng/archive/master.zip
    Found link: https://github.com/drj11/pypng/archive/pypng-0.0.15.tar.gz
    Found link: http://pypng.googlecode.com/files/pypng-0.0.14.tar.gz
    Found link: http://pypng.googlecode.com/files/pypng-0.0.12.tar.gz
    Found link: http://pypng.googlecode.com/files/pypng-0.0.13.tar.gz
    Found link: http://pypng.googlecode.com/files/pypng-0.0.5.tar.gz
    Found link: http://pypng.googlecode.com/files/pypng-0.0.11.tar.gz
    Found link: http://pypng.googlecode.com/files/pypng-0.0.6.tar.gz
    Found link: http://pypng.googlecode.com/files/pypng-0.0.7.tar.gz
    Found link: http://pypng.googlecode.com/files/pypng-0.0.4.tar.gz
    Found link: http://pypng.googlecode.com/files/pypng-0.0.10.tar.gz
    Found link: http://pypng.googlecode.com/files/pypng-0.0.8.tar.gz
    Found link: http://pypng.googlecode.com/files/pypng-0.0.9.tar.gz
    Found link: http://pypi.python.org/packages/source/p/pypng/pypng-0.0.7.tar.gz#md5=456d42b1348074f5212429a3d7fe957e
    Found link: http://pypi.python.org/packages/source/p/pypng/pypng-0.0.2.tar.gz#md5=63189ae59f28323a539a05ac0a5b9890
    Found link: http://pypi.python.org/packages/source/p/pypng/pypng-0.0.3.tar.gz#md5=ddb348b8dcea4204152e537dfd8f2ffa
    Found link: http://pypi.python.org/packages/source/p/pypng/pypng-0.0.10.tar.gz#md5=2be09cb331f4f73873f8b3249afe47cc
    Found link: http://pypi.python.org/packages/source/p/pypng/pypng-0.0.5.tar.gz#md5=230b89d0ee8df07d5aaaefa122ffeeb4
    Found link: http://pypi.python.org/packages/source/p/pypng/pypng-0.0.12.tar.gz#md5=23a99081d593aedffb813b73b1e28c33
    Found link: http://pypi.python.org/packages/source/p/pypng/pypng-0.0.11.tar.gz#md5=dc6d92425ca6e1c4de3553209c86a820
    Found link: http://pypi.python.org/packages/source/p/pypng/pypng-0.0.4.tar.gz#md5=8fce0291ab23016eb8800ad9efbffcf9
    Found link: http://pypi.python.org/packages/source/p/pypng/pypng-0.0.15.tar.gz#md5=4a215bbd518a5e0202071be777bb4433
    Found link: http://pypi.python.org/packages/source/p/pypng/pypng-0.0.8.tar.gz#md5=6cb08a92550c473200ce7dc5a5eba75d
    Found link: http://pypi.python.org/packages/source/p/pypng/pypng-0.0.14.tar.gz#md5=97bca21ab50c1bf7e1092dca569db8c4
    Found link: http://pypi.python.org/packages/source/p/pypng/pypng-0.0.13.tar.gz#md5=535f9932c8589bb02d4043aaeb4c5b72
    Found link: http://pypi.python.org/packages/source/p/pypng/pypng-0.0.9.tar.gz#md5=bafbae2215ca97b6fd22f7f12e60088a
    Found link: http://pypi.python.org/packages/source/p/pypng/pypng-0.0.6.tar.gz#md5=a0dd152bf58a90ce2b1aed9043cea474
    Best match: pypng 0.0.14
    Downloading http://pypng.googlecode.com/files/pypng-0.0.14.tar.gz
    error: Can't download http://pypng.googlecode.com/files/pypng-0.0.14.tar.gz: 404 Not Found
    
    
    
    ➜  ~  pip install -v pypng==0.0.14
    Downloading/unpacking pypng==0.0.14
      Could not fetch URL http://pypng.googlecode.com/files/pypng-0.0.3.tar.gz (from http://pypi.python.org/simple/pypng/): HTTP Error 404: Not Found
      Will skip URL http://pypng.googlecode.com/files/pypng-0.0.3.tar.gz when looking for download links for pypng==0.0.14
      Could not fetch URL http://pypi.python.org/simple/pypng/0.0.14: HTTP Error 404: Not Found (pypng/0.0.14)
      Will skip URL http://pypi.python.org/simple/pypng/0.0.14 when looking for download links for pypng==0.0.14
      Could not fetch URL http://pypng.googlecode.com/files/pypng-0.0.14.tar.gz (from http://pypi.python.org/simple/pypng/): HTTP Error 404: Not Found
      Will skip URL http://pypng.googlecode.com/files/pypng-0.0.14.tar.gz when looking for download links for pypng==0.0.14
      Could not fetch URL http://pypng.googlecode.com/files/pypng-0.0.12.tar.gz (from http://pypi.python.org/simple/pypng/): HTTP Error 404: Not Found
      Will skip URL http://pypng.googlecode.com/files/pypng-0.0.12.tar.gz when looking for download links for pypng==0.0.14
      Could not fetch URL http://pypng.googlecode.com/files/pypng-0.0.13.tar.gz (from http://pypi.python.org/simple/pypng/): HTTP Error 404: Not Found
      Will skip URL http://pypng.googlecode.com/files/pypng-0.0.13.tar.gz when looking for download links for pypng==0.0.14
      Could not fetch URL http://pypng.googlecode.com/files/pypng-0.0.5.tar.gz (from http://pypi.python.org/simple/pypng/): HTTP Error 404: Not Found
      Will skip URL http://pypng.googlecode.com/files/pypng-0.0.5.tar.gz when looking for download links for pypng==0.0.14
      Could not fetch URL https://github.com/drj11/pypng/archive/pypng-0.0.15.tar.gz (from http://pypi.python.org/simple/pypng/): HTTP Error 404: Not Found
      Will skip URL https://github.com/drj11/pypng/archive/pypng-0.0.15.tar.gz when looking for download links for pypng==0.0.14
      Could not fetch URL http://pypng.googlecode.com/files/pypng-0.0.11.tar.gz (from http://pypi.python.org/simple/pypng/): HTTP Error 404: Not Found
      Will skip URL http://pypng.googlecode.com/files/pypng-0.0.11.tar.gz when looking for download links for pypng==0.0.14
      Could not fetch URL http://pypng.googlecode.com/files/pypng-0.0.6.tar.gz (from http://pypi.python.org/simple/pypng/): HTTP Error 404: Not Found
      Will skip URL http://pypng.googlecode.com/files/pypng-0.0.6.tar.gz when looking for download links for pypng==0.0.14
      Could not fetch URL http://pypng.googlecode.com/files/pypng-0.0.7.tar.gz (from http://pypi.python.org/simple/pypng/): HTTP Error 404: Not Found
      Will skip URL http://pypng.googlecode.com/files/pypng-0.0.7.tar.gz when looking for download links for pypng==0.0.14
      Could not fetch URL http://pypng.googlecode.com/files/pypng-0.0.4.tar.gz (from http://pypi.python.org/simple/pypng/): HTTP Error 404: Not Found
      Will skip URL http://pypng.googlecode.com/files/pypng-0.0.4.tar.gz when looking for download links for pypng==0.0.14
      Could not fetch URL http://pypng.googlecode.com/files/pypng-0.0.10.tar.gz (from http://pypi.python.org/simple/pypng/): HTTP Error 404: Not Found
      Will skip URL http://pypng.googlecode.com/files/pypng-0.0.10.tar.gz when looking for download links for pypng==0.0.14
      Could not fetch URL http://pypng.googlecode.com/files/pypng-0.0.8.tar.gz (from http://pypi.python.org/simple/pypng/): HTTP Error 404: Not Found
      Will skip URL http://pypng.googlecode.com/files/pypng-0.0.8.tar.gz when looking for download links for pypng==0.0.14
      Could not fetch URL http://pypng.googlecode.com/files/pypng-0.0.9.tar.gz (from http://pypi.python.org/simple/pypng/): HTTP Error 404: Not Found
      Will skip URL http://pypng.googlecode.com/files/pypng-0.0.9.tar.gz when looking for download links for pypng==0.0.14
      Ignoring link http://pypi.python.org/packages/source/p/pypng/pypng-0.0.7.tar.gz#md5=456d42b1348074f5212429a3d7fe957e (from http://pypi.python.org/simple/pypng/), version 0.0.7 doesn't match ==0.0.14
      Ignoring link http://pypi.python.org/packages/source/p/pypng/pypng-0.0.2.tar.gz#md5=63189ae59f28323a539a05ac0a5b9890 (from http://pypi.python.org/simple/pypng/), version 0.0.2 doesn't match ==0.0.14
      Ignoring link http://pypi.python.org/packages/source/p/pypng/pypng-0.0.3.tar.gz#md5=ddb348b8dcea4204152e537dfd8f2ffa (from http://pypi.python.org/simple/pypng/), version 0.0.3 doesn't match ==0.0.14
      Ignoring link http://pypi.python.org/packages/source/p/pypng/pypng-0.0.10.tar.gz#md5=2be09cb331f4f73873f8b3249afe47cc (from http://pypi.python.org/simple/pypng/), version 0.0.10 doesn't match ==0.0.14
      Ignoring link http://pypi.python.org/packages/source/p/pypng/pypng-0.0.5.tar.gz#md5=230b89d0ee8df07d5aaaefa122ffeeb4 (from http://pypi.python.org/simple/pypng/), version 0.0.5 doesn't match ==0.0.14
      Ignoring link http://pypi.python.org/packages/source/p/pypng/pypng-0.0.12.tar.gz#md5=23a99081d593aedffb813b73b1e28c33 (from http://pypi.python.org/simple/pypng/), version 0.0.12 doesn't match ==0.0.14
      Ignoring link http://pypi.python.org/packages/source/p/pypng/pypng-0.0.11.tar.gz#md5=dc6d92425ca6e1c4de3553209c86a820 (from http://pypi.python.org/simple/pypng/), version 0.0.11 doesn't match ==0.0.14
      Ignoring link http://pypi.python.org/packages/source/p/pypng/pypng-0.0.4.tar.gz#md5=8fce0291ab23016eb8800ad9efbffcf9 (from http://pypi.python.org/simple/pypng/), version 0.0.4 doesn't match ==0.0.14
      Ignoring link http://pypi.python.org/packages/source/p/pypng/pypng-0.0.15.tar.gz#md5=4a215bbd518a5e0202071be777bb4433 (from http://pypi.python.org/simple/pypng/), version 0.0.15 doesn't match ==0.0.14
      Ignoring link http://pypi.python.org/packages/source/p/pypng/pypng-0.0.8.tar.gz#md5=6cb08a92550c473200ce7dc5a5eba75d (from http://pypi.python.org/simple/pypng/), version 0.0.8 doesn't match ==0.0.14
      Ignoring link http://pypi.python.org/packages/source/p/pypng/pypng-0.0.13.tar.gz#md5=535f9932c8589bb02d4043aaeb4c5b72 (from http://pypi.python.org/simple/pypng/), version 0.0.13 doesn't match ==0.0.14
      Ignoring link http://pypi.python.org/packages/source/p/pypng/pypng-0.0.9.tar.gz#md5=bafbae2215ca97b6fd22f7f12e60088a (from http://pypi.python.org/simple/pypng/), version 0.0.9 doesn't match ==0.0.14
      Ignoring link http://pypi.python.org/packages/source/p/pypng/pypng-0.0.6.tar.gz#md5=a0dd152bf58a90ce2b1aed9043cea474 (from http://pypi.python.org/simple/pypng/), version 0.0.6 doesn't match ==0.0.14
      Ignoring link http://pypng.googlecode.com/files/pypng-0.0.3.tar.gz (from http://pypi.python.org/simple/pypng/), version 0.0.3 doesn't match ==0.0.14
      Ignoring link https://github.com/drj11/pypng/archive/pypng-0.0.15.tar.gz (from http://pypi.python.org/simple/pypng/), version 0.0.15 doesn't match ==0.0.14
      Ignoring link http://pypng.googlecode.com/files/pypng-0.0.12.tar.gz (from http://pypi.python.org/simple/pypng/), version 0.0.12 doesn't match ==0.0.14
      Ignoring link http://pypng.googlecode.com/files/pypng-0.0.13.tar.gz (from http://pypi.python.org/simple/pypng/), version 0.0.13 doesn't match ==0.0.14
      Ignoring link http://pypng.googlecode.com/files/pypng-0.0.5.tar.gz (from http://pypi.python.org/simple/pypng/), version 0.0.5 doesn't match ==0.0.14
      Ignoring link http://pypng.googlecode.com/files/pypng-0.0.11.tar.gz (from http://pypi.python.org/simple/pypng/), version 0.0.11 doesn't match ==0.0.14
      Ignoring link http://pypng.googlecode.com/files/pypng-0.0.6.tar.gz (from http://pypi.python.org/simple/pypng/), version 0.0.6 doesn't match ==0.0.14
      Ignoring link http://pypng.googlecode.com/files/pypng-0.0.7.tar.gz (from http://pypi.python.org/simple/pypng/), version 0.0.7 doesn't match ==0.0.14
      Ignoring link http://pypng.googlecode.com/files/pypng-0.0.4.tar.gz (from http://pypi.python.org/simple/pypng/), version 0.0.4 doesn't match ==0.0.14
      Ignoring link http://pypng.googlecode.com/files/pypng-0.0.10.tar.gz (from http://pypi.python.org/simple/pypng/), version 0.0.10 doesn't match ==0.0.14
      Ignoring link http://pypng.googlecode.com/files/pypng-0.0.8.tar.gz (from http://pypi.python.org/simple/pypng/), version 0.0.8 doesn't match ==0.0.14
      Ignoring link http://pypng.googlecode.com/files/pypng-0.0.9.tar.gz (from http://pypi.python.org/simple/pypng/), version 0.0.9 doesn't match ==0.0.14
      Using version 0.0.14 (newest of versions: 0.0.14, 0.0.14)
      Downloading pypng-0.0.14.tar.gz (287kB): 287kB downloaded
      Running setup.py egg_info for package pypng
        /Users/arnony/.pythonbrew/pythons/Python-default/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'use_2to3'
          warnings.warn(msg)
        running egg_info
        creating pip-egg-info/pypng.egg-info
        writing pip-egg-info/pypng.egg-info/PKG-INFO
        writing top-level names to pip-egg-info/pypng.egg-info/top_level.txt
        writing dependency_links to pip-egg-info/pypng.egg-info/dependency_links.txt
        writing manifest file 'pip-egg-info/pypng.egg-info/SOURCES.txt'
        warning: manifest_maker: standard file '-c' not found
    
        reading manifest file 'pip-egg-info/pypng.egg-info/SOURCES.txt'
        reading manifest template 'MANIFEST.in'
        warning: no previously-included files matching '*.pyc' found anywhere in distribution
        warning: no previously-included files matching '*.DS_Store' found anywhere in distribution
        writing manifest file 'pip-egg-info/pypng.egg-info/SOURCES.txt'
    Installing collected packages: pypng
      Running setup.py install for pypng
        /Users/arnony/.pythonbrew/pythons/Python-default/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'use_2to3'
          warnings.warn(msg)
        running install
        running build
        running build_py
        creating build
        creating build/lib
        copying code/png.py -> build/lib
        running install_lib
        copying build/lib/png.py -> /Users/arnony/.pythonbrew/venvs/Python-default/test/lib/python2.7/site-packages
        byte-compiling /Users/arnony/.pythonbrew/venvs/Python-default/test/lib/python2.7/site-packages/png.py to png.pyc
        running install_egg_info
        running egg_info
        writing code/pypng.egg-info/PKG-INFO
        writing top-level names to code/pypng.egg-info/top_level.txt
        writing dependency_links to code/pypng.egg-info/dependency_links.txt
        warning: manifest_maker: standard file '-c' not found
    
        reading manifest file 'code/pypng.egg-info/SOURCES.txt'
        reading manifest template 'MANIFEST.in'
        warning: no previously-included files matching '*.pyc' found anywhere in distribution
        warning: no previously-included files matching '*.DS_Store' found anywhere in distribution
        writing manifest file 'code/pypng.egg-info/SOURCES.txt'
        Copying code/pypng.egg-info to /Users/arnony/.pythonbrew/venvs/Python-default/test/lib/python2.7/site-packages/pypng-0.0.14-py2.7.egg-info
        running install_scripts
        writing list of installed files to '/var/folders/dc/5_pz7h8n6wz_68gs_zh1tvm40000gn/T/pip-GA1HYg-record/install-record.txt'
    Successfully installed pypng
    Cleaning up...
      Removing temporary dir /Users/arnony/.pythonbrew/venvs/Python-default/test/build...
    
  4. Log in to comment