Unable to install any package with a dot in its name, as of about 7 hours ago.

Issue #343 on hold
Tai Lee
created an issue

Suddenly all our test builds on all our projects started failing on Travis CI. All our projects have a nested dependency on backports.ssl-match-hostname, and it seems that suddenly, devpi is unable to properly mirror any package with a dot in its name.

Here's how you can verify for yourself with the latest version of devpi-server.

First, install and run devpi-server in a new virtualenv:

~ ❯❯❯ mkvirtualenv devpi-server
New python executable in /Users/tailee/.virtualenvs/devpi-server/bin/python2.7
Also creating executable in /Users/tailee/.virtualenvs/devpi-server/bin/python
Please make sure you remove any previous custom paths from your /Users/tailee/.pydistutils.cfg file.
Installing setuptools, pip, wheel...done.
virtualenvwrapper.user_scripts creating /Users/tailee/.virtualenvs/devpi-server/bin/predeactivate
virtualenvwrapper.user_scripts creating /Users/tailee/.virtualenvs/devpi-server/bin/postdeactivate
virtualenvwrapper.user_scripts creating /Users/tailee/.virtualenvs/devpi-server/bin/preactivate
virtualenvwrapper.user_scripts creating /Users/tailee/.virtualenvs/devpi-server/bin/postactivate
virtualenvwrapper.user_scripts creating /Users/tailee/.virtualenvs/devpi-server/bin/get_env_details
(devpi-server) ~ ❯❯❯ pip install -i https://pypi.python.org/simple devpi-server devpi-web
Collecting devpi-server
  Using cached devpi_server-3.1.0-py2.py3-none-any.whl
Collecting devpi-web
  Downloading devpi_web-3.1.0-py2.py3-none-any.whl (85kB)
    100% |████████████████████████████████| 92kB 291kB/s
Collecting devpi-common<2.1,>=2.0.7.dev0 (from devpi-server)
  Using cached devpi_common-2.0.8-py2.py3-none-any.whl
Collecting py>=1.4.23 (from devpi-server)
  Using cached py-1.4.31-py2.py3-none-any.whl
Collecting pluggy<1.0,>=0.3.0 (from devpi-server)
  Using cached pluggy-0.3.1-py2.py3-none-any.whl
Collecting execnet>=1.2 (from devpi-server)
  Using cached execnet-1.4.1-py2.py3-none-any.whl
Collecting pyramid>=1.5.1 (from devpi-server)
  Using cached pyramid-1.6.1-py2.py3-none-any.whl
Collecting itsdangerous>=0.24 (from devpi-server)
Collecting repoze.lru>=0.6 (from devpi-server)
Collecting waitress>=0.8.9 (from devpi-server)
  Using cached waitress-0.9.0-py2.py3-none-any.whl
Collecting defusedxml (from devpi-web)
  Downloading defusedxml-0.4.1.tar.gz (48kB)
    100% |████████████████████████████████| 51kB 66kB/s
Collecting pygments>=1.6 (from devpi-web)
  Downloading Pygments-2.1.3-py2.py3-none-any.whl (755kB)
    100% |████████████████████████████████| 757kB 353kB/s
Collecting docutils>=0.11 (from devpi-web)
  Downloading docutils-0.12.tar.gz (1.6MB)
    100% |████████████████████████████████| 1.6MB 314kB/s
Collecting beautifulsoup4>=4.3.2 (from devpi-web)
  Downloading beautifulsoup4-4.4.1-py2-none-any.whl (81kB)
    100% |████████████████████████████████| 81kB 262kB/s
Collecting readme-renderer (from devpi-web)
  Downloading readme_renderer-0.7.0-py2.py3-none-any.whl
Collecting Whoosh<3 (from devpi-web)
  Downloading Whoosh-2.7.4-py2.py3-none-any.whl (468kB)
    100% |████████████████████████████████| 471kB 275kB/s
Collecting pyramid-chameleon (from devpi-web)
  Downloading pyramid_chameleon-0.3.tar.gz
Collecting requests>=2.3.0 (from devpi-common<2.1,>=2.0.7.dev0->devpi-server)
  Using cached requests-2.10.0-py2.py3-none-any.whl
Collecting apipkg>=1.4 (from execnet>=1.2->devpi-server)
  Using cached apipkg-1.4-py2.py3-none-any.whl
Collecting translationstring>=0.4 (from pyramid>=1.5.1->devpi-server)
  Using cached translationstring-1.3-py2.py3-none-any.whl
Collecting zope.deprecation>=3.5.0 (from pyramid>=1.5.1->devpi-server)
Collecting WebOb>=1.3.1 (from pyramid>=1.5.1->devpi-server)
  Using cached WebOb-1.6.0-py2.py3-none-any.whl
Collecting venusian>=1.0a3 (from pyramid>=1.5.1->devpi-server)
Collecting zope.interface>=3.8.0 (from pyramid>=1.5.1->devpi-server)
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.virtualenvs/devpi-server/lib/python2.7/site-packages (from pyramid>=1.5.1->devpi-server)
Collecting PasteDeploy>=1.5.0 (from pyramid>=1.5.1->devpi-server)
  Using cached PasteDeploy-1.5.2-py2.py3-none-any.whl
Collecting six (from readme-renderer->devpi-web)
  Downloading six-1.10.0-py2.py3-none-any.whl
Collecting bleach (from readme-renderer->devpi-web)
  Downloading bleach-1.4.2-py2.py3-none-any.whl
Collecting Chameleon (from pyramid-chameleon->devpi-web)
  Downloading Chameleon-2.24.tar.gz (193kB)
    100% |████████████████████████████████| 194kB 316kB/s
Collecting html5lib>=0.999 (from bleach->readme-renderer->devpi-web)
  Downloading html5lib-0.9999999.tar.gz (889kB)
    100% |████████████████████████████████| 890kB 298kB/s
Building wheels for collected packages: defusedxml, docutils, pyramid-chameleon, Chameleon, html5lib
  Running setup.py bdist_wheel for defusedxml ... done
  Stored in directory: /Users/tailee/Library/Caches/pip/wheels/47/d0/73/a3232f5cd866e8ce7091049aebfd537db5f9454110fc202eef
  Running setup.py bdist_wheel for docutils ... done
  Stored in directory: /Users/tailee/Library/Caches/pip/wheels/db/de/bd/b99b1e12d321fbc950766c58894c6576b1a73ae3131b29a151
  Running setup.py bdist_wheel for pyramid-chameleon ... done
  Stored in directory: /Users/tailee/Library/Caches/pip/wheels/08/36/d7/da0c457b8655377361915150aa7d2f4b251a4f6249819257d3
  Running setup.py bdist_wheel for Chameleon ... done
  Stored in directory: /Users/tailee/Library/Caches/pip/wheels/50/b1/98/93b1293613b91119ad873fc9566a3b02e697afb9dbc690aa3b
  Running setup.py bdist_wheel for html5lib ... done
  Stored in directory: /Users/tailee/Library/Caches/pip/wheels/6f/85/6c/56b8e1292c6214c4eb73b9dda50f53e8e977bf65989373c962
Successfully built defusedxml docutils pyramid-chameleon Chameleon html5lib
Installing collected packages: py, requests, devpi-common, pluggy, apipkg, execnet, translationstring, zope.deprecation, WebOb, venusian, zope.interface, PasteDeploy, repoze.lru, pyramid, itsdangerous, waitress, devpi-server, defusedxml, pygments, docutils, beautifulsoup4, six, html5lib, bleach, readme-renderer, Whoosh, Chameleon, pyramid-chameleon, devpi-web
Successfully installed Chameleon-2.24 PasteDeploy-1.5.2 WebOb-1.6.0 Whoosh-2.7.4 apipkg-1.4 beautifulsoup4-4.4.1 bleach-1.4.2 defusedxml-0.4.1 devpi-common-2.0.8 devpi-server-3.1.0 devpi-web-3.1.0 docutils-0.12 execnet-1.4.1 html5lib-0.9999999 itsdangerous-0.24 pluggy-0.3.1 py-1.4.31 pygments-2.1.3 pyramid-1.6.1 pyramid-chameleon-0.3 readme-renderer-0.7.0 repoze.lru-0.6 requests-2.10.0 six-1.10.0 translationstring-1.3 venusian-1.0 waitress-0.9.0 zope.deprecation-4.1.2 zope.interface-4.1.3
(devpi-server) ~ ❯❯❯ devpi-server --start
2016-05-11 16:31:02,276 INFO  NOCTX Loading node info from /Users/tailee/.devpi/server/.nodeinfo
2016-05-11 16:31:02,280 INFO  NOCTX wrote nodeinfo to: /Users/tailee/.devpi/server/.nodeinfo
starting background devpi-server at http://localhost:3141
/Users/tailee/.devpi/server/.xproc/devpi-server$ /Users/tailee/.virtualenvs/devpi-server/bin/devpi-server
process u'devpi-server' started pid=18314
devpi-server process startup detected
logfile is at /Users/tailee/.devpi/server/.xproc/devpi-server/xprocess.log

Then install simplejson as per the quick start guide, also in a fresh virtualenv, to verify that devpi-server is working:

~ ❯❯❯ mkvirtualenv devpi-test
New python executable in /Users/tailee/.virtualenvs/devpi-test/bin/python2.7
Also creating executable in /Users/tailee/.virtualenvs/devpi-test/bin/python
Please make sure you remove any previous custom paths from your /Users/tailee/.pydistutils.cfg file.
Installing setuptools, pip, wheel...done.
virtualenvwrapper.user_scripts creating /Users/tailee/.virtualenvs/devpi-test/bin/predeactivate
virtualenvwrapper.user_scripts creating /Users/tailee/.virtualenvs/devpi-test/bin/postdeactivate
virtualenvwrapper.user_scripts creating /Users/tailee/.virtualenvs/devpi-test/bin/preactivate
virtualenvwrapper.user_scripts creating /Users/tailee/.virtualenvs/devpi-test/bin/postactivate
virtualenvwrapper.user_scripts creating /Users/tailee/.virtualenvs/devpi-test/bin/get_env_details
(devpi-test) ~ ❯❯❯ pip install -i http://localhost:3141/root/pypi/ simplejson
Collecting simplejson
  Downloading http://localhost:3141/root/pypi/+f/e49/4f1df2cd3b7e6/simplejson-3.8.2-cp27-cp27m-macosx_10_9_x86_64.whl (67kB)
    100% |████████████████████████████████| 71kB 34.0MB/s
Installing collected packages: simplejson
Successfully installed simplejson-3.8.2

Now try to install a couple of packages with dots in their names:

(devpi-test) ~ ❯❯❯ pip install -i http://localhost:3141/root/pypi/ backports.ssl-match-hostname
Collecting backports.ssl-match-hostname
  Could not find a version that satisfies the requirement backports.ssl-match-hostname (from versions: )
No matching distribution found for backports.ssl-match-hostname
(devpi-test) ~ ❯❯❯ pip install -i http://localhost:3141/root/pypi/ repoze.lru
Collecting repoze.lru
  Could not find a version that satisfies the requirement repoze.lru (from versions: )
No matching distribution found for repoze.lru

Now try to install the same packages directly from PyPI:

(devpi-test) ~ ❯❯❯ pip install -i https://pypi.python.org/simple backports.ssl-match-hostname
Collecting backports.ssl-match-hostname
Installing collected packages: backports.ssl-match-hostname
Successfully installed backports.ssl-match-hostname-3.5.0.1
(devpi-test) ~ ❯❯❯ pip install -i https://pypi.python.org/simple repoze.lru
Collecting repoze.lru
Installing collected packages: repoze.lru
Successfully installed repoze.lru-0.6

I'm not sure if PyPI suddenly changed something that broke devpi-server, but it seems odd that this suddenly became a problem 8 hours ago. We were using an old version of devpi-server (2.1.5 I think) and we upgraded to 3.1 and still see the same problem.

I'm not sure how we can even work around this on all our projects besides updating the .travis.yml file or tox.ini file on every project to explicitly install backports.ssl-match-hostname from PyPI before installing the rest of our packages from devpi?

Comments (8)

  1. Log in to comment