DistributionNotFound: mypackage==dev

If you use

install_requires = ['mypackage==dev']

in your requirements, running any installed console scripts triggers the following exception ... some debugging shows that it finds the suitable package but decides that ==dev is part of the package name and considers them not the same!

Traceback (most recent call last):
  File "/home/hatem/.virtualenvs/masterpackage/bin/somecommand", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/home/hatem/.virtualenvs/masterpackage/local/lib/python2.7/site-packages/distribute-0.6.31-py2.7.egg/pkg_resources.py", line 2833, in <module>
    parse_requirements(__requires__), Environment()
  File "/home/hatem/.virtualenvs/masterpackage/local/lib/python2.7/site-packages/distribute-0.6.31-py2.7.egg/pkg_resources.py", line 604, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: mypackage==dev

to recreate this bug, follow these steps

virtualenv ~/.virtualenvs/samplepackage
source ~/.virtualenvs/samplepackage/bin/activate
pip install hg+https://bitbucket.org/nassrat/samplepackage
somescript 
# if you installed it with -e, you can even try "python setup.py test"

The same error shows up when you do python setup.py test ... it actually tries to reinstall the package by recloning it (which is fine) and then say it does not exist.

% python setup.py test
running test
Checking .pth file support in .
/home/hatem/.virtualenvs/masterpackage/bin/python -E -c pass
Searching for mypackage==dev
Best match: mypackage dev
Doing hg clone from ssh://hg@dev/mypackage to /tmp/easy_install-X0R2pu/mypackage@production
Updating to production
Processing mypackage@production
Writing /tmp/easy_install-X0R2pu/mypackage@production/setup.cfg
Running setup.py -q bdist_egg --dist-dir /tmp/easy_install-X0R2pu/mypackage@production/egg-dist-tmp-zTnOco
zip_safe flag not set; analyzing archive contents...

Installed /home/hatem/projects/masterpackage/mypackage-5.4.3-py2.7.egg
Traceback (most recent call last):
  File "setup.py", line 48, in <module>
    test_suite='tests',
  File "/usr/lib/python2.7/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/home/hatem/.virtualenvs/masterpackage/local/lib/python2.7/site-packages/distribute-0.6.31-py2.7.egg/setuptools/command/test.py", line 128, in run
    self.distribution.fetch_build_eggs(self.distribution.install_requires)
  File "/home/hatem/.virtualenvs/masterpackage/local/lib/python2.7/site-packages/distribute-0.6.31-py2.7.egg/setuptools/dist.py", line 246, in fetch_build_eggs
    replace_conflicting=True
  File "/home/hatem/.virtualenvs/masterpackage/local/lib/python2.7/site-packages/distribute-0.6.31-py2.7.egg/pkg_resources.py", line 604, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound:mypackage==dev

I have setup a couple of bitbucket repos to help recreate this bug ... here is how

virtualenv ~/.virtualenvs/samplepackage
source ~/.virtualenvs/samplepackage/bin/activate
pip install hg+https://bitbucket.org/nassrat/samplepackage
somescript 
# if you installed it with -e, you can even try python setup.py test

I dug into the code, and it seems the outcome I want may not be supported ... this bug is just a side-effect I guess ... so here is some info on what I want to accomplish.

Basically I want that everytime I install samplepackage (from my example repo) otherpackage is downloaded and installed (at the very least installed if the version is newer).

I accomplished this by using ==dev in the requirements ... but from what I can see, the very same reason it works is the very same reason the console scripts do not work (since they also check for the requirements being present before running).

So my first question is, is there something I can do to get the desired effect. The second question is, why do the console scripts have to check for the presence or not of all the dependencies and refuse to run, it seems like overkill and essentially having a script refuse to run even though it would run perfectly fine (seems like a Java thing to do, being overprotective that is).

To the best of my knowledge, neither distribute nor setuptools have support for always upgrading dependencies. If dependencies are met (according to whatever requirement is specified), then the install is skipped. IIUC, pip is different, and will always upgrade dependencies if -U is specified, so you may consider using pip for installing your packages.

I believe the console scripts check for the presence of the package as a sanity check to provide a consistent error message when the environment isn't configured as the script intended (and as easy_install set-up). Removing those checks would mean that removed libraries or inconsistent versions might lead to various spurious errors, rather than indicating the primary cause (the script and library are out of sync). Secondarily, setuptools supports what's called 'multi-install' where multiple versions of a package can be installed but not automatically present in the sys.path. In this case, for the script to work properly, it must 'require' that package as part of its startup. As a result, it's probably preferable to keep these checks in place.

You bring up a very good use case, though. We will keep it in mind as the project progresses.

Sounds about right except for the pip -U part. It does not download/clone dependency_links if it is already satisfied ... which is a different issue.