Reading the PkgResources docs, it appears that in setuptools 0.6c9, the version parsing was changed to support pre-release tags.
In versions of setuptools prior to 0.6a9, "-" characters were not removed, leading to the unintuitive result that "0.2-rc1" was considered a newer version than "0.2"
Looking at the source code histories, I see that in 2005, pje explicitly added support for this behavior in setuptools with this comment:
Changed ``parse_version()`` to remove dashes before pre-release tags, so
that ``0.2-rc1`` is considered an *older* version than ``0.2``, and is equal
to ``0.2rc1``. The idea that a dash *always* meant a post-release version
was highly non-intuitive to setuptools users and Python developers, who
seem to want to use ``-rc`` version numbers a lot.
I then see that last year, in the fix for #208, this behavior was reversed (and the test suite was adapted to no longer catch the behavior), based on the distribute docs.
One of two things needs to happen:
Distribute should explicitly deviate from setuptools behavior and include that deviation in the documentation. This option would indicate that the behavior reported in the pip issue 508 would be a "known deviation".
Distribute should follow the setuptools behavior as much as possible. This option would imply rolling back the fix for #208 and adjusting the documentation instead of the implementation.
Since Distribute is meant to be a drop-in replacement for setuptools, I'm strongly in favor of the second option.
I think the argument of being a drop-in replacement has become moot. Distribute should aim to Do The Right Thing, even if setuptools do not. It's soon three years since it had a release. Compatibility with setuptools is no longer an issue per se, we just need to make sure we don't break everything.
I therefore think that Distribute should aim at compatibility with PEP386, rather than setuptools. PEP386 doesn't have dashes in the version numbers, so therefore I don't see this as a bug, but a feature.
@Lennart Regebro Can you clarify? How is this deviation a feature? How does deviating from the intended setuptools implementation get distribute any closer to PEP386 compliance when the only change is to the semantic meaning of a dash, which is disallowed in the PEP?
Restoring compatibility with setuptools would mean that pip could continue to utilize both distribute and setuptools in a compatible manner, and support for PEP386 should be filed as a separate issue, and I agree that ticket would be a feature.
Maybe I'm unclear on what the actual claimed bug is.
However, my point is that deviating from what setuptools does when it does something wrong is not a bug, it's a feature. 0.2-rc1 should be considered equal to 0.2rc1 in my opinion. That setuptools does not treat those version numbers as equal is in my opinion a mistake in setuptools.
When it comes to whether 0.7.1-docs-html should be greater or smaller than 0.7.1-docs-html the answer is less obvious. To me it seems like this is a fork. Is it greater or lesser? I don't know. It seems to me to be a fork of 0.7.1 with new features, so greater sounds reasonable IMO.
@Lennart Regebro: Yes, apparently 0.2-rc1 == 0.2rc1 is the intuitive response, and that's what setuptools patched to support in 0.6c9, and that's what broke in distribute 0.6.23 with the patch for #208.
As for -docs-html, it's less obvious to me how that should be resolved, but in the name of consistency, I've decided distribute 0.6 will defer to setuptools 0.6 on this matter, although it may adopt another stance in a future, backward-incompatible release.
Ewww... Making two two different spellings evaluate equal leads to all sorts of potential problems. For instance, if I want to install the latest version of a package one frontend tool may choose 0.2-rc1 while a different tool may choose 0.2rc1. (Or even the same tool if the implementation was sticking these into a dict() and hash randomization is in effect).
Note that simply making 0.2-rc1 and 0.2rc1 evaluate as "a release candidate" but making one of those always be higher or lower than the other isn't as big of an issue. PEP386 has a similar contortion for "c" vs "rc" => both are release candidates but if you encounter both foo-1rc1 and foo-1c1, the rc1 will always sort after the c1.
If that's what the code is actually doing and you're just using 0.2-rc1 == 0.2rc1 as a shorthand then that's a lot better. (The sort order should probably make it into documentation somewhere, though).
I can definitely see your view point of following what earlier versions of 0.6 and what setuptools 0.6 do since it is an incompatible change (although it seems that setuptools-0.6c9 is incompatible with prior versions of the setuptools-0.6 series in this manner :-/ ) but if those really are evaluating equal, please strongly consider this a misfeature when you do make your next backwards incompatible release.