Issues

Issue #30 resolved

Drop setuptools dependency

Bouke Haarsma
created an issue
The required version of setuptools (>=1.4) is not available,
and can't be installed while this script is running. Please
install a more recent version first, using
'easy_install -U setuptools'.

(Currently using setuptools 0.9.8 (/Users/bouke/.virtualenvs/suds-jurko/lib/python2.7/site-packages))
Complete output from command python setup.py egg_info:
The required version of setuptools (>=1.4) is not available,

and can't be installed while this script is running. Please

install a more recent version first, using

'easy_install -U setup tools'.

Why does suds depend on this seemingly very old dependency? From my understanding on the subject, distutils or even Distribute for that matter, have long superseded this package. See current state of packaging on the hitchhiker's guide. Also, who uses easy_install these days?

Comments (11)

  1. Jurko Gospodnetić repo owner

    I think setuptools is used so suds' setup.py would not need to list all its package content and instead collect them using setuptools.find_packages(), and possibly some other minor issues I am not recalling at the moment. Feel free to suggest a pull request removing it. smile

    As for why setuptools and not distribute - here's a short python packaging related history track:

    1. no installation packaging support
    2. distutils got created
    3. setuptools got created as a distutils upgrade/wrapper
    4. setuptools development stalled
    5. distribute got created as a setuptools replacement
    6. several other packaging tools came and went - distutils2, packaging, ...
    7. setuptools & distribute merged under the setuptools flag

    So setuptools is the latest/greatest available contemporary packaging tool for Python. smile

    As for the listed version - I think setuptools 0.9.8 is the oldest setuptools version tested and found to be working with our suds project. And the setuptools/distribute merge occurred somewhere around version 0.7.

    In my current suds sandbox I've already upgraded this to be the latest available setuptools version so that installing suds from sources, will automatically install/upgrade setuptools to the most recent tested version. But I won't be pushing out that change until I finish up my current installation & testing cleanup work.

    As for who uses easy_install these days? - it's an installation tool that comes with setuptools. It will eventually become obsolete and completely replaced by pip, but at the moment the two are not completely equivalent. You are under no obligation to use it. It gets listed in that error message you quoted because the message comes from the setuptools package itself and so it refers to its own tool.

    What triggered the error message anyway? Did you get it when attempting to install suds or when attempting to do something else after having installed suds? From reading your quote, I'm guessing it's setuptools complaining about not being able to upgrade the current setuptools version inplace since your Python process already loaded one of setuptools' modules. That should not happen if running the suds setup.py directly, so my guess is that, if that happened during suds installation, you either modified suds' setup.py or your local Python environment site.py script somehow or ran the whole setup via some external process (tox?) that caused the old setuptools version modules to be loaded before the use_compatible_setuptools() call in suds' setup.py.

    Since suds project installation requires setuptools - we can either require that the user installs it manually before installing suds, or we can install it automatically for him. If we install it for him - we can either download the appropriate version from pypi, or we can include a copy of setuptools in our distribution. Currently we choose to install setuptools automatically by downloading an appropriate version from the internet, if needed. Feel free to suggest whatever changes you like.

    FYI, Python 3.4 comes with pip & setuptools included (see py34 -m ensurepip) so the whole issue becomes moot after that version. smile

    Hope this helps.

    Best regards, Jurko Gospodnetić

  2. Bouke Haarsma reporter

    For me, it always triggers in a new virtual environment (Py 2.7) and running pip install suds-jurko. It's a new and clean virtual environment, no changes to setup.py (the latest build on pypi got downloaded). So therefore I feel like there is an issue here.

    Python 2.6 (on my machine) comes with setuptools 0.6c12 and Python 2.7 comes with setuptools 2.2. Although older Python versions are mentioned as being supported, is it worth the effort to keep supporting those?

    See also pull request #35 where I removed ez_setup. It appears to install fine on Python 2.6 and 2.7.

  3. Jurko Gospodnetić repo owner

    Setuptools removal

    At least on Windows, no the official CPython installer for versions prior to 3.4 comes with setuptools included on Windows, so I do not think simply skipping the setuptools installation step fixes anything. It simply changes the suds setup behaviour from:

    install automatically

    to

    user has to install it prior to installing suds

    If you really want to remove setuptools - you need to remove its use completely from the setup.py script, and not just skip auto-installing it. smile

    Virtual environment installation problem

    I'll try to reproduce the behaviour on my end today and get back to you.

    Old Python version support

    I don't know if anyone still uses suds with Python 2.4, but the original project listed that environment as supported and so I was loathe to lose it without good reason when it was not really all that difficult to keep it around. Well, that and I found the whole excercise most enlightening. smile

    Best regards, Jurko Gospodnetić

  4. Bouke Haarsma reporter

    I agree with you that installation should be as painless as possible. The current situation however breaks the automatic installation at least for me (OS X). So yeah, there should be a solution in which installation is OK for all users.

    So what is setuptools actually used for, besides find_packages()?

  5. Jurko Gospodnetić repo owner

    OK, I looked into the virtualenv installation issue now and reproduced the problem. It seems that the virtualenv part is not important here at all and the scenario where the problem occurs is this:

    1. you have a Python environment (virtualenv or not) with setuptools & pip installed
    2. installed setuptools version is older than the one required by suds-jurko
    3. you run the suds-jurko installation using pip

    pip will then import setuptools just before running the suds-jurko setup.py install script. This will cause the suds-jurko setup.py script to call its ez_setup.py script for upgrading the currently installed setuptools version (as the currently installed one is not of high enough version), and unfortunately that step will fail to do an in-place setuptools upgrade because pip already imported setuptools before this script got run.

    Solutions are to either:

    • not use pip to install (seems way too harsh, especially taking into consideration that pip has been selected as the official Python package installation system)
    • update pip to not import setuptools (impossible for older pip versions and possibly even impossible for future ones since I think pip uses setuptools here to collect the run setup.py installation's meta-information needed, for example, so the package can be pip uninstalled later on)
    • leave things as they are, with the error message clearly instructing the user to first upgrade their setuptools installation (this can be done using either pip or easy_install, but not in the same pip call that installs suds)
    • stop using setuptools in the suds installation, thus making it a pure distutils install (would make maintaining the setup.py code a bit more difficult and the resulting installation will not be uninstallable by pip if installed directly using 'setup.py install' as that will not leave being any package meta-information containing the list of installed files)

    so... what to do... what to do...

  6. Jurko Gospodnetić repo owner

    I couldn't stay away and looked at this some more this morning... smile

    Things we use setuptools for in our setup.py:

    1. distutils/py2to3 integration (already implemented a prototype locally that does this without setuptools)
    2. find_packages() (can be worked around not too painfully by writing our own specialized version in setup.py - done it locally)
    3. installing the packages in developent/source mode from its setup.py installation using the setup.py develop command (the same effect can still be achieved for any installation using pip install -e)
    4. automatically installs package meta-data when installing directly using setup.py (allows a clean pip uninstall, could possibly be worked around by using metadata hardcoded in the project itself)
    5. used to implement our current pytest integration, i.e. support for running setup.py test and having it run the full project test suite using the current Python interpreter
    6. when running the setup.py install directly, suds gets installed as a versioned .egg and so does not conflict with a previously installed version, i.e. it does not install everything by simply placing it in the site-packages/suds folder but instead adds a version-specific folder under site-packages and adds that folder to the Python search path using a .pth file (the same effect can be achieved for a non-setuptools based installation using pip install --egg)

    Looking at what some other projects do:

    • pytest - simply imports setuptools and expects it to be already installed
    • psutil & cx_Freeze - uses setuptools if available or distutils if not

    So I guess if we want to make the installation 'as painless as possible', we could do this:

    1. make setup.py use setuptools 'as is' and 'only if available' and manually check whether its version is good enough
    2. support the test command only if setuptools are installed (report a clean error message if not) or possibly implement it ourselves
    3. integrate py2to3 into distutils without using setuptools (already done locally)

    or we could attempt to update setuptools as we do now, and fall back to the current installation if that fails, or to distutils if there is no suitable setuptools installation available.

    Best regards, Jurko Gospodnetić

  7. Jurko Gospodnetić repo owner

    Bouke Haarsma (and whoever else may be interested smile), could you take a look at what I pushed at jurko/suds-without-setuptools ?

    That's just a clone of one of my local development repos (some of the commits there may get retouched before pushing it all to jurko/suds) that includes setup.py updates to make the setup work better with regards to how it uses setuptools.

    Please try different setup.py installations on your system and let me know how it goes.

    I tested:

    • using various Python versions
    • with different preinstalled library versions (py, pytest, setuptools, pip, virtualenv) or without them
    • by running various setup.py commands (build, install, test, develop)

    and encountered no problems so far. angel

    If you're interested and can spare the time, feel free to review the setup.py code and flame me as needed. smile

    Changes in that repo include the following:

    • now attempts to use whatever setuptools version is already available in the target Python environment
    • now attempts to download (from PyPI) and install the most recent tested setuptools version only if no preinstalled setuptools version is found in the target Python envrionment (this avoids the pip based installation problem that originally triggered this issue)
    • now works even without setuptools, with a bit reduced functionality if setuptools is not available (no metadata installed therefore no pip uninstall support, no setup.py test, egg_info or develop command support, no egg distribution package installations), but note that some setuptools version is always available when installing using pip
    • better setup.py test command implementation

      • better error reporting
      • works with Python 2.4 (although that requires carefully preinstalled pytest & py library versions as described in HACKING.rst)
      • correctly lists 'argparse' as its requirement when using Python 3.0 or 3.1 (pytest still has incompatibilities with these Python releases but that's a separate issue)
    • now exits with a clean error message if run using a Python version more ancient than 2.4

    • cleaner code organization smile

    Now I'm off to integrate these changes into my main development repo and then take a quick look at the several tiny issues reported in the last few days. Hopefully, I'll have enough free time to soon finish my work (and/or integrate with related work by Bouke Haarsma) on setting up a better testing environment that will allow automated installation testing with different Python versions - that should help shake out any remaining setup.py bugs.

    Many thanks.

    Best regards, Jurko Gospodnetić

  8. Bouke Haarsma reporter

    I tried installing it on OS X with Python 2.6, 2.7 and 3.4 with succes and without installing setuptools first. This is on a "homebrew" installation of Python, which is quite a normal way to install Python on OS X. So it 'works for me', and probably also for other current configurations. However for older versions of Python (2.4 / 2.5), I don't know. I don't have those versions available on my machines.

    Thanks for your time investigating this and the thorough follow-ups you posted.

  9. Log in to comment