- changed status to wontfix
setup.py misbehaves with respect to the location of shared objects
Using SQLalchemy 0.8.2 I was trying to build and install it into a custom Python build (i.e. not the system one) on Scientific Linux 6.1.
I'm setting $PYTHONHOME
before invoking setup.py install
using the Python into which I want to install the SQLAlchemy module.
Since I am trying to build an RPM and the idea is to have a self-contained Python environment, everything should go into a particular folder (/opt/app
) in this case.
I am using fakechroot
to mimic the target environment on the build machine and it works for Python as well as the extensions I am trying to install. With the sole exception of SQLAlchemy and the .so
files built for it. These end up inside:
/home/oliver/.python-eggs/SQLAlchemy-0.8.2-py2.7-linux-x86_64.egg-tmp/sqlalchemy/cutils.so
/home/oliver/.python-eggs/SQLAlchemy-0.8.2-py2.7-linux-x86_64.egg-tmp/sqlalchemy/cprocessors.so
/home/oliver/.python-eggs/SQLAlchemy-0.8.2-py2.7-linux-x86_64.egg-tmp/sqlalchemy/cresultproxy.so
within the faked chroot
environment which is not the intended behavior. Instead they should all go into the site-package
directory of the Python installation.
Am I doing something wrong or is this a defect?
Also note that from the output of setup.py install
when run inside the faked chroot
environment, I see:
creating stub loader for sqlalchemy/cprocessors.so
creating stub loader for sqlalchemy/cresultproxy.so
creating stub loader for sqlalchemy/cutils.so
byte-compiling build/bdist.linux-x86_64/egg/sqlalchemy/cprocessors.py to cprocessors.pyc
byte-compiling build/bdist.linux-x86_64/egg/sqlalchemy/cresultproxy.py to cresultproxy.pyc
byte-compiling build/bdist.linux-x86_64/egg/sqlalchemy/cutils.py to cutils.pyc
Thanks and best regards
Comments (5)
-
repo owner -
reporter Hi hi. Thanks for reading over it. I'm aware that you use
setuptools
. I am using the same for an extension (albeit proprietary) which I authored and I can tell you that this behavior cannot solely explained bysetuptools
.My extension - by the way with the exact same environment - installs just fine. I'll see whether I can devise a change to
setup.py
to fix this. Would you be willing to take this in or not at all? You closed this pretty quickly, so I'm asking.Thanks for SQLAlchemy, by the way.
-
repo owner The only function we call setup.py is setup(). There are no options to control where .so files go or how to respond to $PYTHONHOME, chroot environments or anything else like that (See http://docs.python.org/2/distutils/apiref.html#distutils.core.setup for core setup() and http://pythonhosted.org/distribute/setuptools.html#new-and-changed-setup-keywords for those added by setuptools). We are calling it in the identical form as other more widely used packages such as psycopg2, giving it a list of ext_modules, that's all it accepts.
If setup() is doing the wrong thing based on your environment, I'd assume the issue is with setuptools. I have no knowledge of what $PYTHONHOME even does in terms of setuptools, and based on my experience I'd guess it does not what you expect - $PYTHONHOME is not a variable that is commonly used these days, standard practice is to use virtualenv for local Python installations. So from all indications this certainly seems like a usage issue with setuptools overall and the folks on disutils-sig should have definitive information on how setuptools works - I'm only a user of it. If people with knowledge of setuptools can point to an actual bug in SQLAlchemy's code, reopen this ticket and we will fix it.
-
You probably want the --root option. It would not surprise me if setuptools gets confused by PYTHONHOME.
-
reporter Replying to dstufft:
You probably want the --root option. It would not surprise me if setuptools gets confused by PYTHONHOME.
Thanks, that is exactly it. Feels a bit awkward to say
--root /
, but it does the job. Thanks again for the help. - Log in to comment
SQLAlchemy's setup.py uses a system called "setuptools" to build its software, which itself builds upon Python's standard build system "distutils". "setuptools" itself is a de-facto standard in Python as well and is used by a large portion of Python projects.
All of the behaviors you're observing here are not specific to SQLAlchemy and are instead part of the behaviors of distutils and setuptools, there's nothing in SQLAlchemy itself that impacts the behavior of where .so files go or anything like that. SQLAlchemy does have some C extensions, which might not be the case for other packages you are comparing to, but for comparison, try installing a package like psycopg2 or lxml which also generates .so files and compare the behavior there.
This is a pretty fundamental Python install issue so I'd suggest asking on a general group such as comp.lang.python, or if you want to talk to distutils developers you can ask on http://www.python.org/community/sigs/current/distutils-sig/, they might be able to shed light on the mechanics of your issue here.