1. Michael Bayer
  2. alembic
  3. Issues

Issues

Issue #102 resolved

alembic 0.4.2 tests does not pass on python 3.2

yaccz
created an issue

reproducer:

virtualenv-3.2 venv
. venv/bin/activate
pip install sqlalchemy==0.7.9
pip install mako==0.7.0
pip install nose==1.1.2
cd alembic-0.4.2
python3.2 setup.py install
nosetests-3.2  

output:

#! cat
======================================================================
ERROR: Failure: SyntaxError (invalid syntax (util.py, line 125))
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib64/python3.2/site-packages/nose/failure.py", line 37, in runTest
    raise self.exc_class(self.exc_val).with_traceback(self.tb)
  File "/usr/lib64/python3.2/site-packages/nose/loader.py", line 390, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib64/python3.2/site-packages/nose/importer.py", line 39, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib64/python3.2/site-packages/nose/importer.py", line 86, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/home/yac/Downloads/alembic-0.4.2/alembic/__init__.py", line 8, in <module>
    from alembic import op
  File "/home/yac/Downloads/alembic-0.4.2/alembic/op.py", line 1, in <module>
    from alembic.operations import Operations
  File "/home/yac/Downloads/alembic-0.4.2/alembic/operations.py", line 1, in <module>
    from alembic import util
  File "<string>", line None
SyntaxError: invalid syntax (util.py, line 125)

======================================================================
ERROR: Failure: SyntaxError (invalid syntax (__init__.py, line 56))
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib64/python3.2/site-packages/nose/failure.py", line 37, in runTest
    raise self.exc_class(self.exc_val).with_traceback(self.tb)
  File "/usr/lib64/python3.2/site-packages/nose/loader.py", line 390, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib64/python3.2/site-packages/nose/importer.py", line 39, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib64/python3.2/site-packages/nose/importer.py", line 86, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "<string>", line None
SyntaxError: invalid syntax (__init__.py, line 56)

----------------------------------------------------------------------
Ran 2 tests in 0.005s

FAILED (errors=2)

The problem

init.py: 56 except ImportError, er1:

http://docs.python.org/3.0/whatsnew/3.0.html#changed-syntax 4th bullet

Comments (15)

  1. Michael Bayer repo owner

    Alembic requires 2to3 be run on the source base, including not just the "alembic" source tree but also the "tests" folder. By performing "python3.2 setup.py install", this installs Alembic in the local Python environment with 2to3, but does not modify the source tree in place nor does it affect the tests folder. The local directory "." is part of sys.path by default so when nose runs, it loads the local source tree in favor of that which you installed environment-wide, and will also fail on files in test/

    See http://jenkins.sqlalchemy.org/job/alembic-default-sqla-default-3.2/210/console for full log output of tests run successfully with Python 3.2.

  2. idella5

    mike bayer;

    On removing tests from exclude which amounts to adding tests to packages in setup.py, and running the tests for py3.2, I get

    =============================================================== ERROR: test suite for <class 'tests.test_autogenerate.AutogenCrossSchemaTest'>


    File "/usr/lib64/python3.2/configparser.py", line 771, in get d = self._unify_values(section, vars) File "/usr/lib64/python3.2/configparser.py", line 1112, in _unify_values raise NoSectionError(section) configparser.NoSectionError: No section: "NoSectionError: No section: 'db'\n-------------------- >> begin captured stdout << ---------------------\n Creating directory /var/tmp/portage/portage/dev-\n python/alembic-0.4.2/work/alembic-0.4.2-python3_2/lib/tests/scratch/scripts...done\n Creating directory /var/tmp/portage/portage/dev-\n python/alembic-0.4.2/work/alembic-0.4.2-python3_2/lib/tests/scratch/scripts/versions...done\n Generating /var/tmp/portage/portage/dev-\n python/alembic-0.4.2/work/alembic-0.4.2-python3_2/lib/tests/scratch/scripts/env.py...done\n Generating /var/tmp/portage/portage/dev-\n python/alembic-0.4.2/work/alembic-0.4.2-python3_2/lib/tests/scratch/scripts/script.py.mako...done\n Generating /var/tmp/portage/portage/dev-\n python/alembic-0.4.2/work/alembic-0.4.2-python3_2/lib/tests/scratch/test_alembic.ini...done\n Generating /var/tmp/portage/portage/dev-\n python/alembic-0.4.2/work/alembic-0.4.2-python3_2/lib/tests/scratch/scripts/README...done\n Please edit configuration/connection/logging settings in '/var/tmp/portage/portage/dev-\n python/alembic-0.4.2/work/alembic-0.4.2-python3_2/lib/tests/scratch/test_alembic.ini' before proceeding.\n\n--------------------- >> end captured stdout << ----------------------"

    ====================================================================== ERROR: test suite for <class 'tests.test_autogenerate.AutogenerateDiffTestWSchema'>


    Traceback (most recent call last): File "/usr/lib64/python3.2/configparser.py", line 1109, in _unify_values sectiondict = self._sections[section] KeyError: 'db'

    During handling of the above exception, another exception occurred: ................................................................................................. ................................................................................................. File "/usr/lib64/python3.2/configparser.py", line 771, in get d = self._unify_values(section, vars) File "/usr/lib64/python3.2/configparser.py", line 1112, in _unify_values raise NoSectionError(section) configparser.NoSectionError: No section: 'db'

    as well as

    ================================================================ ERROR: test suite for <class 'tests.test_autogenerate.ImplicitConstraintNoGenTest'>


    ========================================================== ERROR: test suite for <class 'tests.test_postgresql.PostgresqlDefaultCompareTest'>


    =========================================================== ERROR: test suite for <class 'tests.test_postgresql.PostgresqlInlineLiteralTest'>


    all of which yield

    File "/usr/lib64/python3.2/configparser.py", line 771, in get d = self._unify_values(section, vars) File "/usr/lib64/python3.2/configparser.py", line 1112, in _unify_values raise NoSectionError(section) configparser.NoSectionError: No section: 'db'

    In http://jenkins.sqlalchemy.org/job/alembic-default-sqla-default-3.2/210/console. First 2 and last 2 of above pass happily.

    ImplicitConstraintNoGenTest appears to have been skipped or removed form the run. Is it that /usr/lib64/python3.2/configparser.py in your system has a section for db or is it reading from a prior .py file? Is my py3.2missing samething?I suspect it's not but something else is missing the db section. sqlalchemy-0.7.9 is installed under py3.2 and 2.7

    Why have you ommitted py3.2 from setup.py packages? The run of py3.2 makes it look as if they can 'cope' well.

  3. Michael Bayer repo owner

    On removing tests from exclude which amounts to adding tests to packages in setup.py,

    i don't know what this means.

    ImplicitConstraintNoGenTest appears to have been skipped or removed form the run.

    The MySQLdb DBAPI does not support Python 3, and that test requires MySQL. I've not installed a Python3 driver like OurSQL on that host.

    Why have you ommitted py3.2 from setup.py packages?

    I don't know what "setup.py packages" is. If you can refer to the specific file where "py3.2" is supposed to be present, that would help clarify.

    I'm really not understanding what the problem is for either of you, if you can provide a patch which proposes what you think I've done incorrectly in the build environment, that would be helpful. Otherwise, in my experience "nosetests" runs the tests from the local directory and I've illustrated how to do it - I'm not familiar with how to get it to run the library from the installation directory.

  4. yaccz reporter

    Michael Bayer

    Context: I packaged the alembic for gentoo and idella is helping me with it. I was required to add py3 support but was unable to run the tests even just from source. Your reply is explaining it well for me for now, however I decided to postpone py3 support in the package.

    Neither I understand idella5 message in this issue.

  5. Michael Bayer repo owner

    Here's a log of all the tests passing, minus the MySQL one still skipped, on my local machine using SQLAlchemy 0.7.9 and Python 3.2.2: https://gist.github.com/4700172

    So in that regard this is not very much like #96 at all, since that one was reproducible.

    Again, any pointers on setting up setup.py / nose / whatever in the way you think I'm doing it wrong, please provide patches.

  6. yaccz reporter

    Michael Bayer I wasn't refering the #96, but the one comment about databases in #96. And it was just a feeling.

    I don't see the issue with alembic atm. We need to figure how to execute 2to3 properly, which is downstream from you.

  7. idella5

    mike bayer; hi; from setup.py

      license='MIT',
      packages=find_packages('.', exclude=['examples*', 'test*']),
      include_package_data=True,
    

    "On removing tests from exclude" See here in the setup.py. find packages is EXCLUDED from finding test* which captures the tests folder in the source

    testuser@archtester ~/cvsPortage/gentoo-x86/dev-python/alembic $ ls /mnt/gen2/TmpDir/portage/dev-python/alembic-0.4.2/work/alembic-0.4.2/ alembic CHANGES docs MANIFEST.in README.rst setup.cfg test.cfg alembic.egg-info distribute_setup.py LICENSE PKG-INFO README.unittests setup.py tests

    "which amounts to adding tests to packages in setup.py." is self explanatory, however this is all about explaining I suppose. Taking the tests folder out leaves you with

    packages=find_packages('.', exclude=['examples*'])

    which amounts to adding tests to packages in setup.py. The tests folder is now treated as a package. See

    "what you think I've done incorrectly in the build environment"

    This package managment is an inexact science. When the standard build occurs by standard gentoo portage tools, the 'packages' are copied to a separate build dir for each python version. By tests being left out by virtue of being excluded in the key line packages=find_packages('.', exclude=['examples', 'test'])

    In gentoo we automatically get the error you have now succinctly mapped out in the first reply to yacz, namely

    "but does not modify the source tree in place nor does it affect the tests folder."

    So we end up with attempting to run nosetests-python3.2 on a test folder still only py2 comapatible. This caused us grief. You 're not WRONG in the setup, because you're justified in designing it to be run manually from the source folder. To run it from an ebuild ofcourse is doing so remotely, so what would be most CONVENIENT from a gentoo point of view is to simple simply exclude tests from exclude in setup.py in that key line, making for

    packages=find_packages('.', exclude=['examples*'])

    So that you can effectively consider the patch, it's all I did, and 2to3 converts the tests, set PYTHONPATH to point to the py3 directories and it all works smoothly.

    "Again, any pointers on setting up setup.py / nose"

    this is nice to see, showing a sincere interest to develop and help. For now, The MySQLdb DBAPI does not support Python 3, means I can safely skip that test for py3, and for the others, looks like I shall have to peruse the logs of the effective build. unless you can give me a straght reply to why my system reports being absent of a db section for theose 4 tests.

    Thanks Mike

  8. idella5

    well the log si simply a listing of all the tests passing ok. This

    ../.venv/bin/nosetests

    concerns my slightly, since virtualenv I'm not familiar with, though I probably should be so feel free to give me a hint or 2. The goal is to run the building, testsuite and install from an ebuild which is written in bash, and that about sums up an ebuild. If it absolutely requires a customised environment in which to pass tests, then it absolutely need be established accordingly in an ebuild. This knocks over a number of testsuites in the portage tree, but I really don't see this one fitting such a category. What I need is to get my system equipped with the section db in whichever files it's missing.

  9. Michael Bayer repo owner

    "On removing tests from exclude" See here in the setup.py. find packages is EXCLUDED from finding test* which captures the tests folder in the source

    oh. OK, well, I took a look in my site-packages to see if other packages have "tests" installed and it looks like quite a few do. This would impact all of my packages so I'd need to experiment with making this change across the board.

  10. Michael Bayer repo owner

    OK well, letting it install "tests" doesn't really help matters, "python3.2 setup.py test" still doesn't run 2to3 on the sources and generally just screws up: https://gist.github.com/4708373 nose is still going to find the tests in the local directory even if you install a Python3 source base elsewhere.

    It is an eventual plan to get Alembic to be Python 3 in place, see #55. I've already done this for Mako and it's how I do things going forward, so that should help this situation. I think 2to3 was completely a mistake in the Python world.

  11. Log in to comment