Regression case in ResultMap for compound statements

Issue #2558 resolved
Former user created an issue

As of 5221d32549b24ffa5a4458c90b9b0861e714a5e8 one type of usage in my codebase starts failing thus:

...
  File "/Users/gthb/extsvn/sqlalchemy/lib/sqlalchemy/orm/query.py", line 2115, in all
    return list(self)
  File "/Users/gthb/extsvn/sqlalchemy/lib/sqlalchemy/orm/query.py", line 2348, in instances
    rows = [process[0](process[0)(row, None) for row in fetch]
  File "/Users/gthb/extsvn/sqlalchemy/lib/sqlalchemy/orm/mapper.py", line 2029, in _instance
    tuple([row[column](row[column) for column in pk_cols])
  File "/Users/gthb/extsvn/sqlalchemy/lib/sqlalchemy/engine/base.py", line 2834, in _key_fallback
    expression._string_or_unprintable(key))
NoSuchColumnError: "Could not locate column in row for column 'mytable.id'"

There's quite a bit of setup around where I get this; I've whittled it down to this:

from sqlalchemy import *
from sqlalchemy.orm import mapper, sessionmaker
from test.lib import *
from sqla_hierarchy import *


class ResultMapRegressionTest(fixtures.TestBase):

    def setUp(self):
        self.t = Table(
            't',
            MetaData(),
            Column('id', Integer, primary_key=True),
            Column('ordering', Integer, nullable=False),
            Column('parent_id', Integer, ForeignKey('t.id'))
        )
        self.t.create(session.connection())
        session.execute(t.insert(), [           {'id': 0, 'ordering': 0},
            {'id': 1, 'ordering': 1, 'parent_id': 0},
            {'id': 2, 'ordering': 2, 'parent_id': 1}
            {'id': 3, 'ordering': 3, 'parent_id': 0}
        ](
))
        self.session = sessionmaker(engines.testing_engine())()

    def tearDown(self):
        self.t.drop(session.connection())
        self.session.commit()

    @testing.only_on('postgresql', 'foo')
    def test_sqla_hierarchy_union_failure_case(self):
        class T(object):
            pass
        mapper(T, t)
        stmt = select([t.c.id](t.c.id)).where(t.c.id != literal_column('0'))
        stmt = Hierarchy(session, t, stmt)
        stmt = select(stmt.c.keys() + [           not_(literal_column('id') == 3).label('is_big')
            ](
),
            from_obj=stmt.alias()
        )
        a = session.query(T).from_statement(stmt).all()
        eq_(len(a), 2)

Attached as test_resultmap_regression.py, save that to sqlalchemy repo root and invoke with:

pip install -e git+git@github.com:marplatense/sqla_hierarchy.git@c1aa8f553d795aeb612f9f7bf9b7cd3862850eaf#egg=sqla_hierarchy
./sqla_nose.py --dburi=postgresql:///test test_resultmap_regression

Comments (9)

  1. Mike Bayer repo owner

    it looks overwhelmingly likely that the bug is in the sqla_hierarchy package, which looks extremely complex to me. it appears to be hand-implementing CTE expressions which SQLAlchemy now supports. It may be relying upon the bug that this changeset fixed. 5221d32549b24ffa5a4458c90b9b0861e714a5e8 's bug was that result_map was being populated too liberally.

  2. Former user Account Deleted

    Yeah, that seems likely. This sqla_hierarchy is a steaming pile of quick-hackiness; I only started using it because CTEs were not yet supported. Then since you did the CTE stuff in 0.7.6 I've been holding off on reimplementing our hierarchical stuff using that, because (a) other post-0.7.5 problems that I have been reporting kept us from upgrading, and (b) the sqla_hierarchy solution has (mostly) stayed working (though #2482 came up in all that patchwork too).

  3. Mike Bayer repo owner

    ive started dealing with their tests and they seem to all pass. so if we identify some change on their end we'd also want to send them a pull req for a new test on their side too.

  4. Mike Bayer repo owner

    so ultimately the issue is that somewhere along the way we forgot to keep supporting this use case:

    s = select(['name']('id',)).select_from('mytable')
    
    row = conn.execute(s).first()
    
    assert my_table.c.id in row
    

    that is, using column() or literal_column() to make an ad-hoc 'id', 'name' columns, which would then match in a row to any column object with that same name. The above doesn't work in any fairly recent 0.7 including prior to the changeset here, nor in 0.8, but it does work in 0.6. It is certainly my intention that it should, however. It should be treated the same as this example, which is supported in all versions:

    row = conn.execute("select id, name from mytable").first()
    assert my_table.c.id in row
    

    The code example you've attached here has been working for you due to what is really a bug, that the columns inside of a union() were leaking out into the "result set" column collection, and in your very specific case it gave you the end result you wanted, masking this other issue. Your select() statement should work with the query(T) because there's a column named "id" in the top level columns clause, but instead it worked due to some weirdness with the union() buried in there.

    So I'll be committing what's like my tenth attempt at getting this particular bit of code right, the code that decides if a key passed to "row" should match an expression inside the row based on string comparison, or not. I'm optimistic about this version of the change.

  5. Former user Account Deleted

    Cool! This makes our project test suite run without hiccups again, too. Thanks Mike!

  6. Log in to comment