eager join disabled loading joined mapper entities

Issue #3884 resolved
Torsten Landschoff created an issue

Sorry for the bad title, I have no idea how to summarize this. I tried to upgrade the SQLAlchemy version we are using (still at 0.9.10 because of the change in behaviour that setting a mapped relation overrides the column value that I once reported as a bug).

Upgrading to 1.0 creates unexpected failures where our code tries to load a mapped entity together with a related entity by a join. Basically the query

session.query(A, B).join("b")

stopped working.

This seems to affect both 1.0 and 1.1 branches. Full example code attached as joinfail.py.

For SQLAlchemy 0.9.10 this works as expected:

(python27)torsten.landschoff@horatio:~$ pip install SQLAlchemy==0.9.10
[...]
Successfully installed SQLAlchemy-0.9.10
(python27)torsten.landschoff@horatio:~$ python joinfail.py 
SELECT people.type AS people_type, engineers.id AS engineers_id, people.id AS people_id, engineers.primary_language AS engineers_primary_language, engineers.manager_id AS engineers_manager_id, people_1.type AS people_1_type, managers_1.id AS managers_1_id, people_1.id AS people_1_id, seen_1.id AS seen_1_id, seen_1.timestamp AS seen_1_timestamp, seen_2.id AS seen_2_id, seen_2.timestamp AS seen_2_timestamp 
FROM people JOIN engineers ON people.id = engineers.id JOIN (people AS people_1 JOIN managers AS managers_1 ON people_1.id = managers_1.id) ON managers_1.id = engineers.manager_id LEFT OUTER JOIN seen AS seen_1 ON people.id = seen_1.id LEFT OUTER JOIN seen AS seen_2 ON people_1.id = seen_2.id

Running the same example against SQLAlchemy 1.0.16 results in this:

(python27)torsten.landschoff@horatio:~$ pip install SQLAlchemy==1.0.16
[...]
Successfully installed SQLAlchemy-1.0.16
(python27)torsten.landschoff@horatio:~$ python joinfail.py 
Traceback (most recent call last):
  File "joinfail.py", line 50, in <module>
    print session.query(Engineer, Manager).join("manager")
  File "/opt/scalesdk/python27/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 3417, in __str__
    return str(self._compile_context().statement)
  File "/opt/scalesdk/python27/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 3261, in _compile_context
    strategy(*rec[1:])
  File "/opt/scalesdk/python27/lib/python2.7/site-packages/sqlalchemy/orm/strategies.py", line 1351, in _create_eager_join
    if parentmapper.isa(self.parent) else self.parent)
AttributeError: 'NoneType' object has no attribute 'isa'

The exact same behaviour is still present in 1.1.4:

(python27)torsten.landschoff@horatio:~$ pip install SQLAlchemy==1.1.4
[...]
Successfully installed SQLAlchemy-1.1.4
(python27)torsten.landschoff@horatio:~$ python joinfail.py 
Traceback (most recent call last):
  File "joinfail.py", line 50, in <module>
    print session.query(Engineer, Manager).join("manager")
  File "/opt/scalesdk/python27/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2800, in __str__
    context = self._compile_context()
  File "/opt/scalesdk/python27/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 3312, in _compile_context
    strategy(*rec[1:])
  File "/opt/scalesdk/python27/lib/python2.7/site-packages/sqlalchemy/orm/strategies.py", line 1372, in _create_eager_join
    if parentmapper.isa(self.parent) else self.parent)
AttributeError: 'NoneType' object has no attribute 'isa'

I also tried with the current master 2b4d028a69270c1c7918281a60280dd0b65963a2, still getting the same error.

Removing the `lazy="joined" option on the backref "last_seen" makes it working again, unfortunately we actually want to have that information loaded when available - at least on the toplevel entity. I am not sure how to disable it on the child only.

Still impressed how SQLAlchemy can handle stuff like this (it worked in 0.9.10).

Thanks!

Torsten

Comments (9)

  1. Mike Bayer repo owner

    OK, will look, if a bug + fix is possible this will be one of those inscrutable CHANGES messages again :) (e.g. fixed bug where mapping with widget plus foober combined with query that sets blarb with more than two entities would fail to join if whatsit :) )

  2. Mike Bayer repo owner

    hrm is this the right test case? I see no eager loading here and the script runs on all SQLAlchemy versions here:

    #!
    
    [classic@photon2 sqlalchemy]$ git checkout master
    Already on 'master'
    Your branch is up-to-date with 'origin/master'.
    [classic@photon2 sqlalchemy]$ python test.py 
    SELECT people.type AS people_type, engineers.id AS engineers_id, people.id AS people_id, engineers.primary_language AS engineers_primary_language, engineers.manager_id AS engineers_manager_id, people_1.type AS people_1_type, managers_1.id AS managers_1_id, people_1.id AS people_1_id 
    FROM people JOIN engineers ON people.id = engineers.id JOIN (people AS people_1 JOIN managers AS managers_1 ON people_1.id = managers_1.id) ON managers_1.id = engineers.manager_id
    [classic@photon2 sqlalchemy]$ git checkout rel_1_0
    Switched to branch 'rel_1_0'
    Your branch is up-to-date with 'origin/rel_1_0'.
    [classic@photon2 sqlalchemy]$ python test.py 
    SELECT people.type AS people_type, engineers.id AS engineers_id, people.id AS people_id, engineers.primary_language AS engineers_primary_language, engineers.manager_id AS engineers_manager_id, people_1.type AS people_1_type, managers_1.id AS managers_1_id, people_1.id AS people_1_id 
    FROM people JOIN engineers ON people.id = engineers.id JOIN (people AS people_1 JOIN managers AS managers_1 ON people_1.id = managers_1.id) ON managers_1.id = engineers.manager_id
    [classic@photon2 sqlalchemy]$ git checkout rel_0_9
    Switched to branch 'rel_0_9'
    Your branch is up-to-date with 'origin/rel_0_9'.
    [classic@photon2 sqlalchemy]$ python test.py 
    SELECT people.type AS people_type, engineers.id AS engineers_id, people.id AS people_id, engineers.primary_language AS engineers_primary_language, engineers.manager_id AS engineers_manager_id, people_1.type AS people_1_type, managers_1.id AS managers_1_id, people_1.id AS people_1_id 
    FROM people JOIN engineers ON people.id = engineers.id JOIN (people AS people_1 JOIN managers AS managers_1 ON people_1.id = managers_1.id) ON managers_1.id = engineers.manager_id
    [classic@photon2 sqlalchemy]$ 
    
  3. Mike Bayer repo owner

    if I add lazy=False to the LastSeen relationship that does it. Guess that's what you meant.

  4. Torsten Landschoff reporter

    Genious. I removed the "lazy='joined'" setting to check my assertion that this causes the problem and submitted the form afterwards. I guess Firefox sent the updated file than...

    This makes the bug reproducible again:

    $ diff -u joinfail.py.orig joinfail.py
    --- joinfail.py.orig    2017-01-09 17:38:28.192890518 +0100
    +++ joinfail.py 2017-01-09 17:38:08.696479609 +0100
    @@ -37,7 +37,8 @@
                             backref=backref("last_seen",
                                             uselist=False,
                                             cascade_backrefs=False,
    -                                        cascade='delete, delete-orphan'))
    +                                        cascade='delete, delete-orphan',
    +                                        lazy='joined'))
    
    
     engine = create_engine("sqlite:///")
    
  5. Mike Bayer repo owner

    are you looking for a backport to 1.0.x? Ive been trying to keep most fixes local to 1.1.x since we're well into it now.

  6. Torsten Landschoff reporter

    are you looking for a backport to 1.0.x? Ive been trying to keep most fixes local to 1.1.x since we're well into it now.

    A backport would be most welcome because I'd prefer to switch to 1.0.x and repair the resulting breakage before jumping another release upwards.

  7. Mike Bayer repo owner

    Adapt from "localparent" in joinedloader

    Fixed bug involving joined eager loading against multiple entities when polymorphic inheritance is also in use which would throw "'NoneType' object has no attribute 'isa'". The issue was introduced by the fix for 🎫3611.

    Change-Id: I296ecda38c01ec8f69dcd843beaebed6949cecfa Fixes: #3884

    → <<cset 51a72503b027>>

  8. Mike Bayer repo owner

    Adapt from "localparent" in joinedloader

    Fixed bug involving joined eager loading against multiple entities when polymorphic inheritance is also in use which would throw "'NoneType' object has no attribute 'isa'". The issue was introduced by the fix for 🎫3611.

    Change-Id: I296ecda38c01ec8f69dcd843beaebed6949cecfa Fixes: #3884 (cherry picked from commit 51a72503b0279ca71ee6f0454bfd36a4c84d508f)

    → <<cset c04af8e481ee>>

  9. Log in to comment