CircularDependencyError with jython because WeakSequence doesn't preserve order

Issue #2794 resolved
Sok Ann Yap
created an issue

I am toying with Jython, and the same app that works fine with CPython somehow throws a CircularDependencyError when trying to insert an instance of a mapped class that participates in a joined table inheritance.

I trace down the culprit to be WeakSequence.__iter__(), which doesn't preserve the insertion order.

Changing the line from:

# 0.8
return self._storage.itervalues()

or:

# 0.9
return iter(self._storage.values())

to:

for idx in sorted(self._storage):
    yield self._storage[idx](idx)

make things work again on Jython.

Comments (10)

  1. Sok Ann Yap reporter

    Here's some sample code that causes CircularDependencyError with Jython 2.7b1+ (current hg tip 7eb5574d023d):

    from sqlalchemy.engine import create_engine
    from sqlalchemy.ext.declarative import declarative_base
    from sqlalchemy.orm import scoped_session, sessionmaker
    from sqlalchemy.schema import Column, ForeignKey
    from sqlalchemy.types import Integer, String
    
    url = 'postgresql+zxjdbc://postgres@localhost/test'
    engine = create_engine(url)
    
    Session = scoped_session(sessionmaker())
    Session.configure(bind=engine)
    
    Base = declarative_base()
    
    class A(Base):
        __tablename__ = 'a'
    
        id = Column(Integer, primary_key=True)
        discriminator = Column(String(10))
    
        __mapper_args__ = {
            'polymorphic_on': discriminator,
            'polymorphic_identity': 'a',
        }
    
    class B(A):
        __tablename__ = 'b'
    
        __mapper_args__ = {
            'polymorphic_identity': 'b',
        }
    
        id = Column(Integer, ForeignKey(A.id), primary_key=True)
    
    class C(B):
        __tablename__ = 'c'
    
        __mapper_args__ = {
            'polymorphic_identity': 'c',
        }
    
        id = Column(Integer, ForeignKey(B.id), primary_key=True)
    
    Base.metadata.create_all(engine)
    
    c = C()
    Session.add(c)
    Session.flush()
    

    In this case, the random ordering would cause Mapper.self_and_descendants() to return C first instead of A, subsequently causing table_to_mapper to have table A -> mapper C, which then causes the circular dependency.

  2. Michael Bayer repo owner

    OK great, yeah if I do this and run tests with --reversetop:

    diff --git a/lib/sqlalchemy/orm/util.py b/lib/sqlalchemy/orm/util.py
    index ae1ca20..1d2d30d 100644
    --- a/lib/sqlalchemy/orm/util.py
    +++ b/lib/sqlalchemy/orm/util.py
    @@ -1327,3 +1327,11 @@ def randomize_unitofwork():
         from sqlalchemy.testing.util import RandomSet
         topological.set = unitofwork.set = session.set = mapper.set = \
                 dependency.set = RandomSet
    +    from sqlalchemy.util import WeakSequence
    +    import random
    +    def _random_iter(self):
    +        l = self._storage.values()
    +        random.shuffle(l)
    +        return iter(l)
    +    WeakSequence.__iter__ = _random_iter
    +
    

    crazy failures. very weird that one isn't hitting in pypy.

  3. Michael Bayer repo owner

    strange i can't get WeakSequence to naturally return the wrong ordering on cpython or pypy even with large lists, makes me wonder if WeakValueDictionary is somehow preserving insert ordering on those platforms.

  4. Sok Ann Yap reporter

    Yeah. It looks like because an integer always hashes to itself, a dict in cpython with keys in sequential integers (and inserted in the same order) will always have its ordering "preserved".

    Meanwhile, in the Javaland, while an integer still hashes to itself, a Map would perform another hash function to determine the final hash key, so the ordering become rather random.

  5. Log in to comment