CircularDependencyError with jython because WeakSequence doesn't preserve order

[Sok Ann Yap]

I am toying with Jython, and the same app that works fine with CPython somehow throws a CircularDependencyError when trying to insert an instance of a mapped class that participates in a joined table inheritance.

I trace down the culprit to be WeakSequence.__iter__(), which doesn't preserve the insertion order.

Changing the line from:

# 0.8
return self._storage.itervalues()

or:

# 0.9
return iter(self._storage.values())

to:

for idx in sorted(self._storage):
    yield self._storage[idx](idx)

make things work again on Jython.

[Mike Bayer]

can you give me some hint of a clue as to a stack trace, test case, something illustrating what particular operation/where has this issue ? thanks.

[Sok Ann Yap]

Here's some sample code that causes CircularDependencyError with Jython 2.7b1+ (current hg tip 7eb5574d023d):

from sqlalchemy.engine import create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import scoped_session, sessionmaker
from sqlalchemy.schema import Column, ForeignKey
from sqlalchemy.types import Integer, String

url = 'postgresql+zxjdbc://postgres@localhost/test'
engine = create_engine(url)

Session = scoped_session(sessionmaker())
Session.configure(bind=engine)

Base = declarative_base()

class A(Base):
    __tablename__ = 'a'

    id = Column(Integer, primary_key=True)
    discriminator = Column(String(10))

    __mapper_args__ = {
        'polymorphic_on': discriminator,
        'polymorphic_identity': 'a',
    }

class B(A):
    __tablename__ = 'b'

    __mapper_args__ = {
        'polymorphic_identity': 'b',
    }

    id = Column(Integer, ForeignKey(A.id), primary_key=True)

class C(B):
    __tablename__ = 'c'

    __mapper_args__ = {
        'polymorphic_identity': 'c',
    }

    id = Column(Integer, ForeignKey(B.id), primary_key=True)

Base.metadata.create_all(engine)

c = C()
Session.add(c)
Session.flush()

In this case, the random ordering would cause Mapper.self_and_descendants() to return C first instead of A, subsequently causing table_to_mapper to have table A -> mapper C, which then causes the circular dependency.

[Mike Bayer]

OK great, yeah if I do this and run tests with --reversetop:

diff --git a/lib/sqlalchemy/orm/util.py b/lib/sqlalchemy/orm/util.py
index ae1ca20..1d2d30d 100644
--- a/lib/sqlalchemy/orm/util.py
+++ b/lib/sqlalchemy/orm/util.py
@@ -1327,3 +1327,11 @@ def randomize_unitofwork():
     from sqlalchemy.testing.util import RandomSet
     topological.set = unitofwork.set = session.set = mapper.set = \
             dependency.set = RandomSet
+    from sqlalchemy.util import WeakSequence
+    import random
+    def _random_iter(self):
+        l = self._storage.values()
+        random.shuffle(l)
+        return iter(l)
+    WeakSequence.__iter__ = _random_iter
+

crazy failures. very weird that one isn't hitting in pypy.

[Mike Bayer]

strange i can't get WeakSequence to naturally return the wrong ordering on cpython or pypy even with large lists, makes me wonder if WeakValueDictionary is somehow preserving insert ordering on those platforms.

[Sok Ann Yap]

Yeah. It looks like because an integer always hashes to itself, a dict in cpython with keys in sequential integers (and inserted in the same order) will always have its ordering "preserved".

Meanwhile, in the Javaland, while an integer still hashes to itself, a Map would perform another hash function to determine the final hash key, so the ordering become rather random.

[Mike Bayer]

0.8

17274222f1bd3b844faea4d848756ae3ee3545dd

4b1ca23027493d8d1b7ea48d4585699b0d126d16

master

676876f4668520af267d7db883d9486a8924b320

[Mike Bayer]

crap, I forgot to handle the reference being lost.

[Mike Bayer]

ef73845ba773cbf4247355cde37aff8ca3b9f5f8

9769628ac1c080dc69b3812bc546ff

also added an append() which I'm going to use for #2779

[Mike Bayer]

Removing milestone: 0.8.xx (automated comment)