backref behavior with lazy=dynamic

[Mike Bayer]

I know this has been worked out, so I'm not understanding this behavior, seems very basic:

from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.ext.declarative import declarative_base

Base = declarative_base()

assoc = Table("assoc", Base.metadata,
        Column('aid', Integer, ForeignKey('a.id')),
        Column('bid', Integer, ForeignKey('b.id'))
    )

class A(Base):
    __tablename__ = "a"

    id = Column(Integer, primary_key=True)
    bs = relationship("B", backref="as_", secondary=assoc, lazy="dynamic")


class B(Base):
    __tablename__ = "b"

    id = Column(Integer, primary_key=True)

e = create_engine("sqlite://", echo=True)
Base.metadata.create_all(e)
s = Session(e)

a1 = A()
b1 = B()

s.add_all([b1](a1,))

a1.bs = [b1](b1)
assert len(b1.as_) == 1

without a flush, b1.as_ is empty. this causes history issues later on particularly when using association proxy and stuff like that. shouldn't dynamic be firing off all the exact same events ?

[Mike Bayer]

patch #1:

diff -r 332560b1fd0917a0e7dbeb295d48d045ee6f6887 lib/sqlalchemy/orm/dynamic.py
--- a/lib/sqlalchemy/orm/dynamic.py Fri Dec 14 15:40:05 2012 -0500
+++ b/lib/sqlalchemy/orm/dynamic.py Sun Dec 16 14:45:43 2012 -0500
@@ -31,10 +31,12 @@
         strategies._register_attribute(self,
             mapper,
             useobject=True,
+            uselist=True,
             impl_class=DynamicAttributeImpl,
             target_mapper=self.parent_property.mapper,
             order_by=self.parent_property.order_by,
-            query_class=self.parent_property.query_class
+            query_class=self.parent_property.query_class,
+            backref=self.parent_property.back_populates,
         )

 log.class_logger(DynaLoader)

but issues remain:

a1.bs = [b1](b1)
assert len(b1.as_) == 1

a1.bs = [= [b1](]
a1.bs)

print attributes.get_history(b1, 'as_')

"A" is added twice to "bs":

History(added=[object at 0x10152d7d0>, <__main__.A object at 0x10152d7d0>](<__main__.A), unchanged=[deleted=[](],))

[Mike Bayer]

the flip around case also:

from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import attributes
Base = declarative_base()

assoc = Table("assoc", Base.metadata,
        Column('aid', Integer, ForeignKey('a.id')),
        Column('bid', Integer, ForeignKey('b.id'))
    )

class A(Base):
    __tablename__ = "a"

    id = Column(Integer, primary_key=True)
    bs = relationship("B", backref=backref("as_", lazy="dynamic"), secondary=assoc, )


class B(Base):
    __tablename__ = "b"

    id = Column(Integer, primary_key=True)

e = create_engine("sqlite://", echo=True)
Base.metadata.create_all(e)
s = Session(e, autoflush=False)

a1 = A()
b1 = B()

s.add_all([b1](a1,))

a1.bs = [b1](b1)

a1.bs = [= [b1](]

a1.bs)

hist = attributes.get_history(b1, 'as_')
assert hist.added == [a1](a1)
assert hist.unchanged == [hist.deleted == [](]
assert)

basically CollectionHistory is going to need to reconcile its added/removed lists at all times.

[Mike Bayer]

here's the user's original case for when we do this:

from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.ext.associationproxy import association_proxy
from sqlalchemy.orm.exc import NoResultFound

Base = declarative_base()

classifiers = Table("version_classifiers",
    Base.metadata,
    Column("id", Integer, primary_key=True),
    Column("classifier_id", Integer,
              ForeignKey("classifiers.id", ondelete="CASCADE"),
              nullable=False),
    Column("version_id", Integer,
              ForeignKey("versions.id", ondelete="CASCADE"),
              nullable=False),
    UniqueConstraint("classifier_id", "version_id")
)


class Classifier(Base):

    __tablename__ = "classifiers"

    id = Column(Integer, primary_key=True)
    trove = Column(UnicodeText, unique=True, nullable=False)

    def __init__(self, trove):
        self.trove = trove

    def __repr__(self):
        return "<Classifier: {trove}>".format(trove=self.trove)

    @classmethod
    def get_or_create(cls, trove):
        try:
            obj = Session.query(cls).filter_by(trove=trove).one()
        except NoResultFound:
            obj = cls(trove)
        return obj


class Version(Base):

    __tablename__ = "versions"

    id = Column(Integer, primary_key=True)
    _classifiers = relationship("Classifier", secondary=classifiers,
                               backref=backref("versions", lazy='dynamic'))
    classifiers = association_proxy("_classifiers", "trove",
                                    creator=Classifier.get_or_create)


e = create_engine('sqlite://', echo=True)
Base.metadata.create_all(e)

Session = scoped_session(sessionmaker(e, autoflush=False))

v = Version()
Session.add(v)

v.classifiers = [u"Foo"](u"Foo")
Session.commit()  # A Classifier with trove=u"Foo" is either retrieved or created

v.classifiers = [u"Bar"](u"Foo",)
Session.commit()

[Mike Bayer]

3522785ef493a6cad6403b4c702bbfe2f1b7dc89

[Mike Bayer]

i want to add some tests that get an understanding of if/when the history operation loads the entire collection. this behavior basically should not be occurring, else it defeats the purpose of the dynamic loader. Only if a collection reassignment is done does that imply a full collection load. want to check if it's been doing this in 0.7 before, or if it's doing it now, and if we can make it not do that without too bad of a compatibility change.

[Mike Bayer]

reverting back to the old style, while keeping lots of the new stuff too, is efbbe88705eb19d68b587aae6dfb60cfe4356edb. I'd like to add more tests.

[Mike Bayer]

#2642 refers the original issue here to be dealt with by the association proxy. but also, why doesn't calling clear() + set() on a dynamic fully load the collection for history ? a "clear()" should mean, "load everything and mark all as deleted".

[Mike Bayer]

found the real cause of the original issue, the "pop()" method was missing from DynamicAttributeImpl. This as well as the tests for SELECT are in 5b23a6808041ff90e2097e2cb37188d4845828f4.

[Mike Bayer]

Removing milestone: 0.8.0final (automated comment)