sqlalchemy.orm.mapper.Mapper.attrs doesn't maintain order

[Former user]

sqlalchemy.orm.mapper.Mapper.attrs is an immutable wrapper around sqlalchemy.orm.mapper.Mapper._props. However it uses util.ImmutableProperties which uses a regular dict as a backend and thus loses the order (_props is an OrderedDict).

I think it should be changed to use something like:

class ImmutableOrderedProperties(ImmutableContainer, OrderedProperties):
    """Provide immutable dict/object attribute to an underlying OrderedDict."""

[Former user]

forgot to set the proper "Component" on the ticket, sorry.

[Mike Bayer]

OK hold on. ImmutableProperties is passed the data dictionary, it does not use any kind of dict on its own. .attrs specifically invokes util.ImmutableProperties(self._props), so it's ordered.

Are you observing that mapper.attrs is not maintaining ordering?

[Mike Bayer]

this is tested already in test_inspect, added another test in 84b03137eb16b5e2a34e69. I think we are OK here please reopen if you can illustrate ordering not being maintained.

[Former user]

yeah, I'm seeing the attrs is not maintaining the order that attributes are defined in the object. I was looking through the code and thought that was the reason, but I'm wrong.

Here's the object I'm defining:

class Person(Base):
    __tablename__ = 'persons'

    id = Column(Integer, primary_key=True)
    name = Column(Unicode(128), nullable=False)
    surname = Column(Unicode(128), nullable=False)
    phones = relationship(Phone)

The order I consistently get back is

Person.phones
Person.id
Person.name
Person.surname

When I'm expecting to get Person.phones at the end of the list. Does the OrderedDict not maintain the order the attributes are defined in the class?

[Mike Bayer]

the order of columns when using Declarative is determined by the order in which each Column is constructed, since otherwise the ordering of items on a class in Python is unordered. So there is actually significant logic in order to maintain this ordering within the Table object that's created.

This ordering counter is also shared on relationship objects and such, but declarative does not pass the objects to the mapper in such a way that this ordering is maintained; nor does it specify column/relationship objects as individuals, instead the mapper maps the whole Table object at once, so there is no way right now that an "interleaved" pattern of plain columns/relationships can be produced using standard declarative, unless declarative re-implemented the mapper's usual job of iterating through table columns which is redundant. the ordering logic thus far is only concerned with making sure column order on the Table is maintained, as this results in DDL passed to the database.

probably the most expedient way to sort in object creation order is this:

from sqlalchemy import inspect
from sqlalchemy.orm import ColumnProperty
mapper = inspect(Person)

def creation_order(obj):
    if isinstance(obj, ColumnProperty):
        return obj.columns[0](0)._creation_order
    else:
        return obj._creation_order

mapper._props.sort(key=lambda k: creation_order(mapper._props[k](k)))

why exactly would it be important such that the order in which attributes are declared on a class be mirrored when inspecting the mapping? This is not the case for Python builtins such as dir(). The attributes on a Python class are not ordered.

[Former user]

The reason is for ColanderAlchemy. Colander schemas have a particular order and are used to generate HTML forms with Deform in that same order.

[Mike Bayer]

safest bet is to stick with _creation_order as above for now, sort by that as you pull off of mapper.attrs. Since there's really no other thing you can "order by" on a Python class.

[Former user]

Thank you!