TypeDecorator / SchemaType combination not firing attachment events

[Adrian]

We are using a custom class handling int-based enums (since 1.0 didn't have python-native enum support) with an autogenerated CHECK constraint. In 1.1 this constraint is not added anymore.

In the changelog entry regarding TypeDecorator I couldn't see any reason why _set_table would not be called anymore.

from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.sql.sqltypes import SchemaType
from enum import Enum


Base = declarative_base()


class PyIntEnum(TypeDecorator, SchemaType):
    impl = SmallInteger

    def __init__(self, enum=None, exclude_values=None):
        self.enum = enum
        self.exclude_values = set(exclude_values or ())
        TypeDecorator.__init__(self)
        SchemaType.__init__(self)

    def process_bind_param(self, value, dialect):
        return int(value) if value is not None else None

    def process_result_value(self, value, dialect):
        pass  # not relevant for DDL

    def _set_table(self, column, table):
        e = CheckConstraint(type_coerce(column, self).in_(x.value for x in self.enum if x not in self.exclude_values),
                            'valid_enum_{}'.format(column.name))
        e.info['alembic_dont_render'] = True
        assert e.table is table


class MyEnum(int, Enum):
    a = 1
    b = 2
    c = 3


class Foo(Base):
    __tablename__ = 'foo'
    id = Column(Integer, primary_key=True)
    enum = Column(PyIntEnum(MyEnum))


e = create_engine('postgresql:///test', echo=True)
Base.metadata.drop_all(e)
Base.metadata.create_all(e)

SQL emitted in 1.1:

CREATE TABLE foo (
        id SERIAL NOT NULL,
        enum SMALLINT,
        PRIMARY KEY (id)
)

SQL emitted in 1.0

CREATE TABLE foo (
        id SERIAL NOT NULL,
        enum SMALLINT,
        PRIMARY KEY (id),
        CONSTRAINT valid_enum_enum CHECK (enum IN (1, 2, 3))
)

[Adrian]

Adding this solves the problem, but no idea if it's the proper way:

    def _set_parent(self, column):
        TypeDecorator._set_parent(self, column)
        SchemaType._set_parent(self, column)

    def _set_parent_with_dispatch(self, parent):
        TypeDecorator._set_parent_with_dispatch(self, parent)
        SchemaType._set_parent_with_dispatch(self, parent)

[Mike Bayer]

Well, _set_table() is a private method anyway. I'm all for supporting your use case but I think you'd want to use normal event hooks here.

[Adrian]

How would I do this with event hooks on the type level (i.e. without having to add extra code for each model using the type)?

[Mike Bayer]

so thats the bug, that before_parent_attach() seems to not be working when TypeDecorator is involved.

from sqlalchemy import *
from sqlalchemy.types import SchemaType
from sqlalchemy import event


class MyType(SchemaType, TypeDecorator):
    impl = String
    pass


@event.listens_for(MyType, "before_parent_attach")
def go(target, parent):
    print "yes"

Column('q', MyType())

[Mike Bayer]

which is ....because this style of inheritance has never been tested or done before.

[Adrian]

I remember it was mostly trial&error to get this working when I wrote it some time ago ;)

Never noticed the parent_attach events before (are they somewhat new?), but now I managed to get it to work with an event hook. Having to use the internal _on_table_attach is not super pretty but it seems cleaner than copying it (even the method is pretty simple)

@listens_for(PyIntEnum, 'after_parent_attach')
def _after_parent_attach(type_, column):
    @column._on_table_attach
    def _on_table_attach(column, table):
        values = {x.value for x in type_.enum if x not in type_.exclude_values}
        e = CheckConstraint(type_coerce(column, column.type).in_(values), name='valid_enum_{}'.format(column.name))
        e.info['alembic_dont_render'] = True
        assert e.table is table

[Mike Bayer]

yep so this is #2919 in ed535649d423020c816e66869016992df25e456e , which ensured that the vastly more typical approach of TypeDecorator around the SchemaType worked, and at the same time never thought the TypeDecorator itself would be the SchemaType.

[Mike Bayer]

here is your public API for _set_table:

@event.listens_for(PyIntEnum, "before_parent_attach")
def evt(typ, col):
    @event.listens_for(col, "after_parent_attach")
    def evt2(col, table):
        e = CheckConstraint(type_coerce(col, typ).in_(x.value for x in typ.enum if x not in typ.exclude_values),
                            'valid_enum_{}'.format(col.name))
        e.info['alembic_dont_render'] = True
        assert e.table is table

[Adrian]

Thanks!

Small suggestion for the docs: Mention when to use before/after parent attach (what's the reason in your example to use before on the type level but then after on the column level)?

[Mike Bayer]

https://gerrit.sqlalchemy.org/217

[Mike Bayer]

well you're doing "e.table is table" so that event needs to be "after". The outer event is "before" because I just tend to use that event first, it's before things have been set up so tends to be more useful. might work either way here.

[Mike Bayer]

Ensure TypeDecorator delegates _set_parent_with_dispatch

Ensure TypeDecorator delegates _set_parent_with_dispatch as well as _set_parent to itself as well as its impl, as the TypeDecorator class itself may have an active SchemaType implementation as well.

Fixed regression which occurred as a side effect of 2919, which in the less typical case of a user-defined :class:.TypeDecorator that was also itself an instance of :class:.SchemaType (rather than the implementation being such) would cause the column attachment events to be skipped for the type itself.

Change-Id: I0afb498fd91ab7d948e4439e7323a89eafcce0bc Fixes: #3832

→ <<cset 30bd28fca209>>