support "down" adaptions? (was: sqlalchemy.dialects.mysql.base.ENUM is not adaptable to sqlalchemy.sql.sqltypes.Enum)

[Former user]

Given a MySQL table like:

CREATE TABLE foo ( bar enum('false','true') );

the reflected foo.bar column has a type of sqlalchemy.dialects.mysql.base.ENUM. Attempting to use the adapt method results in a TypeError due to bad arguments being passed to SchemaType.__init__:

>>> meta.tables['foo']('foo').columns['bar']('bar').type.adapt(Enum)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "SQLAlchemy-0.9.2dev-py2.7-linux-x86_64.egg/sqlalchemy/dialects/mysql/base.py", line 1194, in adapt
    return sqltypes.Enum.adapt(self, impltype, **kw)
  File "SQLAlchemy-0.9.2dev-py2.7-linux-x86_64.egg/sqlalchemy/sql/sqltypes.py", line 1145, in adapt
    **kw
  File "SQLAlchemy-0.9.2dev-py2.7-linux-x86_64.egg/sqlalchemy/sql/sqltypes.py", line 1110, in __init__
    SchemaType.__init__(self, **kw)
TypeError: __init__() got an unexpected keyword argument 'strict'
>>>

It looks like git rev 2692238f may have introduced this by changing the signature of SchemaType.__init__ from **kw to something a little more restricted.

Consider strict was ignored even before 2692238f I wonder if sqlalchemy.dialects.mysql.base.ENUM.adapt should just be deleted to let the inherited implementation be used instead.

Encountered while trying the sqlacodegen project. Bug first reported against that project: https://bitbucket.org/agronholm/sqlacodegen/issue/9/get-typeerror-init-got-an-unexpected

[Mike Bayer]

strict is not ignored, it is used by mysql.ENUM.bind_processor. It's tested in mysql.test_types:EnumSetTest.test_enum which fails if you comment that line out, as currently the ENUM is adapted to a new object with the same type.

The issue here is related to what I would characterize as unintended use of the adapt() method - this method is until now only intended for "up" adaptions, that is from a more generic type to a dialect-specific type, and not for a "down" adaption from a specific type to a more general. That use case doesn't exist within SQLAlchemy internals.

What's special about "strict" here as opposed to all the other custom keyword arguments that other MySQL types and other types from other dialects include, is that it is dialect-specific but also affects the runtime type-coercion behavior. That's why this special adapt() method exists - because adapt() is already only used to produce a new type object that's appropriate for run-time type coercion.

in the vast majority of cases elsewhere I've just looked at, types have predictable argument signatures because all keyword arguments are ultimately explicitly within the __init__ method of the type itself or a superclass; the default adapt() method looks at __bases__ to get at these additional arguments. Enum is special because it starts out with *varargs, and Python 2 doesn't provide Py3K's nice "keyword-only arguments" feature that we'd otherwise be using here.

We have a suite that tests all possible legal "up" adaptions but not "down" adaptions. That would need to be added. Autocode should be able to fix this right now just by using the adapt() of the general type:

   Enum.adapt(meta.tables['foo']('foo').columns['bar']('bar').type, Enum)

[Mike Bayer]

I made some alterations to test.sql.test_types:AdaptTest.test_adapt_method to also run through all possible "down" adaptions, it currently does all potential "up" adaptions. Most of the additional failures are of the same nature as this one:

FAILURE: <class 'sqlalchemy.dialects.sqlite.base.DATETIME'> -> <class 'sqlalchemy.sql.sqltypes.DateTime'>  __init__() got an unexpected keyword argument 'storage_format'
FAILURE: <class 'sqlalchemy.dialects.mysql.base.ENUM'> -> <class 'sqlalchemy.sql.sqltypes.Enum'>  __init__() got an unexpected keyword argument 'strict'
FAILURE: <class 'sqlalchemy.dialects.sqlite.base.DATE'> -> <class 'sqlalchemy.sql.sqltypes.Date'>  object() takes no parameters
FAILURE: <class 'sqlalchemy.dialects.sqlite.base.TIME'> -> <class 'sqlalchemy.sql.sqltypes.Time'>  __init__() got an unexpected keyword argument 'storage_format'

[Mike Bayer]

1732414076677e8fb8413 includes down adaption tests and fixes to MySQL, SQLite

[Former user]

Thanks!