table.tometadata() fails on index that was created on interim version of table

Issue #4279 resolved
Michael Bayer
repo owner created an issue

This seems to be a regression in 1.2 due to #4147

from sqlalchemy import *

m1 = MetaData()

t1 = Table(
    'test',
    m1,
    Column('id', Integer, primary_key=True),
    Column('eventtime', DateTime),
)

Index("ix_test", t1.c.eventtime)

t1 = Table(
    'test',
    m1,
    Column('id', Integer, primary_key=True),
    Column('eventtime', DateTime),
    extend_existing=True
)


m2 = MetaData()

t2 = t1.tometadata(m2)

output:

Traceback (most recent call last):
  File "test.py", line 25, in <module>
    t2 = t1.tometadata(m2)
  File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/sql/schema.py", line 911, in tometadata
    **index.kwargs)
  File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/sql/schema.py", line 3447, in __init__
    self._set_parent(table)
  File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/sql/schema.py", line 3458, in _set_parent
    table.description
sqlalchemy.exc.ArgumentError: Index 'ix_test' is against table 'test', and cannot be associated with table 'test'.

Comments (9)

  1. colladoman

    Thank you!

    For me it would a be perfect workaround (and a simple and great new feature) doing something like that in tometadata method:

        def tometadata(self, metadata, schema=RETAIN_SCHEMA,
                       referred_schema_fn=None, name=None, copy_indexes=True):
    ...
    ...
    ...
        if copy_indexes:
            for index in self.indexes:
                ...
                ...
                ...
    
  2. Michael Bayer reporter

    fine grained control over tometadata() would be nice but if you make it "COPY_X=True" kind of thing, what about: unique constraints, primary key constraints, check constraints, Posgresql-specific EXCLUDES contraints, etc, then you have an API like:

     def tometadata(self, metadata, schema=RETAIN_SCHEMA,
                   referred_schema_fn=None, name=None, copy_indexes=True, copy_pk=True, copy_check=True, copy_unqiue=True, postgresql_copy_exclude=True, ...):
    

    dialects like Postgresql need to be able to inject new "mydialect_copy_myddl=True" options into it, so that would be **kw as well.

    I usually do exclusion APIs more like this:

     def tometadata(self, metadata, schema=RETAIN_SCHEMA,
                   referred_schema_fn=None, name=None, include_object=None):
    

    include_object is a function that receives each object, it can then return True/False to indicate if that object should be included. metadata.reflect() has this as does the autogenerate system in Alembic.

  3. colladoman

    Yours is really a much better aproach, but i can't fully understand how it should be used. I would appreciate an example when it is implemented. Thank you again!

  4. Michael Bayer reporter

    Lookup index columns in parent table by key for copy

    Fixed regression in 1.2 due to 🎫4147 where a :class:.Table that has had some of its indexed columns redefined with new ones, as would occur when overriding columns during reflection or when using :paramref:.Table.extend_existing, such that the :meth:.Table.tometadata method would fail when attempting to copy those indexes as they still referred to the replaced column. The copy logic now accommodates for this condition.

    Change-Id: I521aa2c9f3baa0e84598bbdd6ffe4bf07b6e3ba8 Fixes: #4279

    → <<cset 8f7766cc6147>>

  5. Michael Bayer reporter

    Lookup index columns in parent table by key for copy

    Fixed regression in 1.2 due to 🎫4147 where a :class:.Table that has had some of its indexed columns redefined with new ones, as would occur when overriding columns during reflection or when using :paramref:.Table.extend_existing, such that the :meth:.Table.tometadata method would fail when attempting to copy those indexes as they still referred to the replaced column. The copy logic now accommodates for this condition.

    Change-Id: I521aa2c9f3baa0e84598bbdd6ffe4bf07b6e3ba8 Fixes: #4279 (cherry picked from commit 8f7766cc61479f3c9220c640230eeecd3d49ccc8)

    → <<cset b6f479b5398d>>

  6. Log in to comment