disregard compiled.prefetch for executemany; optimize default usage; consider dumping most of DefaultRunner

Issue #1566 resolved
Mike Bayer repo owner created an issue
  • for an executemany we don't set up prefetch_cols or postfetch_cols. we shouldn't look at compiled.prefetch either - we are looping through each parameter set so if there are any python-level defaults, we really have to go through the whole list and look in those keys for None.

  • if the column default is a constant value and not a callable, just determine that up front before looping through all the parameter sets.

  • none of this will work well with DefaultRunner, which at this point is a silly object from an ancient line of thinking (that of using "visitor" pattern). dealing with ColumnDefault is entirely an isinstance() game these days. Just move the "visit_sequence()" part of it onto a dialect.execute_sequence() method. The rest of it just shove into Connection or some simpler object.

  • ColumnDefaults and all that are static objects. We can kill all the calls to isinstance() in compiler and the ones we're adding here by just putting some quickie memoized flags on the object - is_sequence, is_python_callable, is_server_side, etc. that will almost certainly give us some methods back in zoomark.

Comments (5)

  1. Mike Bayer reporter

    I've got the flags in ad89932715193275d37b5e22b830f092e350b1fe. Getting rid of DefaultRunner is much harder than I realized, since PG has an elaborate get_column_default() method. It's also difficult to create the same optimized behavior. DefaultRunner should still be optimized though and I'd favor still dropping the "visit" behavior from it.

  2. Mike Bayer reporter

    in 6400, took a different tack. the keys in the executemany() dict are now required based on what was in the first one. This allows us to keep all the guesswork out of dealing with defaults and such on a per-paramset basis, prevents anyone from being surprised about defaults not firing off, and maintains complete consistency between python-side, SQL-side, and server side defaults. hooray ! plus put the "default.arg" optimization in.

  3. Log in to comment