implement "clausewalker", "schemawalker"

[Mike Bayer]

the accept_visitor mechanisms in SA are inconsistent. in both cases, Columns and ColumnElements are visited inconsistently, particularly for ClauseVisitor where Columns get visited within WHERE clauses but not as they appear in tables, selects, and aliases ! also, cranking up the "visiting" for the various selectables to also include their columns is redundant since in many cases we are already visiting columns in other ways....this version of the fix adds 5-10 seconds to the unit test suite which is not acceptable.

so move all "traversal" to a set of "walker" objects, something like AST walkers but a little more comprehensive, which can be configured as to what theyre going to visit and what theyre not...so that people who want to use the visitor system can have something clear to work with, instead of arbitrary elements of the node just not getting visited.

i think what we should do is all ClauseElements get a get_elements() method, which returns the full "child" list of stuff, like a select has its froms, columns, etc. the accept_visitor on each ClauseElement just contains a single visit_XXX line within it.

The default "walker" will, just like ast's, call get_elements() on every node to just visit everything by default. then ansicompiler can make its own walker subclass that doesnt visit as deeply for selects, tables, etc. so we can save the redundant hits in that case.

so we decouple traversal from visiting, etc, all that stuff.

[Former user]

(original author: sdobrev) this get_elements(): - may have argument/s specifing what to include, e.g. i want columns-like-stuff only - or i want selectables only... - may be an iterator instead of list - is it possible that the thing being visited changes it structure in the meantime? i guess so - if the visitor does that...

so one may get a traverse( select1, columns=True) which walk and visit any column-like stuff within this select1. some visitor state/context (e.g. levels/stack/...) would be useful too - so the visitor knows where it is now.

[Mike Bayer]

get_elements() returns everything. if you dont want everything, that assumes you know about that particular type; then you dont use get_elements(), you call visit methods for the individual elements of that construct.

read python's 'compiler' module docs/source to get an idea for this. particularly compiler.ast, compiler.visitor. Each Node subclass contains a getChildNodes() method which is used by the basic visitor to fully traverse the tree without having to know about each specific node type. implementing specific visitXXX methods allows you to put a specific visit pattern in each method that does not necessarily use getChildNodes().

[Mike Bayer]

Ok did it slightly differently, more simply (well more explicitly/faster) than the "ast" version. check out changeset:2403 for details.

[Mike Bayer]

Removing milestone: 0.4.0 (automated comment)