-*- coding: utf-8; fill-column: 68 -*-
- This section documents those changes from 0.7b4
to 0.7.0. For an overview of what's new in
SQLAlchemy 0.7, see
- Fixed regression introduced in 0.7b4 (!) whereby
query.options(someoption("nonexistent name")) would
fail to raise an error. Also added additional
error catching for cases where the option would
try to build off a column-based element, further
fixed up some of the error messages tailored
- query.count() emits "count(*)" instead of
- Fine tuning of Query clause adaptation when
from_self(), union(), or other "select from
myself" operation, such that plain SQL expression
elements added to filter(), order_by() etc.
which are present in the nested "from myself"
query *will* be adapted in the same way an ORM
expression element will, since these
elements are otherwise not easily accessible.
- Fixed bug where determination of "self referential"
relationship would fail with no workaround
for joined-inh subclass related to itself,
or joined-inh subclass related to a subclass
of that with no cols in the sub-sub class
in the join condition. [ticket:2149]
Also in 0.6.8.
- mapper() will ignore non-configured foreign keys
to unrelated tables when determining inherit
condition between parent and child class,
but will raise as usual for unresolved
columns and table names regarding the inherited
table. This is an enhanced generalization of
behavior that was already applied to declarative
previously. [ticket:2153] 0.6.8 has a more
conservative version of this which doesn't
fundamentally alter how join conditions
- It is an error to call query.get() when the
given entity is not a single, full class
entity or mapper (i.e. a column). This is
a deprecation warning in 0.6.8.
- Fixed a potential KeyError which under some
circumstances could occur with the identity
map, part of [ticket:2148]
- added Query.with_session() method, switches
Query to use a different session.
- horizontal shard query should use execution
options per connection as per [ticket:2131]
- a non_primary mapper will inherit the _identity_class
of the primary mapper. This so that a non_primary
established against a class that's normally in an
inheritance mapping will produce results that are
identity-map compatible with that of the primary
mapper [ticket:2151] (also in 0.6.8)
- Fixed the error message emitted for "can't
execute syncrule for destination column 'q';
mapper 'X' does not map this column" to
reference the correct mapper. [ticket:2163].
Also in 0.6.8.
- polymorphic_union() gets a "cast_nulls" option,
disables the usage of CAST when it renders
the labeled NULL columns. [ticket:1502]
- polymorphic_union() renders the columns in their
original table order, as according to the first
table/selectable in the list of polymorphic
unions in which they appear. (which is itself
an unordered mapping unless you pass an OrderedDict).
- Fixed bug whereby mapper mapped to an anonymous
alias would fail if logging were used, due to
unescaped % sign in the alias name. [ticket:2171]
Also in 0.6.8.
- Fixed bug whereby nesting a label of a select()
with another label in it would produce incorrect
exported columns. Among other things this would
break an ORM column_property() mapping against
another column_property(). [ticket:2167].
Also in 0.6.8
- Changed the handling in determination of join
conditions such that foreign key errors are
only considered between the two given tables.
That is, t1.join(t2) will report FK errors
that involve 't1' or 't2', but anything
involving 't3' will be skipped. This affects
join(), as well as ORM relationship and
inherit condition logic.
- Some improvements to error handling inside
of the execute procedure to ensure auto-close
connections are really closed when very
unusual DBAPI errors occur.
- metadata.reflect() and reflection.Inspector()
had some reliance on GC to close connections
which were internally procured, fixed this.
- Added explicit check for when Column .name
is assigned as blank string [ticket:2140]
- Fixed bug whereby if FetchedValue was passed
to column server_onupdate, it would not
have its parent "column" assigned, added
test coverage for all column default assignment
patterns. [ticket:2147] also in 0.6.8
- Fixed the psycopg2_version parsing in the
- Fixed bug affecting PG 9 whereby index reflection
would fail if against a column whose name
had changed. [ticket:2141]. Also in 0.6.8.
- Fixed bug in MSSQL dialect whereby the aliasing
applied to a schema-qualified table would leak
into enclosing select statements [ticket:2169].
Also in 0.6.8.
- Removed the usage of the "collections.MutableMapping"
abc from the ext.mutable docs as it was being used
incorrectly and makes the example more difficult
to understand in any case. [ticket:2152]
- removed the ancient "polymorphic association"
examples and replaced with an updated set of
examples that use declarative mixins,
"generic_associations". Each presents an alternative
- Fixed bugs in sqlalchemy.ext.mutable extension where
`None` was not appropriately handled, replacement
events were not appropriately handled.
- Changes to the format of CHANGES, this file.
The format changes have been applied to
the 0.7 releases.
- The "-declarative" changes will now be listed
directly under the "-orm" section, as these
are closely related.
- The 0.5 series changes have been moved to
the file CHANGES_PRE_06 which replaces
- The changelog for 0.6.7 and subsequent within
the 0.6 series is now listed only in the
CHANGES file within the 0.6 branch.
In the 0.7 CHANGES file (i.e. this file), all the
0.6 changes are listed inline within the 0.7
section in which they were also applied
(since all 0.6 changes are in 0.7 as well).
Changes that apply to an 0.6 version here
are noted as are if any differences in
implementation/behavior are present.
- Some fixes to "evaulate" and "fetch" evaluation
when query.update(), query.delete() are called.
The retrieval of records is done after autoflush
in all cases, and before update/delete is
emitted, guarding against unflushed data present
as well as expired objects failing during
the evaluation. [ticket:2122]
- Reworded the exception raised when a flush
is attempted of a subclass that is not polymorphic
against the supertype. [ticket:2063]
- Still more wording adjustments when a query option
can't find the target entity. Explain that the
path must be from one of the root entities.
- Some fixes to the state handling regarding
backrefs, typically when autoflush=False, where
the back-referenced collection wouldn't
properly handle add/removes with no net
change. Thanks to Richard Murri for the
test case + patch. [ticket:2123]
(also in 0.6.7).
- Added checks inside the UOW to detect the unusual
condition of being asked to UPDATE or DELETE
on a primary key value that contains NULL
in it. [ticket:2127]
- Some refinements to attribute history. More
changes are pending possibly in 0.8, but
for now history has been modified such that
scalar history doesn't have a "side effect"
of populating None for a non-present value.
This allows a slightly better ability to
distinguish between a None set and no actual
change, affects [ticket:2127] as well.
- a "having" clause would be copied from the
inside to the outside query if from_self()
were used; in particular this would break
an 0.7 style count() query [ticket:2130].
(also in 0.6.7)
- the Query.execution_options() method now passes
those options to the Connection rather than
the SELECT statement, so that all available
options including isolation level and
compiled cache may be used. [ticket:2131]
- The "compiled_cache" execution option now raises
an error when passed to a SELECT statement
rather than a Connection. Previously it was
being ignored entirely. We may look into
having this option work on a per-statement
level at some point. [ticket:2131]
- Restored the "catchall" constructor on the base
TypeEngine class, with a deprecation warning.
This so that code which does something like
Integer(11) still succeeds.
- Fixed regression whereby MetaData() coming
back from unpickling did not keep track of
new things it keeps track of now, i.e.
collection of Sequence objects, list
of schema names. [ticket:2104]
- The limit/offset keywords to select() as well
as the value passed to select.limit()/offset()
will be coerced to integer. [ticket:2116]
(also in 0.6.7)
- fixed bug where "from" clause gathering from an
over() clause would be an itertools.chain() and
not a list, causing "can only concatenate list"
TypeError when combined with other clauses.
- Fixed incorrect usage of "," in over() clause
being placed between the "partition" and "order by"
- Before/after attach events for PrimaryKeyConstraint
now function, tests added for before/after events
on all constraint types. [ticket:2105]
- Added explicit true()/false() constructs to expression
lib - coercion rules will intercept "False"/"True"
into these constructs. In 0.6, the constructs were
typically converted straight to string, which was
no longer accepted in 0.7. [ticket:2117]
- The C extension is now enabled by default on CPython
2.x with a fallback to pure python if it fails to
- The 'useexisting' flag on Table has been superceded
by a new pair of flags 'keep_existing' and
'extend_existing'. 'extend_existing' is equivalent
to 'useexisting' - the existing Table is returned,
and additional constructor elements are added.
With 'keep_existing', the existing Table is returned,
but additional constructor elements are not added -
these elements are only applied when the Table
is newly created. [ticket:2109]
- REAL has been added to the core types. Supported
by Postgresql, SQL Server, MySQL, SQLite. Note
that the SQL Server and MySQL versions, which
add extra arguments, are also still available
from those dialects. [ticket:2081]
- Added @event.listens_for() decorator, given
target + event name, applies the decorated
function as a listener. [ticket:2106]
- AssertionPool now stores the traceback indicating
where the currently checked out connection was
acquired; this traceback is reported within
the assertion raised upon a second concurrent
checkout; courtesy Gunnlaugur Briem
- The "pool.manage" feature doesn't use pickle
anymore to hash the arguments for each pool.
- Fixed bug where reflection of foreign key
created as "REFERENCES <tablename>" without
col name would fail. [ticket:2115]
(also in 0.6.7)
- Psycopg2 for Python 3 is now supported.
- Fixed support for precision numerics when using
- Using column names that would require quotes
for the column itself or for a name-generated
bind parameter, such as names with special
characters, underscores, non-ascii characters,
now properly translate bind parameter keys when
talking to cx_oracle. [ticket:2100] (Also
- Oracle dialect adds use_binds_for_limits=False
create_engine() flag, will render the LIMIT/OFFSET
values inline instead of as binds, reported to
modify the execution plan used by Oracle.
[ticket:2116] (Also in 0.6.7)
- Documented SQLite DATE/TIME/DATETIME types.
[ticket:2029] (also in 0.6.7)
- Fixed mutable extension docs to show the
correct type-association methods.
- Lots of fixes to unit tests when run under Pypy
(courtesy Alex Gaynor).
- Changed the underlying approach to query.count().
query.count() is now in all cases exactly:
That is, "select count(1) from (<full query>)".
This produces a subquery in all cases, but
vastly simplifies all the guessing count()
tried to do previously, which would still
fail in many scenarios particularly when
joined table inheritance and other joins
were involved. If the subquery produced
for an otherwise very simple count is really
an issue, use query(func.count()) as an
- some changes to the identity map regarding
rare weakref callbacks during iterations.
The mutex has been removed as it apparently
can cause a reentrant (i.e. in one thread) deadlock,
perhaps when gc collects objects at the point of
iteration in order to gain more memory. It is hoped
that "dictionary changed during iteration" will
be exceedingly rare as iteration methods internally
acquire the full list of objects in a single values()
call. Note 0.6.7 has a more conservative fix here
which still keeps the mutex in place. [ticket:2087]
- A tweak to the unit of work causes it to order
the flush along relationship() dependencies even if
the given objects don't have any inter-attribute
references in memory, which was the behavior in
0.5 and earlier, so a flush of Parent/Child with
only foreign key/primary key set will succeed.
This while still maintaining 0.6 and above's not
generating a ton of useless internal dependency
structures within the flush that don't correspond
to state actually within the current flush.
- Improvements to the error messages emitted when
querying against column-only entities in conjunction
with (typically incorrectly) using loader options,
where the parent entity is not fully present.
- Fixed bug in query.options() whereby a path
applied to a lazyload using string keys could
overlap a same named attribute on the wrong
entity. Note 0.6.7 has a more conservative fix
to this. [ticket:2098]
- Arguments in __mapper_args__ that aren't "hashable"
aren't mistaken for always-hashable, possibly-column
arguments. [ticket:2091] (also in 0.6.7)
- Added a fully descriptive error message for the
case where Column is subclassed and _make_proxy()
fails to make a copy due to TypeError on the
constructor. The method _constructor should
be implemented in this case.
- Added new event "column_reflect" for Table objects.
Receives the info dictionary about a Column before
the object is generated within reflection, and allows
modification to the dictionary for control over
most aspects of the resulting Column including
key, name, type, info dictionary. [ticket:2095]
- To help with the "column_reflect" event being used
with specific Table objects instead of all instances
of Table, listeners can be added to a Table object
inline with its construction using a new argument
"listeners", a list of tuples of the form
(<eventname>, <fn>), which are applied to the Table
before the reflection process begins.
- Added new generic function "next_value()", accepts
a Sequence object as its argument and renders the
appropriate "next value" generation string on the
target platform, if supported. Also provides
".next_value()" method on Sequence itself.
- func.next_value() or other SQL expression can
be embedded directly into an insert() construct,
and if implicit or explicit "returning" is used
in conjunction with a primary key column,
the newly generated value will be present in
- Added accessors to ResultProxy "returns_rows",
"is_insert" [ticket:2089] (also in 0.6.7)
- Fixed AssertionPool regression bug. [ticket:2097]
- Changed exception raised to ArgumentError when an
invalid dialect is specified. [ticket:2060]
- Added RESERVED_WORDS for postgresql dialect. [ticket:2092]
(also in 0.6.7)
- Fixed the BIT type to allow a "length" parameter, "varying"
parameter. Reflection also fixed. [ticket:2073]
(also in 0.6.7)
- Rewrote the query used to get the definition of a view,
typically when using the Inspector interface, to
use sys.sql_modules instead of the information schema,
thereby allowing views definitions longer than 4000
characters to be fully returned. [ticket:2071]
(also in 0.6.7)
- The "implicit_returning" flag on create_engine() is
honored if set to False. [ticket:2083] (also in 0.6.7)
- Added RESERVED_WORDS informix dialect. [ticket:2092]
(also in 0.6.7)
- The horizontal_shard ShardedSession class accepts the common
Session argument "query_cls" as a constructor argument,
to enable further subclassing of ShardedQuery.
[ticket:2090] (also in 0.6.7)
- Updated the association, association proxy examples
to use declarative, added a new example
dict_of_sets_with_default.py, a "pushing the envelope"
example of association proxy.
- The Beaker caching example allows a "query_cls" argument
to the query_callable() function. [ticket:2090]
(also in 0.6.7)
- Fixed bug whereby Session.merge() would call the
load() event with one too few arguments.
- Added logic which prevents the generation of
events from a MapperExtension or SessionExtension
from generating do-nothing events for all the methods
not overridden. [ticket:2052]
- Fixed regression whereby composite() with
Column objects placed inline would fail
to initialize. The Column objects can now
be inline with the composite() or external
and pulled in via name or object ref.
- Fix error message referencing old @classproperty
name to reference @declared_attr [ticket:2061]
(also in 0.6.7)
- the dictionary at the end of the __table_args__
tuple is now optional. [ticket:1468]
- Renamed the EngineEvents event class to
ConnectionEvents. As these classes are never
accessed directly by end-user code, this strictly
is a documentation change for end users. Also
simplified how events get linked to engines
and connections internally. [ticket:2059]
- The Sequence() construct, when passed a MetaData()
object via its 'metadata' argument, will be
included in CREATE/DROP statements within
metadata.create_all() and metadata.drop_all(),
including "checkfirst" logic. [ticket:2055]
- The Column.references() method now returns True
if it has a foreign key referencing the
given column exactly, not just it's parent
- Fixed regression from 0.6 where SMALLINT and
BIGINT types would both generate SERIAL
on an integer PK column, instead of
SMALLINT and BIGSERIAL [ticket:2065]
- Association proxy now has correct behavior for
any(), has(), and contains() when proxying
a many-to-one scalar attribute to a one-to-many
collection (i.e. the reverse of the 'typical'
association proxy use case) [ticket:2054]
- Beaker example now takes into account 'limit'
and 'offset', bind params within embedded
FROM clauses (like when you use union() or
from_self()) when generating a cache key.
- Detailed descriptions of each change below are
- New event system, supercedes all extensions, listeners,
- Logging enhancements
- Setup no longer installs a Nose plugin
- The "sqlalchemy.exceptions" alias in sys.modules
has been removed. Base SQLA exceptions are
available via "from sqlalchemy import exc".
The "exceptions" alias for "exc" remains in
"sqlalchemy" for now, it's just not patched into
- More succinct form of query.join(target, onclause)
- Hybrid Attributes, implements/supercedes synonym()
- Rewrite of composites [ticket:2008]
- Mutation Event Extension, supercedes "mutable=True"
- PickleType and ARRAY mutability turned off by default
- Simplified polymorphic_on assignment
- Flushing of Orphans that have no parent is allowed
- Adjusted flush accounting step to occur before
the commit in the case of autocommit=True. This allows
autocommit=True to work appropriately with
expire_on_commit=True, and also allows post-flush session
hooks to operate in the same transactional context
as when autocommit=False. [ticket:2041]
- Warnings generated when collection members, scalar referents
not part of the flush
- Non-`Table`-derived constructs can be mapped
- Tuple label names in Query Improved
- Mapped column attributes reference the most specific
- Mapping to joins with two or more same-named columns
requires explicit declaration
- Mapper requires that polymorphic_on column be present
in the mapped selectable
- compile_mappers() renamed configure_mappers(), simplified
- the aliased() function, if passed a SQL FromClause element
(i.e. not a mapped class), will return element.alias()
instead of raising an error on AliasedClass. [ticket:2018]
- Session.merge() will check the version id of the incoming
state against that of the database, assuming the mapping
uses version ids and incoming state has a version_id
assigned, and raise StaleDataError if they don't
- Session.connection(), Session.execute() accept 'bind',
to allow execute/connection operations to participate
in the open transaction of an engine explicitly.
- Query.join(), Query.outerjoin(), eagerload(),
eagerload_all(), others no longer allow lists
of attributes as arguments (i.e. option([x, y, z])
form, deprecated since 0.5)
- ScopedSession.mapper is removed (deprecated since 0.5).
- Horizontal shard query places 'shard_id' in
context.attributes where it's accessible by the
"load()" event. [ticket:2031]
- A single contains_eager() call across
multiple entities will indicate all collections
along that path should load, instead of requiring
distinct contains_eager() calls for each endpoint
(which was never correctly documented).
- The "name" field used in orm.aliased() now renders
in the resulting SQL statement.
- Session weak_instance_dict=False is deprecated.
- An exception is raised in the unusual case that an
append or similar event on a collection occurs after
the parent object has been dereferenced, which
prevents the parent from being marked as "dirty"
in the session. Was a warning in 0.6.6.
- Query.distinct() now accepts column expressions
as *args, interpreted by the Postgresql dialect
as DISTINCT ON (<expr>). [ticket:1069]
- Additional tuning to "many-to-one" relationship
loads during a flush(). A change in version 0.6.6
([ticket:2002]) required that more "unnecessary" m2o
loads during a flush could occur. Extra loading modes have
been added so that the SQL emitted in this
specific use case is trimmed back, while still
retrieving the information the flush needs in order
to not miss anything. [ticket:2049]
- the value of "passive" as passed to
attributes.get_history() should be one of the
constants defined in the attributes package. Sending
True or False is deprecated.
- Added a `name` argument to `Query.subquery()`, to allow
a fixed name to be assigned to the alias object.
[ticket:2030] (also in 0.6.7)
- A warning is emitted when a joined-table inheriting mapper
has no primary keys on the locally mapped table
(but has pks on the superclass table). [ticket:2019]
(also in 0.6.7)
- Fixed bug where "middle" class in a polymorphic hierarchy
would have no 'polymorphic_on' column if it didn't also
specify a 'polymorphic_identity', leading to strange
errors upon refresh, wrong class loaded when querying
from that target. Also emits the correct WHERE criterion
when using single table inheritance. [ticket:2038]
(also in 0.6.7)
- Fixed bug where a column with a SQL or server side default
that was excluded from a mapping with include_properties
or exclude_properties would result in UnmappedColumnError.
[ticket:1995] (also in 0.6.7)
- A warning is emitted in the unusual case that an
append or similar event on a collection occurs after
the parent object has been dereferenced, which
prevents the parent from being marked as "dirty"
in the session. This will be an exception in 0.7.
[ticket:2046] (also in 0.6.7)
- Added an explicit check for the case that the name
'metadata' is used for a column attribute on a
declarative class. [ticket:2050] (also in 0.6.7)
- Added over() function, method to FunctionElement
classes, produces the _Over() construct which
in turn generates "window functions", i.e.
"<window function> OVER (PARTITION BY <partition by>,
ORDER BY <order by>)".
- LIMIT/OFFSET clauses now use bind parameters
- select.distinct() now accepts column expressions
as *args, interpreted by the Postgresql dialect
as DISTINCT ON (<expr>). Note this was already
available via passing a list to the `distinct`
keyword argument to select(). [ticket:1069]
- select.prefix_with() accepts multiple expressions
(i.e. *expr), 'prefix' keyword argument to select()
accepts a list or tuple.
- Passing a string to the `distinct` keyword argument
of `select()` for the purpose of emitting special
MySQL keywords (DISTINCTROW etc.) is deprecated -
use `prefix_with()` for this.
- TypeDecorator works with primary key columns
- DDL() constructs now escape percent signs
- Table.c / MetaData.tables refined a bit, don't allow direct
mutation [ticket:1893] [ticket:1917]
- Callables passed to `bindparam()` don't get evaluated
- types.type_map is now private, types._type_map
- Non-public Pool methods underscored
- Added NULLS FIRST and NULLS LAST support. It's implemented
as an extension to the asc() and desc() operators, called
nullsfirst() and nullslast(). [ticket:723]
- The Index() construct can be created inline with a Table
definition, using strings as column names, as an alternative
to the creation of the index outside of the Table.
- execution_options() on Connection accepts
"isolation_level" argument, sets transaction isolation
level for that connection only until returned to the
connection pool, for thsoe backends which support it
(SQLite, Postgresql) [ticket:2001]
- A TypeDecorator of Integer can be used with a primary key
column, and the "autoincrement" feature of various dialects
as well as the "sqlite_autoincrement" flag will honor
the underlying database type as being Integer-based.
- Established consistency when server_default is present
on an Integer PK column. SQLA doesn't pre-fetch these,
nor do they come back in cursor.lastrowid (DBAPI).
Ensured all backends consistently return None
in result.inserted_primary_key for these. Regarding
reflection for this case, reflection of an int PK col
with a server_default sets the "autoincrement" flag to False,
except in the case of a PG SERIAL col where we detected a
sequence default. [ticket:2020] [ticket:2021]
- Result-row processors are applied to pre-executed SQL
defaults, as well as cursor.lastrowid, when determining
the contents of result.inserted_primary_key.
- Bind parameters present in the "columns clause" of a select
are now auto-labeled like other "anonymous" clauses,
which among other things allows their "type" to be meaningful
when the row is fetched, as in result row processors.
- TypeDecorator is present in the "sqlalchemy" import space.
- Non-DBAPI errors which occur in the scope of an `execute()`
call are now wrapped in sqlalchemy.exc.StatementError,
and the text of the SQL statement and repr() of params
is included. This makes it easier to identify statement
executions which fail before the DBAPI becomes
- The concept of associating a ".bind" directly with a
ClauseElement has been explicitly moved to Executable,
i.e. the mixin that describes ClauseElements which represent
engine-executable constructs. This change is an improvement
to internal organization and is unlikely to affect any
real-world usage. [ticket:2048]
- Column.copy(), as used in table.tometadata(), copies the
'doc' attribute. [ticket:2028] (also in 0.6.7)
- Added some defs to the resultproxy.c extension so that
the extension compiles and runs on Python 2.4.
[ticket:2023] (also in 0.6.7)
- The compiler extension now supports overriding the default
compilation of expression._BindParamClause including that
the auto-generated binds within the VALUES/SET clause
of an insert()/update() statement will also use the new
compilation rules. [ticket:2042] (also in 0.6.7)
- SQLite dialect now uses `NullPool` for file-based databases
- The path given as the location of a sqlite database is now
normalized via os.path.abspath(), so that directory changes
within the process don't affect the ultimate location
of a relative file path. [ticket:2036]
- When explicit sequence execution derives the name
of the auto-generated sequence of a SERIAL column,
which currently only occurs if implicit_returning=False,
now accommodates if the table + column name is greater
than 63 characters using the same logic Postgresql uses.
[ticket:1083] (also in 0.6.7)
- Added an additional libpq message to the list of "disconnect"
exceptions, "could not receive data from server"
[ticket:2044] (also in 0.6.7)
- the String/Unicode types, and their counterparts VARCHAR/
NVARCHAR, emit "max" as the length when no length is
specified, so that the default length, normally '1'
as per SQL server documentation, is instead
'unbounded'. This also occurs for the VARBINARY type.
This behavior makes these types more closely compatible
with Postgresql's VARCHAR type which is similarly unbounded
when no length is specified.
- New DBAPI support for pymysql, a pure Python port
of MySQL-python. [ticket:1991]
- oursql dialect accepts the same "ssl" arguments in
create_engine() as that of MySQLdb. [ticket:2047]
(also in 0.6.7)
- Some adjustments so that Interbase is supported as well.
FB/Interbase version idents are parsed into a structure
such as (8, 1, 1, 'interbase') or (2, 1, 588, 'firebird')
so they can be distinguished. [ticket:1885]
- New dialect for Drizzle, a MySQL variant. Uses MySQL-python
for the DBAPI. [ticket:2003]
0.6.7 - 0.6.xx
Changes which apply to 0.6.7 and subsequent versions of 0.6
are listed in the CHANGES file within the 0.6 branch. All
those changes which are also in the 0.7 series (which is typically
all of them) are listed inline within the 0.7 changes above,
those which apply to an 0.6 release are noted.
- Fixed bug whereby a non-"mutable" attribute modified event
which occurred on an object that was clean except for
preceding mutable attribute changes would fail to strongly
reference itself in the identity map. This would cause the
object to be garbage collected, losing track of any changes
that weren't previously saved in the "mutable changes"
- Fixed bug whereby "passive_deletes='all'" wasn't passing
the correct symbols to lazy loaders during flush, thereby
causing an unwarranted load. [ticket:2013]
- Fixed bug which prevented composite mapped
attributes from being used on a mapped select statement.
[ticket:1997]. Note the workings of composite are slated to
change significantly in 0.7.
- active_history flag also added to composite().
The flag has no effect in 0.6, but is instead
a placeholder flag for forwards compatibility,
as it applies in 0.7 for composites.
- Fixed uow bug whereby expired objects passed to
Session.delete() would not have unloaded references
or collections taken into account when deleting
objects, despite passive_deletes remaining at
its default of False. [ticket:2002]
- A warning is emitted when version_id_col is specified
on an inheriting mapper when the inherited mapper
already has one, if those column expressions are not
the same. [ticket:1987]
- "innerjoin" flag doesn't take effect along the chain
of joinedload() joins if a previous join in that chain
is an outer join, thus allowing primary rows without
a referenced child row to be correctly returned
in results. [ticket:1954]
- Fixed bug regarding "subqueryload" strategy whereby
strategy would fail if the entity was an aliased()
- Fixed bug regarding "subqueryload" strategy whereby
the join would fail if using a multi-level load
of the form from A->joined-subclass->C [ticket:2014]
- Fixed indexing of Query objects by -1. It was erroneously
transformed to the empty slice -1:0 that resulted in
- The mapper argument "primary_key" can be passed as a
single column as well as a list or tuple. [ticket:1971]
The documentation examples that illustrated it as a
scalar value have been changed to lists.
- Added active_history flag to relationship()
and column_property(), forces attribute events to
always load the "old" value, so that it's available to
- Query.get() will raise if the number of params
in a composite key is too large, as well as too
- Backport of "optimized get" fix from 0.7,
improves the generation of joined-inheritance
"load expired row" behavior. [ticket:1992]
- A little more verbiage to the "primaryjoin" error,
in an unusual condition that the join condition
"works" for viewonly but doesn't work for non-viewonly,
and foreign_keys wasn't used - adds "foreign_keys" to
the suggestion. Also add "foreign_keys" to the
suggestion for the generic "direction" error.
- Fixed operator precedence rules for multiple
chains of a single non-associative operator.
I.e. "x - (y - z)" will compile as "x - (y - z)"
and not "x - y - z". Also works with labels,
i.e. "x - (y - z).label('foo')"
- The 'info' attribute of Column is copied during
Column.copy(), i.e. as occurs when using columns
in declarative mixins. [ticket:1967]
- Added a bind processor for booleans which coerces
to int, for DBAPIs such as pymssql that naively call
str() on values.
- CheckConstraint will copy its 'initially', 'deferrable',
and '_create_rule' attributes within a copy()/tometadata()
- The "unicode warning" against non-unicode bind data
is now raised only when the
Unicode type is used explictly; not when
convert_unicode=True is used on the engine
or String type.
- Fixed memory leak in C version of Decimal result
- Implemented sequence check capability for the C
version of RowProxy, as well as 2.7 style
"collections.Sequence" registration for RowProxy.
- Threadlocal engine methods rollback(), commit(),
prepare() won't raise if no transaction is in progress;
this was a regression introduced in 0.6. [ticket:1998]
- Threadlocal engine returns itself upon begin(),
begin_nested(); engine then implements contextmanager
methods to allow the "with" statement. [ticket:2004]
- Single element tuple expressions inside an IN clause
parenthesize correctly, also from [ticket:1984]
- Ensured every numeric, float, int code, scalar + array,
are recognized by psycopg2 and pg8000's "numeric"
base type. [ticket:1955]
- Added as_uuid=True flag to the UUID type, will receive
and return values as Python UUID() objects rather than
strings. Currently, the UUID type is only known to
work with psycopg2. [ticket:1956]
- Fixed bug whereby KeyError would occur with non-ENUM
supported PG versions after a pool dispose+recreate
would occur, [ticket:1989]
- Fixed error handling for Jython + zxjdbc, such that
has_table() property works again. Regression from
0.6.3 (we don't have a Jython buildbot, sorry)
- The REFERENCES clause in a CREATE TABLE that includes
a remote schema to another table with the same schema
name now renders the remote name without
the schema clause, as required by SQLite. [ticket:1851]
- On the same theme, the REFERENCES clause in a CREATE TABLE
that includes a remote schema to a *different* schema
than that of the parent table doesn't render at all,
as cross-schema references do not appear to be supported.
- The rewrite of index reflection in [ticket:1770] was
unfortunately not tested correctly, and returned incorrect
results. This regression is now fixed.
- The cx_oracle "decimal detection" logic, which takes place
for for result set columns with ambiguous numeric characteristics,
now uses the decimal point character determined by the locale/
NLS_LANG setting, using an on-first-connect detection of
this character. cx_oracle 5.0.3 or greater is also required
when using a non-period-decimal-point NLS_LANG setting.
- Firebird numeric type now checks for Decimal explicitly,
lets float() pass right through, thereby allowing
special values such as float('inf'). [ticket:2012]
- An error is raised if __table_args__ is not in tuple
or dict format, and is not None. [ticket:1972]
- Added "map_to()" method to SqlSoup, which is a "master"
method which accepts explicit arguments for each aspect of
the selectable and mapping, including a base class per
- Mapped selectables used with the map(), with_labels(),
join() methods no longer put the given argument into the
internal "cache" dictionary. Particularly since the
join() and select() objects are created in the method
itself this was pretty much a pure memory leaking behavior.
- The versioning example now supports detection of changes
in an associated relationship().
- Added a new "lazyload" option "immediateload".
Issues the usual "lazy" load operation automatically
as the object is populated. The use case
here is when loading objects to be placed in
an offline cache, or otherwise used after
the session isn't available, and straight 'select'
loading, not 'joined' or 'subquery', is desired.
- New Query methods: query.label(name), query.as_scalar(),
return the query's statement as a scalar subquery
with /without label [ticket:1920];
query.with_entities(*ent), replaces the SELECT list of
the query with new entities.
Roughly equivalent to a generative form of query.values()
which accepts mapped entities as well as column
- Fixed recursion bug which could occur when moving
an object from one reference to another, with
backrefs involved, where the initiating parent
was a subclass (with its own mapper) of the
- Fixed a regression in 0.6.4 which occurred if you
passed an empty list to "include_properties" on
- Fixed labeling bug in Query whereby the NamedTuple
would mis-apply labels if any of the column
expressions were un-labeled.
- Patched a case where query.join() would adapt the
right side to the right side of the left's join
- Query.select_from() has been beefed up to help
ensure that a subsequent call to query.join()
will use the select_from() entity, assuming it's
a mapped entity and not a plain selectable,
as the default "left" side, not the first entity
in the Query object's list of entities.
- The exception raised by Session when it is used
subsequent to a subtransaction rollback (which is what
happens when a flush fails in autocommit=False mode) has
now been reworded (this is the "inactive due to a
rollback in a subtransaction" message). In particular,
if the rollback was due to an exception during flush(),
the message states this is the case, and reiterates the
string form of the original exception that occurred
during flush. If the session is closed due to explicit
usage of subtransactions (not very common), the message
just states this is the case.
- The exception raised by Mapper when repeated requests to
its initialization are made after initialization already
failed no longer assumes the "hasattr" case, since
there's other scenarios in which this message gets
emitted, and the message also does not compound onto
itself multiple times - you get the same message for
each attempt at usage. The misnomer "compiles" is being
traded out for "initialize".
- Fixed bug in query.update() where 'evaluate' or 'fetch'
expiration would fail if the column expression key was
a class attribute with a different keyname as the
actual column name. [ticket:1935]
- Added an assertion during flush which ensures
that no NULL-holding identity keys were generated
on "newly persistent" objects.
This can occur when user defined code inadvertently
triggers flushes on not-fully-loaded objects.
- lazy loads for relationship attributes now use
the current state, not the "committed" state,
of foreign and primary key attributes
when issuing SQL, if a flush is not in process.
Previously, only the database-committed state would
be used. In particular, this would cause a many-to-one
get()-on-lazyload operation to fail, as autoflush
is not triggered on these loads when the attributes are
determined and the "committed" state may not be
- A new flag on relationship(), load_on_pending, allows
the lazy loader to fire off on pending objects without a
flush taking place, as well as a transient object that's
been manually "attached" to the session. Note that this
flag blocks attribute events from taking place when an
object is loaded, so backrefs aren't available until
after a flush. The flag is only intended for very
specific use cases.
- Another new flag on relationship(), cascade_backrefs,
disables the "save-update" cascade when the event was
initiated on the "reverse" side of a bidirectional
relationship. This is a cleaner behavior so that
many-to-ones can be set on a transient object without
it getting sucked into the child object's session,
while still allowing the forward collection to
cascade. We *might* default this to False in 0.7.
- Slight improvement to the behavior of
"passive_updates=False" when placed only on the
many-to-one side of a relationship; documentation has
been clarified that passive_updates=False should really
be on the one-to-many side.
- Placing passive_deletes=True on a many-to-one emits
a warning, since you probably intended to put it on
the one-to-many side.
- Fixed bug that would prevent "subqueryload" from
working correctly with single table inheritance
for a relationship from a subclass - the "where
type in (x, y, z)" only gets placed on the inside,
instead of repeatedly.
- When using from_self() with single table inheritance,
the "where type in (x, y, z)" is placed on the outside
of the query only, instead of repeatedly. May make
some more adjustments to this.
- scoped_session emits a warning when configure() is
called if a Session is already present (checks only the
current thread) [ticket:1924]
- reworked the internals of mapper.cascade_iterator() to
cut down method calls by about 9% in some circumstances.
- Fixed bug in TypeDecorator whereby the dialect-specific
type was getting pulled in to generate the DDL for a
given type, which didn't always return the correct result.
- TypeDecorator can now have a fully constructed type
specified as its "impl", in addition to a type class.
- TypeDecorator will now place itself as the resulting
type for a binary expression where the type coercion
rules would normally return its impl type - previously,
a copy of the impl type would be returned which would
have the TypeDecorator embedded into it as the "dialect"
impl, this was probably an unintentional way of achieving
the desired effect.
- TypeDecorator.load_dialect_impl() returns "self.impl" by
default, i.e. not the dialect implementation type of
"self.impl". This to support compilation correctly.
Behavior can be user-overridden in exactly the same way
as before to the same effect.
- Added type_coerce(expr, type_) expression element.
Treats the given expression as the given type when evaluating
expressions and processing result rows, but does not
affect the generation of SQL, other than an anonymous
- Table.tometadata() now copies Index objects associated
with the Table as well.
- Table.tometadata() issues a warning if the given Table
is already present in the target MetaData - the existing
Table object is returned.
- An informative error message is raised if a Column
which has not yet been assigned a name, i.e. as in
declarative, is used in a context where it is
exported to the columns collection of an enclosing
select() construct, or if any construct involving
that column is compiled before its name is
- as_scalar(), label() can be called on a selectable
which contains a Column that is not yet named.
- Fixed recursion overflow which could occur when operating
with two expressions both of type "NullType", but
not the singleton NULLTYPE instance. [ticket:1907]
- @classproperty (soon/now @declared_attr) takes effect for
__mapper_args__, __table_args__, __tablename__ on
a base class that is not a mixin, as well as mixins.
- @classproperty 's official name/location for usage
with declarative is sqlalchemy.ext.declarative.declared_attr.
Same thing, but moving there since it is more of a
"marker" that's specific to declararative,
not just an attribute technique. [ticket:1915]
- Fixed bug whereby columns on a mixin wouldn't propagate
correctly to a single-table, or joined-table,
inheritance scheme where the attribute name is
different than that of the column. [ticket:1930],
- A mixin can now specify a column that overrides
a column of the same name associated with a superclass.
Thanks to Oystein Haaland.
- Fixed a regression in 0.6.4 whereby the change that
allowed cursor errors to be raised consistently broke
the result.lastrowid accessor. Test coverage has
been added for result.lastrowid. Note that lastrowid
is only supported by Pysqlite and some MySQL drivers,
so isn't super-useful in the general case.
- the logging message emitted by the engine when
a connection is first used is now "BEGIN (implicit)"
to emphasize that DBAPI has no explicit begin().
- added "views=True" option to metadata.reflect(),
will add the list of available views to those
being reflected. [ticket:1936]
- engine_from_config() now accepts 'debug' for
'echo', 'echo_pool', 'force' for 'convert_unicode',
boolean values for 'use_native_unicode'.
- Added "as_tuple" flag to ARRAY type, returns results
as tuples instead of lists to allow hashing.
- Fixed bug which prevented "domain" built from a
custom type such as "enum" from being reflected.
- Fixed bug involving reflection of CURRENT_TIMESTAMP
default used with ON UPDATE clause, thanks to
Taavi Burns [ticket:1940]
- The implicit_retunring argument to create_engine()
is now honored regardless of detected version of
Oracle. Previously, the flag would be forced
to False if server version info was < 10.
- Fixed reflection bug which did not properly handle
reflection of unknown types. [ticket:1946]
- Fixed bug where aliasing of tables with "schema" would
fail to compile properly. [ticket:1943]
- Rewrote the reflection of indexes to use sys.
catalogs, so that column names of any configuration
(spaces, embedded commas, etc.) can be reflected.
Note that reflection of indexes requires SQL
Server 2005 or greater. [ticket:1770]
- mssql+pymssql dialect now honors the "port" portion
of the URL instead of discarding it. [ticket:1952]
- *Major* cleanup / modernization of the Informix
dialect for 0.6, courtesy Florian Apolloner.
- the NoseSQLAlchemyPlugin has been moved to a
new package "sqlalchemy_nose" which installs
along with "sqlalchemy". This so that the "nosetests"
script works as always but also allows the
--with-coverage option to turn on coverage before
SQLAlchemy modules are imported, allowing coverage
to work correctly.
- CircularDependencyError now has .cycles and .edges
members, which are the set of elements involved in
one or more cycles, and the set of edges as 2-tuples.
- The name ConcurrentModificationError has been
changed to StaleDataError, and descriptive
error messages have been revised to reflect
exactly what the issue is. Both names will
remain available for the forseeable future
for schemes that may be specifying
ConcurrentModificationError in an "except:"
- Added a mutex to the identity map which mutexes
remove operations against iteration methods,
which now pre-buffer before returning an
iterable. This because asyncrhonous gc
can remove items via the gc thread at any time.
- The Session class is now present in sqlalchemy.orm.*.
We're moving away from the usage of create_session(),
which has non-standard defaults, for those situations
where a one-step Session constructor is desired. Most
users should stick with sessionmaker() for general use,
- query.with_parent() now accepts transient objects
and will use the non-persistent values of their pk/fk
attributes in order to formulate the criterion.
Docs are also clarified as to the purpose of with_parent().
- The include_properties and exclude_properties arguments
to mapper() now accept Column objects as members in
addition to strings. This so that same-named Column
objects, such as those within a join(), can be
- A warning is now emitted if a mapper is created against a
join or other single selectable that includes multiple
columns with the same name in its .c. collection,
and those columns aren't explictly named as part of
the same or separate attributes (or excluded).
In 0.7 this warning will be an exception. Note that
this warning is not emitted when the combination occurs
as a result of inheritance, so that attributes
still allow being overridden naturally.
[ticket:1896]. In 0.7 this will be improved further.
- The primary_key argument to mapper() can now specify
a series of columns that are only a subset of
the calculated "primary key" columns of the mapped
selectable, without an error being raised. This
helps for situations where a selectable's effective
primary key is simpler than the number of columns
in the selectable that are actually marked as
"primary_key", such as a join against two
tables on their primary key columns [ticket:1896].
- An object that's been deleted now gets a flag
'deleted', which prohibits the object from
being re-add()ed to the session, as previously
the object would live in the identity map
silently until its attributes were accessed.
The make_transient() function now resets this
flag along with the "key" flag.
- make_transient() can be safely called on an
already transient instance.
- a warning is emitted in mapper() if the polymorphic_on
column is not present either in direct or derived
form in the mapped selectable or in the
with_polymorphic selectable, instead of silently
ignoring it. Look for this to become an
exception in 0.7.
- Another pass through the series of error messages
emitted when relationship() is configured with
ambiguous arguments. The "foreign_keys"
setting is no longer mentioned, as it is almost
never needed and it is preferable users set up
correct ForeignKey metadata, which is now the
recommendation. If 'foreign_keys'
is used and is incorrect, the message suggests
the attribute is probably unnecessary. Docs
for the attribute are beefed up. This
because all confused relationship() users on the
ML appear to be attempting to use foreign_keys
due to the message, which only confuses them
further since Table metadata is much clearer.
- If the "secondary" table has no ForeignKey metadata
and no foreign_keys is set, even though the
user is passing screwed up information, it is assumed
that primary/secondaryjoin expressions should
consider only and all cols in "secondary" to be
foreign. It's not possible with "secondary" for
the foreign keys to be elsewhere in any case.
A warning is now emitted instead of an error,
and the mapping succeeds. [ticket:1877]
- Moving an o2m object from one collection to
another, or vice versa changing the referenced
object by an m2o, where the foreign key is also a
member of the primary key, will now be more
carefully checked during flush if the change in
value of the foreign key on the "many" side is the
result of a change in the primary key of the "one"
side, or if the "one" is just a different object.
In one case, a cascade-capable DB would have
cascaded the value already and we need to look at
the "new" PK value to do an UPDATE, in the other we
need to continue looking at the "old". We now look
at the "old", assuming passive_updates=True,
unless we know it was a PK switch that
triggered the change. [ticket:1856]
- The value of version_id_col can be changed
manually, and this will result in an UPDATE
of the row. Versioned UPDATEs and DELETEs
now use the "committed" value of the
version_id_col in the WHERE clause and
not the pending changed value. The
version generator is also bypassed if
manual changes are present on the attribute.
- Repaired the usage of merge() when used with
concrete inheriting mappers. Such mappers frequently
have so-called "concrete" attributes, which are
subclass attributes that "disable" propagation from
the parent - these needed to allow a merge()
operation to pass through without effect.
- Specifying a non-column based argument
for column_mapped_collection, including string,
text() etc., will raise an error message that
specifically asks for a column element, no longer
misleads with incorrect information about
text() or literal(). [ticket:1863]
- Similarly, for relationship(), foreign_keys,
remote_side, order_by - all column-based
expressions are enforced - lists of strings
are explicitly disallowed since this is a
very common error
- Dynamic attributes don't support collection
population - added an assertion for when
set_committed_value() is called, as well as
when joinedload() or subqueryload() options
are applied to a dynamic attribute, instead
of failure / silent failure. [ticket:1864]
- Fixed bug whereby generating a Query derived
from one which had the same column repeated
with different label names, typically
in some UNION situations, would fail to
propagate the inner columns completely to
the outer query. [ticket:1852]
- object_session() raises the proper
UnmappedInstanceError when presented with an
unmapped instance. [ticket:1881]
- Applied further memoizations to calculated Mapper
properties, with significant (~90%) runtime mapper.py
call count reduction in heavily polymorphic mapping
- mapper _get_col_to_prop private method used
by the versioning example is deprecated;
now use mapper.get_property_by_column() which
will remain the public method for this.
- the versioning example works correctly now
if versioning on a col that was formerly
- Calling execute() on an alias() construct is pending
deprecation for 0.7, as it is not itself an
"executable" construct. It currently "proxies" its
inner element and is conditionally "executable" but
this is not the kind of ambiguity we like these days.
- The execute() and scalar() methods of ClauseElement
are now moved appropriately to the Executable
subclass. ClauseElement.execute()/ scalar() are still
present and are pending deprecation in 0.7, but note
these would always raise an error anyway if you were
not an Executable (unless you were an alias(), see
- Added basic math expression coercion for
so that resulting type is Numeric regardless
of the direction of the expression.
- Changed the scheme used to generate truncated
"auto" index names when using the "index=True"
flag on Column. The truncation only takes
place with the auto-generated name, not one
that is user-defined (an error would be
raised instead), and the truncation scheme
itself is now based on a fragment of an md5
hash of the identifier name, so that multiple
indexes on columns with similar names still
have unique names. [ticket:1855]
- The generated index name also is based on
a "max index name length" attribute which is
separate from the "max identifier length" -
this to appease MySQL who has a max length
of 64 for index names, separate from their
overall max length of 255. [ticket:1412]
- the text() construct, if placed in a column
oriented situation, will at least return NULLTYPE
for its type instead of None, allowing it to
be used a little more freely for ad-hoc column
expressions than before. literal_column()
is still the better choice, however.
- Added full description of parent table/column,
target table/column in error message raised when
ForeignKey can't resolve target.
- Fixed bug whereby replacing composite foreign key
columns in a reflected table would cause an attempt
to remove the reflected constraint from the table
a second time, raising a KeyError. [ticket:1865]
- the _Label construct, i.e. the one that is produced
whenever you say somecol.label(), now counts itself
in its "proxy_set" unioned with that of it's
contained column's proxy set, instead of
directly returning that of the contained column.
This allows column correspondence
operations which depend on the identity of the
_Labels themselves to return the correct result
- fixes ORM bug [ticket:1852].
- Calling fetchone() or similar on a result that
has already been exhausted, has been closed,
or is not a result-returning result now
raises ResourceClosedError, a subclass of
InvalidRequestError, in all cases, regardless
of backend. Previously, some DBAPIs would
raise ProgrammingError (i.e. pysqlite), others
would return None leading to downstream breakages
- Fixed bug in Connection whereby if a "disconnect"
event occurred in the "initialize" phase of the
first connection pool connect, an AttributeError
would be raised when the Connection would attempt
to invalidate the DBAPI connection. [ticket:1894]
- Connection, ResultProxy, as well as Session use
ResourceClosedError for all "this
connection/transaction/result is closed" types of
- Connection.invalidate() can be called more than
once and subsequent calls do nothing.
- if @classproperty is used with a regular class-bound
mapper property attribute, it will be called to get the
actual attribute value during initialization. Currently,
there's no advantage to using @classproperty on a column
or relationship attribute of a declarative class that
isn't a mixin - evaluation is at the same time as if
@classproperty weren't used. But here we at least allow
it to function as expected.
- Fixed bug where "Can't add additional column" message
would display the wrong name.
- Fixed the psycopg2 dialect to use its
set_isolation_level() method instead of relying
upon the base "SET SESSION ISOLATION" command,
as psycopg2 resets the isolation level on each new
- Fixed "default schema" query to work with
- Fixed bug whereby a column default would fail to
reflect if the "default" keyword were lower case.
- Added ROWID type to the Oracle dialect, for those
cases where an explicit CAST might be needed.
- Oracle reflection of indexes has been tuned so
that indexes which include some or all primary
key columns, but not the same set of columns
as that of the primary key, are reflected.
Indexes which contain the identical columns
as that of the primary key are skipped within
reflection, as the index in that case is assumed
to be the auto-generated primary key index.
Previously, any index with PK columns present
would be skipped. Thanks to Kent Bower
for the patch. [ticket:1867]
- Oracle now reflects the names of primary key
constraints - also thanks to Kent Bower.
- Applied patches from [ticket:1904] to get
basic Informix functionality up again. We
rely upon end-user testing to ensure that
Informix is working to some degree.
- The docs have been reorganized such that the "API
Reference" section is gone - all the docstrings from
there which were public API are moved into the
context of the main doc section that talks about it.
Main docs divided into "SQLAlchemy Core" and
"SQLAlchemy ORM" sections, mapper/relationship docs
have been broken out. Lots of sections rewritten
- The beaker_caching example has been reorgnized
such that the Session, cache manager,
declarative_base are part of environment, and
custom cache code is portable and now within
"caching_query.py". This allows the example to
be easier to "drop in" to existing projects.
- the history_meta versioning recipe sets "unique=False"
when copying columns, so that the versioning
table handles multiple rows with repeating values.
- Removed errant many-to-many load in unitofwork
which triggered unnecessarily on expired/unloaded
collections. This load now takes place only if
passive_updates is False and the parent primary
key has changed, or if passive_deletes is False
and a delete of the parent has occurred.
- Column-entities (i.e. query(Foo.id)) copy their
state more fully when queries are derived from
themselves + a selectable (i.e. from_self(),
union(), etc.), so that join() and such have the
correct state to work from. [ticket:1853]
- Fixed bug where Query.join() would fail if
querying a non-ORM column then joining without
an on clause when a FROM clause is already
present, now raises a checked exception the
same way it does when the clause is not
- Improved the check for an "unmapped class",
including the case where the superclass is mapped
but the subclass is not. Any attempts to access
cls._sa_class_manager.mapper now raise
- Added "column_descriptions" accessor to Query,
returns a list of dictionaries containing
naming/typing information about the entities
the Query will return. Can be helpful for
building GUIs on top of ORM queries.
- The _extract_error_code() method now works
correctly with each MySQL dialect (
MySQL-python, OurSQL, MySQL-Connector-Python,
the reconnect logic would fail for OperationalError
conditions, however since MySQLdb and OurSQL
have their own reconnect feature, there was no
symptom for these drivers here unless one
watched the logs. [ticket:1848]
- More tweaks to cx_oracle Decimal handling.
"Ambiguous" numerics with no decimal place
are coerced to int at the connection handler
level. The advantage here is that ints
come back as ints without SQLA type
objects being involved and without needless
conversion to Decimal first.
Unfortunately, some exotic subquery cases
can even see different types between
individual result rows, so the Numeric
handler, when instructed to return Decimal,
can't take full advantage of "native decimal"
mode and must run isinstance() on every value
to check if its Decimal already. Reopen of
- Query.join() will check for a call of the
form query.join(target, clause_expression),
i.e. missing the tuple, and raise an informative
error message that this is the wrong calling form.
- Fixed bug regarding flushes on self-referential
bi-directional many-to-many relationships, where
two objects made to mutually reference each other
in one flush would fail to insert a row for both
sides. Regression from 0.5. [ticket:1824]
- the post_update feature of relationship() has been
reworked architecturally to integrate more closely
with the new 0.6 unit of work. The motivation
for the change is so that multiple "post update"
calls, each affecting different foreign key
columns of the same row, are executed in a single
UPDATE statement, rather than one UPDATE
statement per column per row. Multiple row
updates are also batched into executemany()s as
possible, while maintaining consistent row ordering.
- Query.statement, Query.subquery(), etc. now transfer
the values of bind parameters, i.e. those specified
by query.params(), into the resulting SQL expression.
Previously the values would not be transferred
and bind parameters would come out as None.
- Subquery-eager-loading now works with Query objects
which include params(), as well as get() Queries.
- Can now call make_transient() on an instance that
is referenced by parent objects via many-to-one,
without the parent's foreign key value getting
temporarily set to None - this was a function
of the "detect primary key switch" flush handler.
It now ignores objects that are no longer
in the "persistent" state, and the parent's
foreign key identifier is left unaffected.
- query.order_by() now accepts False, which cancels
any existing order_by() state on the Query, allowing
subsequent generative methods to be called which do
not support ORDER BY. This is not the same as the
already existing feature of passing None, which
suppresses any existing order_by() settings, including
those configured on the mapper. False will make it
as though order_by() was never called, while
None is an active setting.
- An instance which is moved to "transient", has
an incomplete or missing set of primary key
attributes, and contains expired attributes, will
raise an InvalidRequestError if an expired attribute
is accessed, instead of getting a recursion overflow.
- The make_transient() function is now in the generated
- make_transient() removes all "loader" callables from
the state being made transient, removing any
"expired" state - all unloaded attributes reset back
to undefined, None/empty on access.
- The warning emitted by the Unicode and String types
with convert_unicode=True no longer embeds the actual
value passed. This so that the Python warning
registry does not continue to grow in size, the warning
is emitted once as per the warning filter settings,
and large string values don't pollute the output.
- Fixed bug that would prevent overridden clause
compilation from working for "annotated" expression
elements, which are often generated by the ORM.
- The argument to "ESCAPE" of a LIKE operator or similar
is passed through render_literal_value(), which may
implement escaping of backslashes. [ticket:1400]
- Fixed bug in Enum type which blew away native_enum
flag when used with TypeDecorators or other adaption
- Inspector hits bind.connect() when invoked to ensure
initialize has been called. the internal name ".conn"
is changed to ".bind", since that's what it is.
- Modified the internals of "column annotation" such that
a custom Column subclass can safely override
_constructor to return Column, for the purposes of
making "configurational" column classes that aren't
involved in proxying, etc.
- Column.copy() takes along the "unique" attribute
among others, fixes [ticket:1829] regarding declarative
- render_literal_value() is overridden which escapes
backslashes, currently applies to the ESCAPE clause
of LIKE and similar expressions.
Ultimately this will have to detect the value of
"standard_conforming_strings" for full behavior.
- Won't generate "CREATE TYPE" / "DROP TYPE" if
using types.Enum on a PG version prior to 8.3 -
the supports_native_enum flag is fully
- MySQL dialect doesn't emit CAST() for MySQL version
detected < 4.0.2. This allows the unicode
check on connect to proceed. [ticket:1826]
- MySQL dialect now detects NO_BACKSLASH_ESCAPES sql
mode, in addition to ANSI_QUOTES.
- render_literal_value() is overridden which escapes
backslashes, currently applies to the ESCAPE clause
of LIKE and similar expressions. This behavior
is derived from detecting the value of
- Fixed ora-8 compatibility flags such that they
don't cache a stale value from before the first
database connection actually occurs. [ticket:1819]
- Oracle's "native decimal" metadata begins to return
ambiguous typing information about numerics
when columns are embedded in subqueries as well
as when ROWNUM is consulted with subqueries, as we
do for limit/offset. We've added these ambiguous
conditions to the cx_oracle "convert to Decimal()"
handler, so that we receive numerics as Decimal
in more cases instead of as floats. These are
then converted, if requested, into Integer
or Float, or otherwise kept as the lossless
- If server_version_info is outside the usual
range of (8, ), (9, ), (10, ), a warning is emitted
which suggests checking that the FreeTDS version
configuration is using 7.0 or 8.0, not 4.2.
- Fixed incorrect signature in do_execute(), error
introduced in 0.6.1. [ticket:1823]
- Firebird dialect adds CHAR, VARCHAR types which
accept a "charset" flag, to support Firebird
"CHARACTER SET" clause. [ticket:1813]
- Added support for @classproperty to provide
any kind of schema/mapping construct from a
declarative mixin, including columns with foreign
keys, relationships, column_property, deferred.
This solves all such issues on declarative mixins.
An error is raised if any MapperProperty subclass
is specified on a mixin without using @classproperty.
[ticket:1751] [ticket:1796] [ticket:1805]
- a mixin class can now define a column that matches
one which is present on a __table__ defined on a
subclass. It cannot, however, define one that is
not present in the __table__, and the error message
here now works. [ticket:1821]
- compiler extension
- The 'default' compiler is automatically copied over
when overriding the compilation of a built in
clause construct, so no KeyError is raised if the
user-defined compiler is specific to certain
backends and compilation for a different backend
is invoked. [ticket:1838]
- Added documentation for the Inspector. [ticket:1820]
- Fixed @memoized_property and @memoized_instancemethod
decorators so that Sphinx documentation picks up
these attributes and methods, such as
- Fixed regression introduced in 0.6.0 involving improper
history accounting on mutable attributes. [ticket:1782]
- Fixed regression introduced in 0.6.0 unit of work refactor
that broke updates for bi-directional relationship()
with post_update=True. [ticket:1807]
- session.merge() will not expire attributes on the returned
instance if that instance is "pending". [ticket:1789]
- fixed __setstate__ method of CollectionAdapter to not
fail during deserialize where parent InstanceState not
yet unserialized. [ticket:1802]
- Added internal warning in case an instance without a
full PK happened to be expired and then was asked
to refresh. [ticket:1797]
- Added more aggressive caching to the mapper's usage of
UPDATE, INSERT, and DELETE expressions. Assuming the
statement has no per-object SQL expressions attached,
the expression objects are cached by the mapper after
the first create, and their compiled form is stored
persistently in a cache dictionary for the duration of
the related Engine. The cache is an LRUCache for the
rare case that a mapper receives an extremely
high number of different column patterns as UPDATEs.
- expr.in_() now accepts a text() construct as the argument.
Grouping parenthesis are added automatically, i.e. usage
is like `col.in_(text("select id from table"))`.
- Columns of _Binary type (i.e. LargeBinary, BLOB, etc.)
will coerce a "basestring" on the right side into a
_Binary as well so that required DBAPI processing
- Added table.add_is_dependent_on(othertable), allows manual
placement of dependency rules between two Table objects
for use within create_all(), drop_all(), sorted_tables.
- Fixed bug that prevented implicit RETURNING from functioning
properly with composite primary key that contained zeroes.
- Fixed errant space character when generating ADD CONSTRAINT
for a named UNIQUE constraint.
- Fixed "table" argument on constructor of ForeginKeyConstraint
- Fixed bug in connection pool cursor wrapper whereby if a
cursor threw an exception on close(), the logging of the
message would fail. [ticket:1786]
- the _make_proxy() method of ColumnClause and Column now use
self.__class__ to determine the class of object to be returned
instead of hardcoding to ColumnClause/Column, making it slightly
easier to produce specific subclasses of these which work in
- func.XXX() doesn't inadvertently resolve to non-Function
classes (e.g. fixes func.text()). [ticket:1798]
- Fixed building the C extensions on Python 2.4. [ticket:1781]
- Pool classes will reuse the same "pool_logging_name" setting
after a dispose() occurs.
- Engine gains an "execution_options" argument and
update_execution_options() method, which will apply to
all connections generated by this engine.
- func.sysdate() emits "SYSDATE()", i.e. with the ending
parenthesis, on MySQL. [ticket:1794]
- Fixed concatenation of constraints when "PRIMARY KEY"
constraint gets moved to column level due to SQLite
AUTOINCREMENT keyword being rendered. [ticket:1812]
- Added a check for cx_oracle versions lower than version 5,
in which case the incompatible "output type handler" won't
be used. This will impact decimal accuracy and some
unicode handling issues. [ticket:1775]
- Fixed use_ansi=False mode, which was producing broken
WHERE clauses in pretty much all cases. [ticket:1790]
- Re-established support for Oracle 8 with cx_oracle,
including that use_ansi is set to False automatically,
NVARCHAR2 and NCLOB are not rendered for Unicode,
"native unicode" check doesn't fail, cx_oracle
"native unicode" mode is disabled, VARCHAR() is emitted
with bytes count instead of char count. [ticket:1808]
- oracle_xe 5 doesn't accept a Python unicode object in
its connect string in normal Python 2.x mode - so we coerce
to str() directly. non-ascii characters aren't supported
in connect strings here since we don't know what encoding
we could use. [ticket:1670]
- FOR UPDATE is emitted in the syntactically correct position
when limit/offset is used, i.e. the ROWNUM subquery.
However, Oracle can't really handle FOR UPDATE with ORDER BY
or with subqueries, so its still not very usable, but at
least SQLA gets the SQL past the Oracle parser.
- Added a label to the query used within has_table() and
has_sequence() to work with older versions of Firebird
that don't provide labels for result columns. [ticket:1521]
- Added integer coercion to the "type_conv" attribute when
passed via query string, so that it is properly interpreted
by Kinterbasdb. [ticket:1779]
- Added 'connection shutdown' to the list of exception strings
which indicate a dropped connection. [ticket:1646]
- the SqlSoup constructor accepts a `base` argument which specifies
the base class to use for mapped classes, the default being
- Unit of work internals have been rewritten. Units of work
with large numbers of objects interdependent objects
can now be flushed without recursion overflows
as there is no longer reliance upon recursive calls
[ticket:1081]. The number of internal structures now stays
constant for a particular session state, regardless of
how many relationships are present on mappings. The flow
of events now corresponds to a linear list of steps,
generated by the mappers and relationships based on actual
work to be done, filtered through a single topological sort
for correct ordering. Flush actions are assembled using
far fewer steps and less memory. [ticket:1742]
- Along with the UOW rewrite, this also removes an issue
introduced in 0.6beta3 regarding topological cycle detection
for units of work with long dependency cycles. We now use
an algorithm written by Guido (thanks Guido!).
- one-to-many relationships now maintain a list of positive
parent-child associations within the flush, preventing
previous parents marked as deleted from cascading a
delete or NULL foreign key set on those child objects,
despite the end-user not removing the child from the old
- A collection lazy load will switch off default
eagerloading on the reverse many-to-one side, since
that loading is by definition unnecessary. [ticket:1495]
- Session.refresh() now does an equivalent expire()
on the given instance first, so that the "refresh-expire"
cascade is propagated. Previously, refresh() was
not affected in any way by the presence of "refresh-expire"
cascade. This is a change in behavior versus that
of 0.6beta2, where the "lockmode" flag passed to refresh()
would cause a version check to occur. Since the instance
is first expired, refresh() always upgrades the object
to the most recent version.
- The 'refresh-expire' cascade, when reaching a pending object,
will expunge the object if the cascade also includes
"delete-orphan", or will simply detach it otherwise.
- id(obj) is no longer used internally within topological.py,
as the sorting functions now require hashable objects
- The ORM will set the docstring of all generated descriptors
to None by default. This can be overridden using 'doc'
(or if using Sphinx, attribute docstrings work too).
- Added kw argument 'doc' to all mapper property callables
as well as Column(). Will assemble the string 'doc' as
the '__doc__' attribute on the descriptor.
- Usage of version_id_col on a backend that supports
cursor.rowcount for execute() but not executemany() now works
when a delete is issued (already worked for saves, since those
don't use executemany()). For a backend that doesn't support
cursor.rowcount at all, a warning is emitted the same
as with saves. [ticket:1761]
- The ORM now short-term caches the "compiled" form of
insert() and update() constructs when flushing lists of
objects of all the same class, thereby avoiding redundant
compilation per individual INSERT/UPDATE within an
individual flush() call.
- internal getattr(), setattr(), getcommitted() methods
on ColumnProperty, CompositeProperty, RelationshipProperty
have been underscored (i.e. are private), signature has
- The C extension now also works with DBAPIs which use custom
sequences as row (and not only tuples). [ticket:1757]
- Restored some bind-labeling logic from 0.5 which ensures
that tables with column names that overlap another column
of the form "<tablename>_<columnname>" won't produce
errors if column._label is used as a bind name during
an UPDATE. Test coverage which wasn't present in 0.5
has been added. [ticket:1755]
- somejoin.select(fold_equivalents=True) is no longer
deprecated, and will eventually be rolled into a more
comprehensive version of the feature for [ticket:1729].
- the Numeric type raises an *enormous* warning when expected
to convert floats to Decimal from a DBAPI that returns floats.
This includes SQLite, Sybase, MS-SQL. [ticket:1759]
- Fixed an error in expression typing which caused an endless
loop for expressions with two NULL types.
- Fixed bug in execution_options() feature whereby the existing
Transaction and other state information from the parent
connection would not be propagated to the sub-connection.
- Added new 'compiled_cache' execution option. A dictionary
where Compiled objects will be cached when the Connection
compiles a clause expression into a dialect- and parameter-
specific Compiled object. It is the user's responsibility to
manage the size of this dictionary, which will have keys
corresponding to the dialect, clause element, the column
names within the VALUES or SET clause of an INSERT or UPDATE,
as well as the "batch" mode for an INSERT or UPDATE statement.
- Added get_pk_constraint() to reflection.Inspector, similar
to get_primary_keys() except returns a dict that includes the
name of the constraint, for supported backends (PG so far).
- Table.create() and Table.drop() no longer apply metadata-
level create/drop events. [ticket:1771]
- the compiler extension now allows @compiles decorators
on base classes that extend to child classes, @compiles
decorators on child classes that aren't broken by a
@compiles decorator on the base class.
- Declarative will raise an informative error message
if a non-mapped class attribute is referenced in the
string-based relationship() arguments.
- Further reworked the "mixin" logic in declarative to
additionally allow __mapper_args__ as a @classproperty
on a mixin, such as to dynamically assign polymorphic_identity.
- Postgresql now reflects sequence names associated with
SERIAL columns correctly, after the name of of the sequence
has been changed. Thanks to Kumar McMillan for the patch.
- Repaired missing import in psycopg2._PGNumeric type when
unknown numeric is received.
- psycopg2/pg8000 dialects now aware of REAL, FLOAT,
DOUBLE_PRECISION, NUMERIC return types without
raising an exception.
- Postgresql reflects the name of primary key constraints,
if one exists. [ticket:1769]
- Now using cx_oracle output converters so that the
DBAPI returns natively the kinds of values we prefer:
- NUMBER values with positive precision + scale convert
to cx_oracle.STRING and then to Decimal. This
allows perfect precision for the Numeric type when
using cx_oracle. [ticket:1759]
- STRING/FIXED_CHAR now convert to unicode natively.
SQLAlchemy's String types then don't need to
apply any kind of conversions.
- The functionality of result.rowcount can be disabled on a
per-engine basis by setting 'enable_rowcount=False'
on create_engine(). Normally, cursor.rowcount is called
after any UPDATE or DELETE statement unconditionally,
because the cursor is then closed and Firebird requires
an open cursor in order to get a rowcount. This
call is slightly expensive however so it can be disabled.
To re-enable on a per-execution basis, the
'enable_rowcount=True' execution option may be used.
- Updated attribute_shard.py example to use a more robust
method of searching a Query for binary expressions which
compare columns against literal values.
- Major feature: Added new "subquery" loading capability to
relationship(). This is an eager loading option which
generates a second SELECT for each collection represented
in a query, across all parents at once. The query
re-issues the original end-user query wrapped in a subquery,
applies joins out to the target collection, and loads
all those collections fully in one result, similar to
"joined" eager loading but using all inner joins and not
re-fetching full parent rows repeatedly (as most DBAPIs seem
to do, even if columns are skipped). Subquery loading is
available at mapper config level using "lazy='subquery'" and
at the query options level using "subqueryload(props..)",
- To accomodate the fact that there are now two kinds of eager
loading available, the new names for eagerload() and
eagerload_all() are joinedload() and joinedload_all(). The
old names will remain as synonyms for the foreseeable future.
- The "lazy" flag on the relationship() function now accepts
a string argument for all kinds of loading: "select", "joined",
"subquery", "noload" and "dynamic", where the default is now
"select". The old values of True/
False/None still retain their usual meanings and will remain
as synonyms for the foreseeable future.
- Added with_hint() method to Query() construct. This calls
directly down to select().with_hint() and also accepts
entities as well as tables and aliases. See with_hint() in the
SQL section below. [ticket:921]
- Fixed bug in Query whereby calling q.join(prop).from_self(...).
join(prop) would fail to render the second join outside the
subquery, when joining on the same criterion as was on the
- Fixed bug in Query whereby the usage of aliased() constructs
would fail if the underlying table (but not the actual alias)
were referenced inside the subquery generated by
q.from_self() or q.select_from().
- Fixed bug which affected all eagerload() and similar options
such that "remote" eager loads, i.e. eagerloads off of a lazy
load such as query(A).options(eagerload(A.b, B.c))
wouldn't eagerload anything, but using eagerload("b.c") would
- Query gains an add_columns(*columns) method which is a multi-
version of add_column(col). add_column(col) is future
- Query.join() will detect if the end result will be
"FROM A JOIN A", and will raise an error if so.
- Query.join(Cls.propname, from_joinpoint=True) will check more
carefully that "Cls" is compatible with the current joinpoint,
and act the same way as Query.join("propname", from_joinpoint=True)
in that regard.
- Added with_hint() method to select() construct. Specify
a table/alias, hint text, and optional dialect name, and
"hints" will be rendered in the appropriate place in the
statement. Works for Oracle, Sybase, MySQL. [ticket:921]
- Fixed bug introduced in 0.6beta2 where column labels would
render inside of column expressions already assigned a label.
- The psycopg2 dialect will log NOTICE messages via the
"sqlalchemy.dialects.postgresql" logger name.
- the TIME and TIMESTAMP types are now availble from the
postgresql dialect directly, which add the PG-specific
argument 'precision' to both. 'precision' and
'timezone' are correctly reflected for both TIME and
TIMEZONE types. [ticket:997]
- No longer guessing that TINYINT(1) should be BOOLEAN
when reflecting - TINYINT(1) is returned. Use Boolean/
BOOLEAN in table definition to get boolean conversion
- The Oracle dialect will issue VARCHAR type definitions
using character counts, i.e. VARCHAR2(50 CHAR), so that
the column is sized in terms of characters and not bytes.
Column reflection of character types will also use
ALL_TAB_COLUMNS.CHAR_LENGTH instead of
ALL_TAB_COLUMNS.DATA_LENGTH. Both of these behaviors take
effect when the server version is 9 or higher - for
version 8, the old behaviors are used. [ticket:1744]
- Using a mixin won't break if the mixin implements an
unpredictable __getattribute__(), i.e. Zope interfaces.
- Using @classdecorator and similar on mixins to define
__tablename__, __table_args__, etc. now works if
the method references attributes on the ultimate
- relationships and columns with foreign keys aren't
allowed on declarative mixins, sorry. [ticket:1751]
- The sqlalchemy.orm.shard module now becomes an extension,
sqlalchemy.ext.horizontal_shard. The old import
works with a deprecation warning.
- Improved the installation/test setup regarding Python 3,
now that Distribute runs on Py3k. distribute_setup.py
is now included. See README.py3k for Python 3 installation/
- The official name for the relation() function is now
relationship(), to eliminate confusion over the relational
algebra term. relation() however will remain available
in equal capacity for the foreseeable future. [ticket:1740]
- Added "version_id_generator" argument to Mapper, this is a
callable that, given the current value of the "version_id_col",
returns the next version number. Can be used for alternate
versioning schemes such as uuid, timestamps. [ticket:1692]
- added "lockmode" kw argument to Session.refresh(), will
pass through the string value to Query the same as
in with_lockmode(), will also do version check for a
- Fixed bug whereby calling query(A).join(A.bs).add_entity(B)
in a joined inheritance scenario would double-add B as a
target and produce an invalid query. [ticket:1188]
- Fixed bug in session.rollback() which involved not removing
formerly "pending" objects from the session before
re-integrating "deleted" objects, typically occured with
natural primary keys. If there was a primary key conflict
between them, the attach of the deleted would fail
internally. The formerly "pending" objects are now expunged
- Removed a lot of logging that nobody really cares about,
logging that remains will respond to live changes in the
log level. No significant overhead is added. [ticket:1719]
- Fixed bug in session.merge() which prevented dict-like
collections from merging.
- session.merge() works with relations that specifically
don't include "merge" in their cascade options - the target
is ignored completely.
- session.merge() will not expire existing scalar attributes
on an existing target if the target has a value for that
attribute, even if the incoming merged doesn't have
a value for the attribute. This prevents unnecessary loads
on existing items. Will still mark the attr as expired
if the destination doesn't have the attr, though, which
fulfills some contracts of deferred cols. [ticket:1681]
- The "allow_null_pks" flag is now called "allow_partial_pks",
defaults to True, acts like it did in 0.5 again. Except,
it also is implemented within merge() such that a SELECT
won't be issued for an incoming instance with partially
NULL primary key if the flag is False. [ticket:1680]
- Fixed bug in 0.6-reworked "many-to-one" optimizations
such that a many-to-one that is against a non-primary key
column on the remote table (i.e. foreign key against a
UNIQUE column) will pull the "old" value in from the
database during a change, since if it's in the session
we will need it for proper history/backref accounting,
and we can't pull from the local identity map on a
non-primary key column. [ticket:1737]
- fixed internal error which would occur if calling has()
or similar complex expression on a single-table inheritance
- query.one() no longer applies LIMIT to the query, this to
ensure that it fully counts all object identities present
in the result, even in the case where joins may conceal
multiple identities for two or more rows. As a bonus,
one() can now also be called with a query that issued
from_statement() to start with since it no longer modifies
the query. [ticket:1688]
- query.get() now returns None if queried for an identifier
that is present in the identity map with a different class
than the one requested, i.e. when using polymorphic loading.
- A major fix in query.join(), when the "on" clause is an
attribute of an aliased() construct, but there is already
an existing join made out to a compatible target, query properly
joins to the right aliased() construct instead of sticking
onto the right side of the existing join. [ticket:1706]
- Slight improvement to the fix for [ticket:1362] to not issue
needless updates of the primary key column during a so-called
"row switch" operation, i.e. add + delete of two objects
with the same PK.
- Now uses sqlalchemy.orm.exc.DetachedInstanceError when an
attribute load or refresh action fails due to object
being detached from any Session. UnboundExecutionError
is specific to engines bound to sessions and statements.
- Query called in the context of an expression will render
disambiguating labels in all cases. Note that this does
not apply to the existing .statement and .subquery()
accessor/method, which still honors the .with_labels()
setting that defaults to False.
- Query.union() retains disambiguating labels within the
returned statement, thus avoiding various SQL composition
errors which can result from column name conflicts.
- Fixed bug in attribute history that inadvertently invoked
__eq__ on mapped instances.
- Some internal streamlining of object loading grants a
small speedup for large results, estimates are around
10-15%. Gave the "state" internals a good solid
cleanup with less complexity, datamembers,
method calls, blank dictionary creates.
- Documentation clarification for query.delete()
- Fixed cascade bug in many-to-one relation() when attribute
was set to None, introduced in r6711 (cascade deleted
items into session during add()).
- Calling query.order_by() or query.distinct() before calling
query.select_from(), query.with_polymorphic(), or
query.from_statement() raises an exception now instead of
silently dropping those criterion. [ticket:1736]
- query.scalar() now raises an exception if more than one
row is returned. All other behavior remains the same.
- Fixed bug which caused "row switch" logic, that is an
INSERT and DELETE replaced by an UPDATE, to fail when
version_id_col was in use. [ticket:1692]
- join() will now simulate a NATURAL JOIN by default. Meaning,
if the left side is a join, it will attempt to join the right
side to the rightmost side of the left first, and not raise
any exceptions about ambiguous join conditions if successful
even if there are further join targets across the rest of
the left. [ticket:1714]
- The most common result processors conversion function were
moved to the new "processors" module. Dialect authors are
encouraged to use those functions whenever they correspond
to their needs instead of implementing custom ones.
- SchemaType and subclasses Boolean, Enum are now serializable,
including their ddl listener and other event callables.
- Some platforms will now interpret certain literal values
as non-bind parameters, rendered literally into the SQL
statement. This to support strict SQL-92 rules that are
enforced by some platforms including MS-SQL and Sybase.
In this model, bind parameters aren't allowed in the
columns clause of a SELECT, nor are certain ambiguous
expressions like "?=?". When this mode is enabled, the base
compiler will render the binds as inline literals, but only across
strings and numeric values. Other types such as dates
will raise an error, unless the dialect subclass defines
a literal rendering function for those. The bind parameter
must have an embedded literal value already or an error
is raised (i.e. won't work with straight bindparam('x')).
Dialects can also expand upon the areas where binds are not
accepted, such as within argument lists of functions
(which don't work on MS-SQL when native SQL binding is used).
- Added "unicode_errors" parameter to String, Unicode, etc.
Behaves like the 'errors' keyword argument to
the standard library's string.decode() functions. This flag
requires that `convert_unicode` is set to `"force"` - otherwise,
SQLAlchemy is not guaranteed to handle the task of unicode
conversion. Note that this flag adds significant performance
overhead to row-fetching operations for backends that already
return unicode objects natively (which most DBAPIs do). This
flag should only be used as an absolute last resort for reading
strings from a column with varied or corrupted encodings,
which only applies to databases that accept invalid encodings
in the first place (i.e. MySQL. *not* PG, Sqlite, etc.)
- Added math negation operator support, -x.
- FunctionElement subclasses are now directly executable the
same way any func.foo() construct is, with automatic
SELECT being applied when passed to execute().
- The "type" and "bind" keyword arguments of a func.foo()
construct are now local to "func." constructs and are
not part of the FunctionElement base class, allowing
a "type" to be handled in a custom constructor or
- Restored the keys() method to ResultProxy.
- The type/expression system now does a more complete job
of determining the return type from an expression
as well as the adaptation of the Python operator into
a SQL operator, based on the full left/right/operator
of the given expression. In particular
the date/time/interval system created for Postgresql
EXTRACT in [ticket:1647] has now been generalized into
the type system. The previous behavior which often
occured of an expression "column + literal" forcing
the type of "literal" to be the same as that of "column"
will now usually not occur - the type of
"literal" is first derived from the Python type of the
literal, assuming standard native Python types + date
types, before falling back to that of the known type
on the other side of the expression. If the
"fallback" type is compatible (i.e. CHAR from String),
the literal side will use that. TypeDecorator
types override this by default to coerce the "literal"
side unconditionally, which can be changed by implementing
the coerce_compared_value() method. Also part of
- Made sqlalchemy.sql.expressions.Executable part of public
API, used for any expression construct that can be sent to
execute(). FunctionElement now inherits Executable so that
it gains execution_options(), which are also propagated
to the select() that's generated within execute().
Executable in turn subclasses _Generative which marks
any ClauseElement that supports the @_generative
decorator - these may also become "public" for the benefit
of the compiler extension at some point.
- A change to the solution for [ticket:1579] - an end-user
defined bind parameter name that directly conflicts with
a column-named bind generated directly from the SET or
VALUES clause of an update/insert generates a compile error.
This reduces call counts and eliminates some cases where
undesirable name conflicts could still occur.
- Column() requires a type if it has no foreign keys (this is
not new). An error is now raised if a Column() has no type
and no foreign keys. [ticket:1705]
- the "scale" argument of the Numeric() type is honored when
coercing a returned floating point value into a string
on its way to Decimal - this allows accuracy to function
on SQLite, MySQL. [ticket:1717]
- the copy() method of Column now copies over uninitialized
"on table attach" events. Helps with the new declarative
- Added an optional C extension to speed up the sql layer by
reimplementing RowProxy and the most common result processors.
The actual speedups will depend heavily on your DBAPI and
the mix of datatypes used in your tables, and can vary from
a 30% improvement to more than 200%. It also provides a modest
(~15-20%) indirect improvement to ORM speed for large queries.
Note that it is *not* built/installed by default.
See README for installation instructions.
- the execution sequence pulls all rowcount/last inserted ID
info from the cursor before commit() is called on the
DBAPI connection in an "autocommit" scenario. This helps
mxodbc with rowcount and is probably a good idea overall.
- Opened up logging a bit such that isEnabledFor() is called
more often, so that changes to the log level for engine/pool
will be reflected on next connect. This adds a small
amount of method call overhead. It's negligible and will make
life a lot easier for all those situations when logging
just happens to be configured after create_engine() is called.
- The assert_unicode flag is deprecated. SQLAlchemy will raise
a warning in all cases where it is asked to encode a non-unicode
Python string, as well as when a Unicode or UnicodeType type
is explicitly passed a bytestring. The String type will do nothing
for DBAPIs that already accept Python unicode objects.
- Bind parameters are sent as a tuple instead of a list. Some
backend drivers will not accept bind parameters as a list.
- threadlocal engine wasn't properly closing the connection
upon close() - fixed that.
- Transaction object doesn't rollback or commit if it isn't
"active", allows more accurate nesting of begin/rollback/commit.
- Python unicode objects as binds result in the Unicode type,
not string, thus eliminating a certain class of unicode errors
on drivers that don't support unicode binds.
- Added "logging_name" argument to create_engine(), Pool() constructor
as well as "pool_logging_name" argument to create_engine() which
filters down to that of Pool. Issues the given string name
within the "name" field of logging messages instead of the default
hex identifier string. [ticket:1555]
- The visit_pool() method of Dialect is removed, and replaced with
connect(). This method returns a callable which receives
the raw DBAPI connection after each one is created. The callable
is assembled into a first_connect/connect pool listener by the
connection strategy if non-None. Provides a simpler interface
- StaticPool now initializes, disposes and recreates without
opening a new connection - the connection is only opened when
first requested. dispose() also works on AssertionPool now.
- Added the ability to strip schema information when using
"tometadata" by passing "schema=None" as an argument. If schema
is not specified then the table's schema is retained.
- DeclarativeMeta exclusively uses cls.__dict__ (not dict_)
as the source of class information; _as_declarative exclusively
uses the dict_ passed to it as the source of class information
(which when using DeclarativeMeta is cls.__dict__). This should
in theory make it easier for custom metaclasses to modify
the state passed into _as_declarative.
- declarative now accepts mixin classes directly, as a means
to provide common functional and column-based elements on
all subclasses, as well as a means to propagate a fixed
set of __table_args__ or __mapper_args__ to subclasses.
For custom combinations of __table_args__/__mapper_args__ from
an inherited mixin to local, descriptors can now be used.
New details are all up in the Declarative documentation.
Thanks to Chris Withers for putting up with my strife
on this. [ticket:1707]
- the __mapper_args__ dict is copied when propagating to a subclass,
and is taken straight off the class __dict__ to avoid any
propagation from the parent. mapper inheritance already
propagates the things you want from the parent mapper.
- An exception is raised when a single-table subclass specifies
a column that is already present on the base class.
- Fixed reflection bug whereby when COLLATE was present,
nullable flag and server defaults would not be reflected.
- Fixed reflection of TINYINT(1) "boolean" columns defined with
integer flags like UNSIGNED.
- Further fixes for the mysql-connector dialect. [ticket:1668]
- Composite PK table on InnoDB where the "autoincrement" column
isn't first will emit an explicit "KEY" phrase within
CREATE TABLE thereby avoiding errors, [ticket:1496]
- Added reflection/create table support for a wide range
of MySQL keywords. [ticket:1634]
- Fixed import error which could occur reflecting tables on
a Windows host [ticket:1580]
- Re-established support for the pymssql dialect.
- Various fixes for implicit returning, reflection,
etc. - the MS-SQL dialects aren't quite complete
in 0.6 yet (but are close)
- Added basic support for mxODBC [ticket:1710].
- Removed the text_as_varchar option.
- "out" parameters require a type that is supported by
cx_oracle. An error will be raised if no cx_oracle
type can be found.
- Oracle 'DATE' now does not perform any result processing,
as the DATE type in Oracle stores full date+time objects,
that's what you'll get. Note that the generic types.Date
type *will* still call value.date() on incoming values,
however. When reflecting a table, the reflected type
will be 'DATE'.
- Added preliminary support for Oracle's WITH_UNICODE
mode. At the very least this establishes initial
support for cx_Oracle with Python 3. When WITH_UNICODE
mode is used in Python 2.xx, a large and scary warning
is emitted asking that the user seriously consider
the usage of this difficult mode of operation.
- The except_() method now renders as MINUS on Oracle,
which is more or less equivalent on that platform.
- Added support for rendering and reflecting
TIMESTAMP WITH TIME ZONE, i.e. TIMESTAMP(timezone=True).
- Oracle INTERVAL type can now be reflected.
- Added "native_datetime=True" flag to create_engine().
This will cause the DATE and TIMESTAMP types to skip
all bind parameter and result row processing, under
the assumption that PARSE_DECLTYPES has been enabled
on the connection. Note that this is not entirely
compatible with the "func.current_date()", which
will be returned as a string. [ticket:1685]
- Implemented a preliminary working dialect for Sybase,
with sub-implementations for Python-Sybase as well
as Pyodbc. Handles table
creates/drops and basic round trip functionality.
Does not yet include reflection or comprehensive
support of unicode/special expressions/etc.
- Changed the beaker cache example a bit to have a separate
RelationCache option for lazyload caching. This object
does a lookup among any number of potential attributes
more efficiently by grouping several into a common structure.
Both FromCache and RelationCache are simpler individually.
- Major cleanup work in the docs to link class, function, and
method names into the API docs. [ticket:1700/1702/1703]
- Major Release
- For the full set of feature descriptions, see
This document is a work in progress.
- All bug fixes and feature enhancements from the most
recent 0.5 version and below are also included within 0.6.
- Platforms targeted now include Python 2.4/2.5/2.6, Python
- Changes to query.update() and query.delete():
- the 'expire' option on query.update() has been renamed to
'fetch', thus matching that of query.delete().
'expire' is deprecated and issues a warning.
- query.update() and query.delete() both default to
'evaluate' for the synchronize strategy.
- the 'synchronize' strategy for update() and delete()
raises an error on failure. There is no implicit fallback
onto "fetch". Failure of evaluation is based on the
structure of criteria, so success/failure is deterministic
based on code structure.
- Enhancements on many-to-one relations:
- many-to-one relations now fire off a lazyload in fewer
cases, including in most cases will not fetch the "old"
value when a new one is replaced.
- many-to-one relation to a joined-table subclass now uses
get() for a simple load (known as the "use_get"
condition), i.e. Related->Sub(Base), without the need to
redefine the primaryjoin condition in terms of the base
- specifying a foreign key with a declarative column, i.e.
ForeignKey(MyRelatedClass.id) doesn't break the "use_get"
condition from taking place [ticket:1492]
- relation(), eagerload(), and eagerload_all() now feature
an option called "innerjoin". Specify `True` or `False` to
control whether an eager join is constructed as an INNER
or OUTER join. Default is `False` as always. The mapper
options will override whichever setting is specified on
relation(). Should generally be set for many-to-one, not
nullable foreign key relations to allow improved join
- the behavior of eagerloading such that the main query is
wrapped in a subquery when LIMIT/OFFSET are present now
makes an exception for the case when all eager loads are
many-to-one joins. In those cases, the eager joins are
against the parent table directly along with the
limit/offset without the extra overhead of a subquery,
since a many-to-one join does not add rows to the result.
- Enhancements / Changes on Session.merge():
- the "dont_load=True" flag on Session.merge() is deprecated
and is now "load=False".
- Session.merge() is performance optimized, using half the
call counts for "load=False" mode compared to 0.5 and
significantly fewer SQL queries in the case of collections
for "load=True" mode.
- merge() will not issue a needless merge of attributes if the
given instance is the same instance which is already present.
- merge() now also merges the "options" associated with a given
state, i.e. those passed through query.options() which follow
along with an instance, such as options to eagerly- or
lazyily- load various attributes. This is essential for
the construction of highly integrated caching schemes. This
is a subtle behavioral change vs. 0.5.
- A bug was fixed regarding the serialization of the "loader
path" present on an instance's state, which is also necessary
when combining the usage of merge() with serialized state
and associated options that should be preserved.
- The all new merge() is showcased in a new comprehensive
example of how to integrate Beaker with SQLAlchemy. See
the notes in the "examples" note below.
- Primary key values can now be changed on a joined-table inheritance
object, and ON UPDATE CASCADE will be taken into account when
the flush happens. Set the new "passive_updates" flag to False
on mapper() when using SQLite or MySQL/MyISAM. [ticket:1362]
- flush() now detects when a primary key column was updated by
an ON UPDATE CASCADE operation from another primary key, and
can then locate the row for a subsequent UPDATE on the new PK
value. This occurs when a relation() is there to establish
the relationship as well as passive_updates=True. [ticket:1671]
- the "save-update" cascade will now cascade the pending *removed*
values from a scalar or collection attribute into the new session
during an add() operation. This so that the flush() operation
will also delete or modify rows of those disconnected items.
- Using a "dynamic" loader with a "secondary" table now produces
a query where the "secondary" table is *not* aliased. This
allows the secondary Table object to be used in the "order_by"
attribute of the relation(), and also allows it to be used
in filter criterion against the dynamic relation.
- relation() with uselist=False will emit a warning when
an eager or lazy load locates more than one valid value for
the row. This may be due to primaryjoin/secondaryjoin
conditions which aren't appropriate for an eager LEFT OUTER
JOIN or for other conditions. [ticket:1643]
- an explicit check occurs when a synonym() is used with
map_column=True, when a ColumnProperty (deferred or otherwise)
exists separately in the properties dictionary sent to mapper
with the same keyname. Instead of silently replacing
the existing property (and possible options on that property),
an error is raised. [ticket:1633]
- a "dynamic" loader sets up its query criterion at construction
time so that the actual query is returned from non-cloning
accessors like "statement".
- the "named tuple" objects returned when iterating a
Query() are now pickleable.
- mapping to a select() construct now requires that you
make an alias() out of it distinctly. This to eliminate
confusion over such issues as [ticket:1542]
- query.join() has been reworked to provide more consistent
behavior and more flexibility (includes [ticket:1537])
- query.select_from() accepts multiple clauses to produce
multiple comma separated entries within the FROM clause.
Useful when selecting from multiple-homed join() clauses.
- query.select_from() also accepts mapped classes, aliased()
constructs, and mappers as arguments. In particular this
helps when querying from multiple joined-table classes to ensure
the full join gets rendered.
- query.get() can be used with a mapping to an outer join
where one or more of the primary key values are None.
- query.from_self(), query.union(), others which do a
"SELECT * from (SELECT...)" type of nesting will do
a better job translating column expressions within the subquery
to the columns clause of the outer query. This is
potentially backwards incompatible with 0.5, in that this
may break queries with literal expressions that do not have labels
applied (i.e. literal('foo'), etc.)
- relation primaryjoin and secondaryjoin now check that they
are column-expressions, not just clause elements. this prohibits
things like FROM expressions being placed there directly.
- `expression.null()` is fully understood the same way
None is when comparing an object/collection-referencing
attribute within query.filter(), filter_by(), etc.
- added "make_transient()" helper function which transforms a
persistent/ detached instance into a transient one (i.e.
deletes the instance_key and removes from any session.)
- the allow_null_pks flag on mapper() is deprecated, and
the feature is turned "on" by default. This means that
a row which has a non-null value for any of its primary key
columns will be considered an identity. The need for this
scenario typically only occurs when mapping to an outer join.
- the mechanics of "backref" have been fully merged into the
finer grained "back_populates" system, and take place entirely
within the _generate_backref() method of RelationProperty. This
makes the initialization procedure of RelationProperty
simpler and allows easier propagation of settings (such as from
subclasses of RelationProperty) into the reverse reference.
The internal BackRef() is gone and backref() returns a plain
tuple that is understood by RelationProperty.
- The version_id_col feature on mapper() will raise a warning when
used with dialects that don't support "rowcount" adequately.
- added "execution_options()" to Query, to so options can be
passed to the resulting statement. Currently only
Select-statements have these options, and the only option
used is "stream_results", and the only dialect which knows
"stream_results" is psycopg2.
- Query.yield_per() will set the "stream_results" statement
- Deprecated or removed:
* 'allow_null_pks' flag on mapper() is deprecated. It does
nothing now and the setting is "on" in all cases.
* 'transactional' flag on sessionmaker() and others is
removed. Use 'autocommit=True' to indicate 'transactional=False'.
* 'polymorphic_fetch' argument on mapper() is removed.
Loading can be controlled using the 'with_polymorphic'
* 'select_table' argument on mapper() is removed. Use
'with_polymorphic=("*", <some selectable>)' for this
* 'proxy' argument on synonym() is removed. This flag
did nothing throughout 0.5, as the "proxy generation"
behavior is now automatic.
* Passing a single list of elements to eagerload(),
eagerload_all(), contains_eager(), lazyload(),
defer(), and undefer() instead of multiple positional
*args is deprecated.
* Passing a single list of elements to query.order_by(),
query.group_by(), query.join(), or query.outerjoin()
instead of multiple positional *args is deprecated.
* query.iterate_instances() is removed. Use query.instances().
* Query.query_from_parent() is removed. Use the
sqlalchemy.orm.with_parent() function to produce a
"parent" clause, or alternatively query.with_parent().
* query._from_self() is removed, use query.from_self()
* the "comparator" argument to composite() is removed.
* RelationProperty._get_join() is removed.
* the 'echo_uow' flag on Session is removed. Use
logging on the "sqlalchemy.orm.unitofwork" name.
* session.clear() is removed. use session.expunge_all().
* session.save(), session.update(), session.save_or_update()
are removed. Use session.add() and session.add_all().
* the "objects" flag on session.flush() remains deprecated.
* the "dont_load=True" flag on session.merge() is deprecated
in favor of "load=False".
* ScopedSession.mapper remains deprecated. See the
usage recipe at
* passing an InstanceState (internal SQLAlchemy state object) to
attributes.init_collection() or attributes.get_history() is
deprecated. These functions are public API and normally
expect a regular mapped object instance.
* the 'engine' parameter to declarative_base() is removed.
Use the 'bind' keyword argument.
- the "autocommit" flag on select() and text() as well
as select().autocommit() are deprecated - now call
.execution_options(autocommit=True) on either of those
constructs, also available directly on Connection and orm.Query.
- the autoincrement flag on column now indicates the column
which should be linked to cursor.lastrowid, if that method
is used. See the API docs for details.
- an executemany() now requires that all bound parameter
sets require that all keys are present which are
present in the first bound parameter set. The structure
and behavior of an insert/update statement is very much
determined by the first parameter set, including which
defaults are going to fire off, and a minimum of
guesswork is performed with all the rest so that performance
is not impacted. For this reason defaults would otherwise
silently "fail" for missing parameters, so this is now guarded
- returning() support is native to insert(), update(),
delete(). Implementations of varying levels of
functionality exist for Postgresql, Firebird, MSSQL and
Oracle. returning() can be called explicitly with column
expressions which are then returned in the resultset,
usually via fetchone() or first().
insert() constructs will also use RETURNING implicitly to
get newly generated primary key values, if the database
version in use supports it (a version number check is
performed). This occurs if no end-user returning() was
- union(), intersect(), except() and other "compound" types
of statements have more consistent behavior w.r.t.
parenthesizing. Each compound element embedded within
another will now be grouped with parenthesis - previously,
the first compound element in the list would not be grouped,
as SQLite doesn't like a statement to start with
parenthesis. However, Postgresql in particular has
precedence rules regarding INTERSECT, and it is
more consistent for parenthesis to be applied equally
to all sub-elements. So now, the workaround for SQLite
is also what the workaround for PG was previously -
when nesting compound elements, the first one usually needs
".alias().select()" called on it to wrap it inside
of a subquery. [ticket:1665]
- insert() and update() constructs can now embed bindparam()
objects using names that match the keys of columns. These
bind parameters will circumvent the usual route to those
keys showing up in the VALUES or SET clause of the generated
- the Binary type now returns data as a Python string
(or a "bytes" type in Python 3), instead of the built-
in "buffer" type. This allows symmetric round trips
of binary data. [ticket:1524]
- Added a tuple_() construct, allows sets of expressions
to be compared to another set, typically with IN against
composite primary keys or similar. Also accepts an
IN with multiple columns. The "scalar select can
have only one column" error message is removed - will
rely upon the database to report problems with
- User-defined "default" and "onupdate" callables which
accept a context should now call upon
"context.current_parameters" to get at the dictionary
of bind parameters currently being processed. This
dict is available in the same way regardless of
single-execute or executemany-style statement execution.
- multi-part schema names, i.e. with dots such as
"dbo.master", are now rendered in select() labels
with underscores for dots, i.e. "dbo_master_table_column".
This is a "friendly" label that behaves better
in result sets. [ticket:1428]
- removed needless "counter" behavior with select()
labelnames that match a column name in the table,
i.e. generates "tablename_id" for "id", instead of
"tablename_id_1" in an attempt to avoid naming
conflicts, when the table has a column actually
named "tablename_id" - this is because
the labeling logic is always applied to all columns
so a naming conflict will never occur.
- calling expr.in_(), i.e. with an empty list, emits a warning
before issuing the usual "expr != expr" clause. The
"expr != expr" can be very expensive, and it's preferred
that the user not issue in_() if the list is empty,
instead simply not querying, or modifying the criterion
as appropriate for more complex situations.
- Added "execution_options()" to select()/text(), which set the
default options for the Connection. See the note in "engines".
- Deprecated or removed:
* "scalar" flag on select() is removed, use
* "shortname" attribute on bindparam() is removed.
* postgres_returning, firebird_returning flags on
insert(), update(), delete() are deprecated, use
the new returning() method.
* fold_equivalents flag on join is deprecated (will remain
until [ticket:1131] is implemented)
- transaction isolation level may be specified with
create_engine(... isolation_level="..."); available on
postgresql and sqlite. [ticket:443]
- Connection has execution_options(), generative method
which accepts keywords that affect how the statement
is executed w.r.t. the DBAPI. Currently supports
"stream_results", causes psycopg2 to use a server
side cursor for that statement, as well as
"autocommit", which is the new location for the "autocommit"
option from select() and text(). select() and
text() also have .execution_options() as well as
- fixed the import for entrypoint-driven dialects to
not rely upon silly tb_info trick to determine import
error status. [ticket:1630]
- added first() method to ResultProxy, returns first row and
closes result set immediately.
- RowProxy objects are now pickleable, i.e. the object returned
by result.fetchone(), result.fetchall() etc.
- RowProxy no longer has a close() method, as the row no longer
maintains a reference to the parent. Call close() on
the parent ResultProxy instead, or use autoclose.
- ResultProxy internals have been overhauled to greatly reduce
method call counts when fetching columns. Can provide a large
speed improvement (up to more than 100%) when fetching large
result sets. The improvement is larger when fetching columns
that have no type-level processing applied and when using
results as tuples (instead of as dictionaries). Many
thanks to Elixir's Gaëtan de Menten for this dramatic
improvement ! [ticket:1586]
- Databases which rely upon postfetch of "last inserted id"
to get at a generated sequence value (i.e. MySQL, MS-SQL)
now work correctly when there is a composite primary key
where the "autoincrement" column is not the first primary
key column in the table.
- the last_inserted_ids() method has been renamed to the
- setting echo=False on create_engine() now sets the loglevel
to WARN instead of NOTSET. This so that logging can be
disabled for a particular engine even if logging
for "sqlalchemy.engine" is enabled overall. Note that the
default setting of "echo" is `None`. [ticket:1554]
- ConnectionProxy now has wrapper methods for all transaction
lifecycle events, including begin(), rollback(), commit()
begin_nested(), begin_prepared(), prepare(), release_savepoint(),
- Connection pool logging now uses both INFO and DEBUG
log levels for logging. INFO is for major events such
as invalidated connections, DEBUG for all the acquire/return
logging. `echo_pool` can be False, None, True or "debug"
the same way as `echo` works.
- All pyodbc-dialects now support extra pyodbc-specific
kw arguments 'ansi', 'unicode_results', 'autocommit'.
- the "threadlocal" engine has been rewritten and simplified
and now supports SAVEPOINT operations.
- deprecated or removed
* result.last_inserted_ids() is deprecated. Use
* dialect.get_default_schema_name(connection) is now
public via dialect.default_schema_name.
* the "connection" argument from engine.transaction() and
engine.run_callable() is removed - Connection itself
now has those methods. All four methods accept
*args and **kwargs which are passed to the given callable,
as well as the operating connection.
- the `__contains__()` method of `MetaData` now accepts
strings or `Table` objects as arguments. If given
a `Table`, the argument is converted to `table.key` first,
i.e. "[schemaname.]<tablename>" [ticket:1541]
- deprecated MetaData.connect() and
ThreadLocalMetaData.connect() have been removed - send
the "bind" attribute to bind a metadata.
- deprecated metadata.table_iterator() method removed (use
- deprecated PassiveDefault - use DefaultClause.
- the "metadata" argument is removed from DefaultGenerator
and subclasses, but remains locally present on Sequence,
which is a standalone construct in DDL.
- Removed public mutability from Index and Constraint
These should be constructed declaratively (i.e. in one
- The "start" and "increment" attributes on Sequence now
generate "START WITH" and "INCREMENT BY" by default,
on Oracle and Postgresql. Firebird doesn't support
these keywords right now. [ticket:1545]
- UniqueConstraint, Index, PrimaryKeyConstraint all accept
lists of column names or column objects as arguments.
- Other removed things:
- Table.key (no idea what this was for)
- Table.primary_key is not assignable - use
- Column.bind (get via column.table.bind)
- Column.metadata (get via column.table.metadata)
- Column.sequence (use column.default)
- ForeignKey(constraint=some_parent) (is now private _constraint)
- The use_alter flag on ForeignKey is now a shortcut option
for operations that can be hand-constructed using the
DDL() event system. A side effect of this refactor is
that ForeignKeyConstraint objects with use_alter=True
will *not* be emitted on SQLite, which does not support
ALTER for foreign keys.
- ForeignKey and ForeignKeyConstraint objects now correctly
copy() all their public keyword arguments. [ticket:1605]
- Table reflection has been expanded and generalized into
a new API called "sqlalchemy.engine.reflection.Inspector".
The Inspector object provides fine-grained information about
a wide variety of schema information, with room for expansion,
including table names, column names, view definitions, sequences,
- Views are now reflectable as ordinary Table objects. The same
Table constructor is used, with the caveat that "effective"
primary and foreign key constraints aren't part of the reflection
results; these have to be specified explicitly if desired.
- The existing autoload=True system now uses Inspector underneath
so that each dialect need only return "raw" data about tables
and other objects - Inspector is the single place that information
is compiled into Table objects so that consistency is at a maximum.
- the DDL system has been greatly expanded. the DDL() class
now extends the more generic DDLElement(), which forms the basis
of many new constructs:
These support "on" and "execute-at()" just like plain DDL()
does. User-defined DDLElement subclasses can be created and
linked to a compiler using the sqlalchemy.ext.compiler extension.
- The signature of the "on" callable passed to DDL() and
DDLElement() is revised as follows:
"ddl" - the DDLElement object itself.
"event" - the string event name.
"target" - previously "schema_item", the Table or
MetaData object triggering the event.
"connection" - the Connection object in use for the operation.
**kw - keyword arguments. In the case of MetaData before/after
create/drop, the list of Table objects for which
CREATE/DROP DDL is to be issued is passed as the kw
argument "tables". This is necessary for metadata-level
DDL that is dependent on the presence of specific tables.
- the "schema_item" attribute of DDL has been renamed to
- dialect refactor
- Dialect modules are now broken into database dialects
plus DBAPI implementations. Connect URLs are now
preferred to be specified using dialect+driver://...,
i.e. "mysql+mysqldb://scott:tiger@localhost/test". See
the 0.6 documentation for examples.
- the setuptools entrypoint for external dialects is now
- the "owner" keyword argument is removed from Table. Use
"schema" to represent any namespaces to be prepended to
the table name.
- server_version_info becomes a static attribute.
- dialects receive an initialize() event on initial
connection to determine connection properties.
- dialects receive a visit_pool event have an opportunity
to establish pool listeners.
- cached TypeEngine classes are cached per-dialect class
instead of per-dialect.
- new UserDefinedType should be used as a base class for
new types, which preserves the 0.5 behavior of
- The result_processor() method of all type classes now
accepts a second argument "coltype", which is the DBAPI
type argument from cursor.description. This argument
can help some types decide on the most efficient processing
of result values.
- Deprecated Dialect.get_params() removed.
- Dialect.get_rowcount() has been renamed to a descriptor
"rowcount", and calls cursor.rowcount directly. Dialects
which need to hardwire a rowcount in for certain calls
should override the method to provide different behavior.
- DefaultRunner and subclasses have been removed. The job
of this object has been simplified and moved into
ExecutionContext. Dialects which support sequences should
add a `fire_sequence()` method to their execution context
- Functions and operators generated by the compiler now use
(almost) regular dispatch functions of the form
"visit_<opname>" and "visit_<funcname>_fn" to provide
customed processing. This replaces the need to copy the
"functions" and "operators" dictionaries in compiler
subclasses with straightforward visitor methods, and also
allows compiler subclasses complete control over
rendering, as the full _Function or _BinaryExpression
object is passed in.
- New dialects: pg8000, zxjdbc, and pypostgresql
- The "postgres" dialect is now named "postgresql" !
Connection strings look like:
The "postgres" name remains for backwards compatiblity
in the following ways:
- There is a "postgres.py" dummy dialect which
allows old URLs to work, i.e.
- The "postgres" name can be imported from the old
"databases" module, i.e. "from
sqlalchemy.databases import postgres" as well as
"dialects", "from sqlalchemy.dialects.postgres
import base as pg", will send a deprecation
- Special expression arguments are now named
"postgresql_returning" and "postgresql_where", but
the older "postgres_returning" and
"postgres_where" names still work with a
- "postgresql_where" now accepts SQL expressions which
can also include literals, which will be quoted as needed.
- The psycopg2 dialect now uses psycopg2's "unicode extension"
on all new connections, which allows all String/Text/etc.
types to skip the need to post-process bytestrings into
unicode (an expensive step due to its volume). Other
dialects which return unicode natively (pg8000, zxjdbc)
also skip unicode post-processing.
- Added new ENUM type, which exists as a schema-level
construct and extends the generic Enum type. Automatically
associates itself with tables and their parent metadata
to issue the appropriate CREATE TYPE/DROP TYPE
commands as needed, supports unicode labels, supports
- INTERVAL supports an optional "precision" argument
corresponding to the argument that PG accepts.
- using new dialect.initialize() feature to set up
- somewhat better support for % signs in table/column names;
psycopg2 can't handle a bind parameter name of
%(foobar)s however and SQLA doesn't want to add overhead
just to treat that one non-existent use case.
- Inserting NULL into a primary key + foreign key column
will allow the "not null constraint" error to raise,
not an attempt to execute a nonexistent "col_id_seq"
- autoincrement SELECT statements, i.e. those which
select from a procedure that modifies rows, now work
with server-side cursor mode (the named cursor isn't
used for such statements.)
- postgresql dialect can properly detect pg "devel" version
strings, i.e. "8.5devel" [ticket:1636]
- The psycopg2 now respects the statement option
"stream_results". This option overrides the connection setting
"server_side_cursors". If true, server side cursors will be
used for the statement. If false, they will not be used, even
if "server_side_cursors" is true on the
- New dialects: oursql, a new native dialect,
MySQL Connector/Python, a native Python port of MySQLdb,
and of course zxjdbc on Jython.
- VARCHAR/NVARCHAR will not render without a length, raises
an error before passing to MySQL. Doesn't impact
CAST since VARCHAR is not allowed in MySQL CAST anyway,
the dialect renders CHAR/NCHAR in those cases.
- all the _detect_XXX() functions now run once underneath
- somewhat better support for % signs in table/column names;
MySQLdb can't handle % signs in SQL when executemany() is used,
and SQLA doesn't want to add overhead just to treat that one
non-existent use case. [ticket:1279]
- the BINARY and MSBinary types now generate "BINARY" in all
cases. Omitting the "length" parameter will generate
"BINARY" with no length. Use BLOB to generate an unlengthed
- the "quoting='quoted'" argument to MSEnum/ENUM is deprecated.
It's best to rely upon the automatic quoting.
- ENUM now subclasses the new generic Enum type, and also handles
unicode values implicitly, if the given labelnames are unicode
- a column of type TIMESTAMP now defaults to NULL if
"nullable=False" is not passed to Column(), and no default
is present. This is now consistent with all other types,
and in the case of TIMESTAMP explictly renders "NULL"
due to MySQL's "switching" of default nullability
for TIMESTAMP columns. [ticket:1539]
- unit tests pass 100% with cx_oracle !
- support for cx_Oracle's "native unicode" mode which does
not require NLS_LANG to be set. Use the latest 5.0.2 or
later of cx_oracle.
- an NCLOB type is added to the base types.
- use_ansi=False won't leak into the FROM/WHERE clause of
a statement that's selecting from a subquery that also
- added native INTERVAL type to the dialect. This supports
only the DAY TO SECOND interval type so far due to lack
of support in cx_oracle for YEAR TO MONTH. [ticket:1467]
- usage of the CHAR type results in cx_oracle's
FIXED_CHAR dbapi type being bound to statements.
- the Oracle dialect now features NUMBER which intends
to act justlike Oracle's NUMBER type. It is the primary
numeric type returned by table reflection and attempts
to return Decimal()/float/int based on the precision/scale
- func.char_length is a generic function for LENGTH
- ForeignKey() which includes onupdate=<value> will emit a
warning, not emit ON UPDATE CASCADE which is unsupported
- the keys() method of RowProxy() now returns the result
column names *normalized* to be SQLAlchemy case
insensitive names. This means they will be lower case for
case insensitive names, whereas the DBAPI would normally
return them as UPPERCASE names. This allows row keys() to
be compatible with further SQLAlchemy operations.
- using new dialect.initialize() feature to set up
- using types.BigInteger with Oracle will generate
- "case sensitivity" feature will detect an all-lowercase
case-sensitive column name during reflect and add
"quote=True" to the generated Column, so that proper
quoting is maintained.
- the keys() method of RowProxy() now returns the result
column names *normalized* to be SQLAlchemy case
insensitive names. This means they will be lower case for
case insensitive names, whereas the DBAPI would normally
return them as UPPERCASE names. This allows row keys() to
be compatible with further SQLAlchemy operations.
- using new dialect.initialize() feature to set up
- "case sensitivity" feature will detect an all-lowercase
case-sensitive column name during reflect and add
"quote=True" to the generated Column, so that proper
quoting is maintained.
- MSSQL + Pyodbc + FreeTDS now works for the most part,
with possible exceptions regarding binary data as well as
unicode schema identifiers.
- the "has_window_funcs" flag is removed. LIMIT/OFFSET
usage will use ROW NUMBER as always, and if on an older
version of SQL Server, the operation fails. The behavior
is exactly the same except the error is raised by SQL
server instead of the dialect, and no flag setting is
required to enable it.
- the "auto_identity_insert" flag is removed. This feature
always takes effect when an INSERT statement overrides a
column that is known to have a sequence on it. As with
"has_window_funcs", if the underlying driver doesn't
support this, then you can't do this operation in any
case, so there's no point in having a flag.
- using new dialect.initialize() feature to set up
- removed references to sequence which is no longer used.
implicit identities in mssql work the same as implicit
sequences on any other dialects. Explicit sequences are
enabled through the use of "default=Sequence()". See
the MSSQL dialect documentation for more information.
- DATE, TIME and DATETIME types can now take optional storage_format
and regexp argument. storage_format can be used to store those types
using a custom string format. regexp allows to use a custom regular
expression to match string values from the database.
- Time and DateTime types now use by a default a stricter regular
expression to match strings from the database. Use the regexp
argument if you are using data stored in a legacy format.
- __legacy_microseconds__ on SQLite Time and DateTime types is not
supported anymore. You should use the storage_format argument
- Date, Time and DateTime types are now stricter in what they accept as
bind parameters: Date type only accepts date objects (and datetime
ones, because they inherit from date), Time only accepts time
objects, and DateTime only accepts date and datetime objects.
- Table() supports a keyword argument "sqlite_autoincrement", which
applies the SQLite keyword "AUTOINCREMENT" to the single integer
primary key column when generating DDL. Will prevent generation of
a separate PRIMARY KEY constraint. [ticket:1016]
- new dialects
- postgresql+pypostgresql (partial)
- The construction of types within dialects has been totally
overhauled. Dialects now define publically available types
as UPPERCASE names exclusively, and internal implementation
types using underscore identifiers (i.e. are private).
The system by which types are expressed in SQL and DDL
has been moved to the compiler system. This has the
effect that there are much fewer type objects within
most dialects. A detailed document on this architecture
for dialect authors is in
- Types no longer make any guesses as to default
parameters. In particular, Numeric, Float, NUMERIC,
FLOAT, DECIMAL don't generate any length or scale unless
- types.Binary is renamed to types.LargeBinary, it only
produces BLOB, BYTEA, or a similar "long binary" type.
New base BINARY and VARBINARY
types have been added to access these MySQL/MS-SQL specific
types in an agnostic way [ticket:1664].
- String/Text/Unicode types now skip the unicode() check
on each result column value if the dialect has
detected the DBAPI as returning Python unicode objects
natively. This check is issued on first connect
using "SELECT CAST 'some text' AS VARCHAR(10)" or
equivalent, then checking if the returned object
is a Python unicode. This allows vast performance
increases for native-unicode DBAPIs, including
pysqlite/sqlite3, psycopg2, and pg8000.
- Most types result processors have been checked for possible speed
improvements. Specifically, the following generic types have been
optimized, resulting in varying speed improvements:
Unicode, PickleType, Interval, TypeDecorator, Binary.
Also the following dbapi-specific implementations have been improved:
Time, Date and DateTime on Sqlite, ARRAY on Postgresql,
Time on MySQL, Numeric(as_decimal=False) on MySQL, oursql and
pypostgresql, DateTime on cx_oracle and LOB-based types on cx_oracle.
- Reflection of types now returns the exact UPPERCASE
type within types.py, or the UPPERCASE type within
the dialect itself if the type is not a standard SQL
type. This means reflection now returns more accurate
information about reflected types.
- Added a new Enum generic type. Enum is a schema-aware object
to support databases which require specific DDL in order to
use enum or equivalent; in the case of PG it handles the
details of `CREATE TYPE`, and on other databases without
native enum support will by generate VARCHAR + an inline CHECK
constraint to enforce the enum.
- The Interval type includes a "native" flag which controls
if native INTERVAL types (postgresql + oracle) are selected
if available, or not. "day_precision" and "second_precision"
arguments are also added which propagate as appropriately
to these native types. Related to [ticket:1467].
- The Boolean type, when used on a backend that doesn't
have native boolean support, will generate a CHECK
constraint "col IN (0, 1)" along with the int/smallint-
based column type. This can be switched off if
desired with create_constraint=False.
Note that MySQL has no native boolean *or* CHECK constraint
support so this feature isn't available on that platform.
- PickleType now uses == for comparison of values when
mutable=True, unless the "comparator" argument with a
comparsion function is specified to the type. Objects
being pickled will be compared based on identity (which
defeats the purpose of mutable=True) if __eq__() is not
overridden or a comparison function is not provided.
- The default "precision" and "scale" arguments of Numeric
and Float have been removed and now default to None.
NUMERIC and FLOAT will be rendered with no numeric
arguments by default unless these values are provided.
- AbstractType.get_search_list() is removed - the games
that was used for are no longer necessary.
- Added a generic BigInteger type, compiles to
BIGINT or NUMBER(19). [ticket:1125]
- sqlsoup has been overhauled to explicitly support an 0.5 style
session, using autocommit=False, autoflush=True. Default
behavior of SQLSoup now requires the usual usage of commit()
and rollback(), which have been added to its interface. An
explcit Session or scoped_session can be passed to the
constructor, allowing these arguments to be overridden.
- sqlsoup db.<sometable>.update() and delete() now call
query(cls).update() and delete(), respectively.
- sqlsoup now has execute() and connection(), which call upon
the Session methods of those names, ensuring that the bind is
in terms of the SqlSoup object's bind.
- sqlsoup objects no longer have the 'query' attribute - it's
not needed for sqlsoup's usage paradigm and it gets in the
way of a column that is actually named 'query'.
- The signature of the proxy_factory callable passed to
association_proxy is now (lazy_collection, creator,
value_attr, association_proxy), adding a fourth argument
that is the parent AssociationProxy argument. Allows
serializability and subclassing of the built in collections.
- association_proxy now has basic comparator methods .any(),
.has(), .contains(), ==, !=, thanks to Scott Torborg.
- The "query_cache" examples have been removed, and are replaced
with a fully comprehensive approach that combines the usage of
Beaker with SQLAlchemy. New query options are used to indicate
the caching characteristics of a particular Query, which
can also be invoked deep within an object graph when lazily
loading related objects. See /examples/beaker_caching/README.
For changes prior to 0.6, see CHANGES_PRE_06