pre-execute column defaults don't apply type processing

Issue #3923 resolved
Mike Bayer repo owner created an issue

it seems like pre-execute is much more rarely used but it still applies to primary keys, fails here as SQLite's column processing is missing:

from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.ext.declarative import declarative_base

Base = declarative_base()


class A(Base):
    __tablename__ = 'a'
    id = Column(Integer, primary_key=True)
    date = Column(DateTime, default=text("datetime()"), primary_key=True)


e = create_engine("sqlite://", echo=True)
Base.metadata.create_all(e)

s = Session(e)

s.add(A(id=1))
s.commit()

patch:

diff --git a/lib/sqlalchemy/engine/default.py b/lib/sqlalchemy/engine/default.py
index 18c3276..0210ad7 100644
--- a/lib/sqlalchemy/engine/default.py
+++ b/lib/sqlalchemy/engine/default.py
@@ -1039,7 +1039,11 @@ class DefaultExecutionContext(interfaces.ExecutionContext):
             # TODO: expensive branching here should be
             # pulled into _exec_scalar()
             conn = self.connection
-            c = expression.select([default.arg]).compile(bind=conn)
+            if not default._arg_is_typed:
+                default_arg = expression.type_coerce(default.arg, type_)
+            else:
+                default_arg = default.arg
+            c = expression.select([default_arg]).compile(bind=conn)
             return conn._execute_compiled(c, (), {}).scalar()
         else:
             return default.arg
diff --git a/lib/sqlalchemy/sql/schema.py b/lib/sqlalchemy/sql/schema.py
index 9bb0eee..f5716af 100644
--- a/lib/sqlalchemy/sql/schema.py
+++ b/lib/sqlalchemy/sql/schema.py
@@ -2045,6 +2045,14 @@ class ColumnDefault(DefaultGenerator):
             not self.is_clause_element and \
             not self.is_sequence

+    @util.memoized_property
+    @util.dependencies("sqlalchemy.sql.sqltypes")
+    def _arg_is_typed(self, sqltypes):
+        if self.is_clause_element:
+            return not isinstance(self.arg.type, sqltypes.NullType)
+        else:
+            return False
+
     def _maybe_wrap_callable(self, fn):
         """Wrap callables that don't accept a context.

Comments (4)

  1. Mike Bayer reporter

    Apply type processing to untyped preexec default clause

    Fixed bug where a SQL-oriented Python-side column default could fail to be executed properly upon INSERT in the "pre-execute" codepath, if the SQL itself were an untyped expression, such as plain text. The "pre- execute" codepath is fairly uncommon however can apply to non-integer primary key columns with SQL defaults when RETURNING is not used.

    Tests exist here to ensure typing is applied to a typed expression for default, but in the case of an untyped SQL value, we know the type from the column, so apply this.

    Change-Id: I5d8b391611c137b9f700115a50a2bf5b30abfe94 Fixes: #3923

    → <<cset 4eb4010c1a1c>>

  2. Mike Bayer reporter

    EDIT: fixes #3870

    • Document the most transparent way to set JSON.NULL for a column default.

    The JSON.NULL value is special in that while it is a Python-side value, it represents "do this special behavior", and is not at all like any other Python-side value for which normally, that's exactly the type of data we want back. So in this case, encourage the user to use a SQL expression that is exact, so in the ORM context, you get back what was actually persisted in the column. There's some variants of this such as literal(JSON.NULL, JSON) but text("'null'") is simpler.

    Change-Id: I0339fafdc03e4b2f533d53970b2f74c774f2687b Fixes: #3923 (cherry picked from commit 4d4ea792e7ea7945de9c9b1e005ce09b2feb45b5)

    → <<cset 32300ea9faf7>>

  3. Log in to comment