fallback when reporting expressions in exceptions

[Mike Bayer]

from sqlalchemy.sql import expression
from sqlalchemy.ext.compiler import compiles
from sqlalchemy import create_engine, select, DateTime

e = create_engine('sqlite://')
r = e.execute("select 1")


class utcnow(expression.FunctionElement):
    type = DateTime()

@compiles(utcnow, 'postgresql')
def pg_utcnow(element, compiler, **kw):
    return "TIMEZONE('utc', CURRENT_TIMESTAMP)"

c1 = select([utcnow()](utcnow())).label('foo')

r.fetchone()[c1](c1)

proposal:

diff -r a3a547324eff14c0247c903cb2aa415123661c29 lib/sqlalchemy/engine/base.py
--- a/lib/sqlalchemy/engine/base.py Wed May 25 12:41:53 2011 -0400
+++ b/lib/sqlalchemy/engine/base.py Thu May 26 09:41:36 2011 -0400
@@ -2384,7 +2384,9 @@
                 result = map[key.name.lower()](key.name.lower())
         if result is None:
             raise exc.NoSuchColumnError(
-                "Could not locate column in row for column '%s'" % key)
+                "Could not locate column in row for column '%s'" % 
+                    expression._string_or_unprintable(key)
+                )
         else:
             map[key](key) = result
         return result
diff -r a3a547324eff14c0247c903cb2aa415123661c29 lib/sqlalchemy/sql/expression.py
--- a/lib/sqlalchemy/sql/expression.py  Wed May 25 12:41:53 2011 -0400
+++ b/lib/sqlalchemy/sql/expression.py  Thu May 26 09:41:36 2011 -0400
@@ -1205,6 +1205,15 @@
         element = element.__clause_element__()
     return element.key

+def _string_or_unprintable(element):
+    if isinstance(element, basestring):
+        return element
+    else:
+        try:
+            return str(element)
+        except:
+            return "unprintable element %r" % element
+
 def _literal_as_text(element):
     if isinstance(element, Visitable):
         return element

[Former user]

I've just attached two patches that I'd like to propose in addition to the change you proposed above. I couldn't decide on the best approach, so I decided to do both approaches and let you decide ;) Both pass all the current tests.

Compiling the full expression certainly makes sense in the r.fetchone()c1 use case outlined above, but it would be great to avoid compiling it when simply fetching a result set. Ideally it shouldn't cost extra to have deferred property except when you actually undefer it. These patches reduce total execution time by 30-40% for "DBSession.query(MappedClass).all()" when MappedClass has two nontrivial select expressions with lots of critera to compile.

Admittedly the 30-40% difference is still very small, and it'll be even less significant with a bigger resultset. This is just a slight optimization that I stumbled across while debugging, and I'd be happy to contribute back to the project if either of these patches are of any use :)

Cheers, Nathan Wright

[Mike Bayer]

Did some poking around here, the real issue is just row.__contains__(), otherwise the NSCE isn't really used except if there is an actual error to raise all the way, which is not a performance case. The ORM uses __contains__() a bit. Here's an easy way to squash that from raising an error:

diff -r ba299476b827ada34d01360e3024f87dd56dc967 lib/sqlalchemy/engine/base.py
--- a/lib/sqlalchemy/engine/base.py Thu May 26 13:30:26 2011 -0400
+++ b/lib/sqlalchemy/engine/base.py Fri May 27 21:15:41 2011 -0400
@@ -2369,7 +2369,7 @@
         if self._keymap.setdefault(name, rec) is not rec:
             self._keymap[name](name) = (processor, None)

-    def _key_fallback(self, key):
+    def _key_fallback(self, key, raiseerr=True):
         map = self._keymap
         result = None
         if isinstance(key, basestring):
@@ -2383,8 +2383,11 @@
             elif hasattr(key, 'name') and key.name.lower() in map:
                 result = map[key.name.lower()](key.name.lower())
         if result is None:
-            raise exc.NoSuchColumnError(
-                "Could not locate column in row for column '%s'" % key)
+            if raiseerr:
+                raise exc.NoSuchColumnError(
+                    "Could not locate column in row for column '%s'" % key)
+            else:
+                return None
         else:
             map[key](key) = result
         return result
@@ -2393,11 +2396,7 @@
         if key in self._keymap:
             return True
         else:
-            try:
-                self._key_fallback(key)
-                return True
-            except exc.NoSuchColumnError:
-                return False
+            return self._key_fallback(key, False) is not None

     def __getstate__(self):
         return {

I bet this even fixes your original issue, perhaps it was one of these __contains__() calls....

[Former user]

That looks great and it fixes the original issue, thanks! :)

[Mike Bayer]

both in for 0.6.8 and 0.7:

09080baad1a9f910a5a406dfad9241104ccbc6d8 da043fa75207ee5052af7ff1ae31c627e38d6d5c

[Mike Bayer]

Removing milestone: 0.6.8 (automated comment)