arbitrarily truncate SQL log messages ? allow some option ?

[Former user]

I like SQLAlchemy's logging integration, but when the logged data would be huge (say I'm issuing a 15000-line INSERT statement), SQLAlchemy should recognize that and truncate logged information to a reasonable maximum (say 1000 characters). Otherwise I'm forced to disable logging.

[Former user]

Ah, I forgot to add my e-mail address as cc: [e-mail]. Sorry.

[Former user]

Sorry again :) It's solipsis at pitrou dot net.

[Mike Bayer]

Arbitrarily/implicitly truncating a generated SQL statement in the log sounds awfully unreasonable to me. I'm looking at a SQL statement right in my terminal right now that's 3195 characters, and I need every single one of them to paste into my command line tool for testing.

This use case can be handled easily within the Python logging module itself using a custom handler so for the moment I'm -1 on this as a SQLAlchemy behavior and -10 as an automatic behavior. Will forward to the development list for alternative suggestions.

[Mike Bayer]

Oh, you're talking about the parameter lists? That's a different story. We do truncate those in exception messages already. Can you please add some specifics here as I am open to the latter and not the former.

[Former user]

Yes, I mean the parameter lists :) For example executing conn.execute(sometable.insert(), some_huge_list_of_dicts).

[Mike Bayer]

patch is attached some tests would be nice, for both exception throw (surprised we don't have that) as well as log output

[Mike Bayer]

d3c1f245efb22b45c77561345346469cf6a82ddd

[Mike Bayer]

Removing milestone: 0.7.3 (automated comment)