DBAPI connections go invalid on KeyboardInterrupt, GreenletExit e.g BaseException but not Exception, simple invalidate() fixes.

Issue #3803 resolved
Mike Bayer repo owner created an issue

case test3.py and test5.py illustrate pymysql connections being corrupted both by GreenletExit and KeyboardInterrupt. The program loop has many issues with "commands out of sync" for various reasons, but at the very least because when the error occurs, our context managers as well as pool logic wants to do a rollback(), such as Transaction.exit, which is already called for all BaseException subclasses, not just Exception, and this blows up because the connection is already corrupted; this occurs even if we use NullPool because we are still in the handling of the context.

Similar tests against pymysql alone (test2.py, test4.py) illustrate that largely the same kind of connection corruption can be shown against C-based mysqlclient with the KeyboardInterrupt example, so this is not just a pure Python greenlet thing.

_handle_dbapi_error() already handles Exception, adding support for BaseException and special "connection only, don't invalidate the pool" logic as in attached patch allow test3 / test5 to be run with no errors at all except normal interception of the KeyboardInterrupt / GreenletExit.

Comments (5)

  1. Mike Bayer reporter

    note the issue does not occur with psycopg2. so an existing psycopg2 that might want to intercept ctrl-C and continue on within a transaction would break from this change, unless they implemented a handle_error event to disable this. will need to put that as a workaround

  2. Mike Bayer reporter

    Handle BaseException in all _handle_dbapi_error

    Tests illustrate that exceptions like GreenletExit and even KeyboardInterrupt can corrupt the state of a DBAPI connection like that of pymysql and mysqlclient. Intercept BaseException errors within the handle_error scheme and invalidate just the connection alone in this case, but not the whole pool.

    The change is backwards-incompatible with a program that currently intercepts ctrl-C within a database transaction and wants to continue working on that transaction. Ensure the event hook can be used to reverse this behavior.

    Change-Id: Ifaa013c13826d123eef34e32b7e79fff74f1b21b Fixes: #3803

    → <<cset 7827dfb6726a>>

  3. Log in to comment