Exception cycle

[Guillaume DOUMENC (NeoRezo)]

With python3.4, pyodbc I was able to raise an error which then call reraise with cause same as value, creating then a cycle

I will provide a pull to fix this issue

[Mike Bayer]

I absolutely need to see a test case illustrating how this happens, thanks!

[Guillaume DOUMENC (NeoRezo)]

Very difficult as I have this issue with pyodbc only on AWS server (red hat) not on my server (ubuntu). Anyway I sent a pull request accepted to ipython to avoid traceback to go to infinite loop..

How can I help you? Provide a traceback? Access to AWS server?

[Mike Bayer]

the use case that the pull request claims is occurring is not valid. the "cause" should never match the original exception case. I cannot find anywhere in SQLAlchemy where this occurs. My only guess would be use of the handle_error() event to rethrow the exception that's already present.

I can't accept PRs without a clear use case or error case, otherwise we have no idea what actual issue we are just covering up and making even more hard to find later.

[Mike Bayer]

e.g., if you had at least a stack trace, that would be extremely helpful.

[Guillaume DOUMENC (NeoRezo)]

Sorry for not be able to create a test case.. Looking to try but not easy. I think the issue is in pyodbc over mssql. I have a db with a long name table : ArtTechInvLotEmplacement. Any call to this table crashes (only on mssql not on postgre).. I continue to investigate and hope I will be able to give you more info.

Do you think is it possible that sys.exc_info() with internal issue (pyodbc) breaks the stack call? Any idea of a test I can try?

[Guillaume DOUMENC (NeoRezo)]

Are you sure that you cannot reraise the same exception from here ?:

def raise_from_cause(exception, exc_info=None):
        if exc_info is None:
            exc_info = sys.exc_info()
        exc_type, exc_value, exc_tb = exc_info
        reraise(type(exception), exception, tb=exc_tb, cause=exc_value)

[Mike Bayer]

does your application have logfiles? any stack trace in a logfile would do fine here.

It's not controversial that the method itself, if you pass the same cause in, will cause the endless loop that you see, any more than if you take a Py3K exception and point it to itself on __cause__. The issue is, how is anything calling it like that.

[Mike Bayer]

the raise_from_cause is supposed to receive a new, not-yet-raised exception that is to be linked to the current one. passing in the same one is an error, we can of course add a check for that, but there's no place that this happens in the code. as i said earlier, the only way i can guess to make this happen is a mis-written handle_error() event handler. but if it is happening someplace in the code, I want to add coverage for it, because currently it's an uncovered codepath, and that's a bug.

[Guillaume DOUMENC (NeoRezo)]

I'm able to reproduce it :

(kool)studiogdo@Zalman:~/workspace/kool$ ./manager.py shell
--------------------------------------------------------------------------------
INFO in __init__ [/home/studiogdo/workspace/kool/kool/__init__.py:128]:
Kool started
--------------------------------------------------------------------------------

In [1]: from applications.env import *

In [2]: dest = make_session(aws_mssql('francemet'))

In [3]: from kool.modules.models import ArtTechInvLotEmplacement

In [4]: len(list(dest.query(ArtTechInvLotEmplacement).all()))
/home/studiogdo/.venvs/kool/lib/python3.4/site-packages/sqlalchemy/connectors/pyodbc.py:143: SAWarning: Unrecognized server version info '95.10.13055'.   Version specific behaviors may not function properly.   If using ODBC with FreeTDS, ensure server version 7.0 or 8.0, not 4.2, is configured in the FreeTDS configuration.
 super(PyODBCConnector, self).initialize(connection)
> /home/studiogdo/.venvs/kool/lib/python3.4/site-packages/sqlalchemy/util/compat.py(180)reraise()
    179         ipdb.set_trace()
--> 180         if cause is not None:
    181             value.__cause__ = cause

ipdb> cause
NoSuchColumnError("Could not locate column in row for column 'ArtTechInvLotEmplacement.NUM_INV'",)
ipdb> value
NoSuchColumnError("Could not locate column in row for column 'ArtTechInvLotEmplacement.NUM_INV'",)
ipdb> cause is value
True

[Guillaume DOUMENC (NeoRezo)]

I've found the issue and seems to be a real bug... ;) I've attached a trace file to explain the issue.. For me seems clear as added same exception in traceback thru sys.exc_info() Hope this will help

[Mike Bayer]

OK, so, yes, I can place an error there and get "cause" to be the same as the passed exception. However. No endless loop. Below is a simple test case. Can you figure out what specific error formatting / etc system is causing the loop? I'm wondering if "exc.cause = self" is just a simple case that's already handled by the interpretrer, and maybe your enviornment has an old Python version on it which doesn't handle it?

from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.ext.declarative import declarative_base

Base = declarative_base()


class A(Base):
    __tablename__ = 'a'
    id = Column(Integer, primary_key=True)

e = create_engine("sqlite://", echo=True)
Base.metadata.create_all(e)

s = Session(e)
s.add(A())
s.commit()


from sqlalchemy import event

import logging
logging.basicConfig()

log = logging.getLogger(__name__)


@event.listens_for(A, "load")
def go(*arg, **kw):
    raise Exception("boom")

try:
    s.query(A).first()
except:
    log.error("error", exc_info=True)

[Guillaume DOUMENC (NeoRezo)]

I think I wasn't enough precise so excuse me for that.. The endless loop was in ipython. I put a pull request (accepted) to track such cycle in exception cause dependency (even if not direct as in this case.. https://github.com/ipython/ipython/pull/9121). Anyway I think that such cause cycle should not be raised by any module...

[Mike Bayer]

ah. Yes agree, though also, that's a bug in iPython you should report.

[Mike Bayer]

oh you did, OK!

[Mike Bayer]

• Fixed bug where some exception re-raise scenarios would attach the exception to itself as the "cause"; while the Python 3 interpreter is OK with this, it could cause endless loops in iPython. fixes #3625
• add tests for reraise, raise_from_cause
• raise_from_cause is the same on py2k/3k, use just one function

→ <<cset d4d9a6524886>>

[Mike Bayer]

• Fixed bug where some exception re-raise scenarios would attach the exception to itself as the "cause"; while the Python 3 interpreter is OK with this, it could cause endless loops in iPython. fixes #3625
• add tests for reraise, raise_from_cause
• raise_from_cause is the same on py2k/3k, use just one function

(cherry picked from commit d4d9a6524886eb33644e8ce42212267fa569e555)

→ <<cset 23e60cd8ebc8>>

[Mike Bayer]

thanks for reporting!