Exposing two phase commit to external data managers

I'd like to make use of SQLAlchemy's two phase commit support to integrate SQLAlchemy transactions with other Zope transactions (http://pypi.python.org/pypi/transaction). The collaborations involved (from Zope's perspective) are best documented here: http://svn.zope.org/ZODB/trunk/src/ZODB/collaborations.txt?rev=39908

Currently in collective.lead (http://pypi.python.org/pypi/collective.lead) we do a full commit of the SQLAlchemy transaction as the last data manager to commit in the tpc_vote (prepare) stage. I think that the commit needs to be split into three. Here is my attempt for SessionTransaction:

    def tpc_begin(self):
        if self.session.extension is not None:
            self.session.extension.before_commit(self.session)

        if self.autoflush:
            self.session.flush()

    def prepare(self):
        try:
            for t in util.Set(self.__connections.values()):
                t[1](1).prepare()
        except:
            self.rollback()
            raise

    def finalize(self):
        for t in util.Set(self.__connections.values()):
            t[1](1).commit()

        if self.session.extension is not None:
            self.session.extension.after_commit(self.session)

        self.close()
        return self.__parent


    def commit(self):
        # As external data manager would not call this
        if self.__parent is not None and not self.nested:
            return self.__parent

        self.tpc_begin()

        if self.session.twophase:
            self.prepare()

        return self.finalize()

The idea behind this is that you want to minimize the time between the prepare and commit prepared calls, so it is useful to be able to signal a flush before issuing the prepare statements.

Zope's transaction module defines a separate tpc_rollback() method, but I think SQLAlchemy keeps tabs of whether a prepare() has been issued already and does the correct thing when you call rollback().

I got a bit lost in the SQLAlchemy code trying to follow this all through and ended up with uncommitted prepared transactions.

Comments (4)