sharing a session across processing.Process subclass instances breaks with postgresql

Issue #1156 resolved
Former user created an issue

When I create a subclass of processing.Process and use it to populate an sqldatabase with sqlalchemy (with each process owning a shared session) everything seems to fall apart. When I do the same with sqlite, everything goes smoothly. I am attaching a script that illustrates the problem.

One thing to note is that running that script gives a different error message each time you run it. My postgresql version is 8.3.3

by the way, my name is Craig Swank (craig_swank@nrel.gov)

Comments (2)

  1. Mike Bayer repo owner

    a psycopg2 connection is not going to be portable to a chld process. Add this line so that a new connection pool is opened for each child:

    class RubbishProcess(processing.Process):
        def __init__(self, session, data):
            self.session = session
            session.bind = create_engine('postgres://path_to_database')
            self.data = data
    

    Note that you aren't really "sharing" the Session across processes here, since there's no IPC in use that would communicate state between parent/child Session instances once the subprocess is started. So in general its a better idea to keep a distinct Session local to each subprocess as well (i.e. reusing the parent Session is generally misleading).

  2. Log in to comment