- attached sqlexample.py
sharing a session across processing.Process subclass instances breaks with postgresql
When I create a subclass of processing.Process and use it to populate an sqldatabase with sqlalchemy (with each process owning a shared session) everything seems to fall apart. When I do the same with sqlite, everything goes smoothly. I am attaching a script that illustrates the problem.
One thing to note is that running that script gives a different error message each time you run it. My postgresql version is 8.3.3
by the way, my name is Craig Swank (craig_swank@nrel.gov)
Comments (2)
-
Account Deleted -
repo owner - changed status to wontfix
a psycopg2 connection is not going to be portable to a chld process. Add this line so that a new connection pool is opened for each child:
class RubbishProcess(processing.Process): def __init__(self, session, data): self.session = session session.bind = create_engine('postgres://path_to_database') self.data = data
Note that you aren't really "sharing" the Session across processes here, since there's no IPC in use that would communicate state between parent/child Session instances once the subprocess is started. So in general its a better idea to keep a distinct Session local to each subprocess as well (i.e. reusing the parent Session is generally misleading).
- Log in to comment
example code