Assert that class_manager(cls) is present on InstanceState deserialize

Issue #1610 resolved
Former user created an issue

In a Django application using Elixir to map models to DB, very few often I get this exception on the logs:

mod_wsgi (pid=13845): Exception occurred processing WSGI script '/var/www/webtv/entry.wsgi'.
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/django/core/handlers/wsgi.py", line 241, in __call__
    response = self.get_response(request)
  File "/usr/lib/python2.6/site-packages/django/core/handlers/base.py", line 73, in get_response
    response = middleware_method(request)
  File "/var/www/webtv/master/handler.py", line 38, in process_request
    return handler.processRequest(request)
  File "/var/www/webtv/master/gateway/handler.py", line 24, in processRequest
    return client.select(clientId)
  File "/var/www/webtv/master/client.py", line 61, in select
    databaseCredentials = DatabaseCredentials.get_by(client=client, type=u'main')
  File "/usr/lib/python2.6/site-packages/elixir/entity.py", line 1000, in get_by
    return cls.query.filter_by(*args, **kwargs).first()
  File "/usr/lib/python2.6/site-packages/sqlalchemy/orm/query.py", line 1231, in first
    ret = list(self[0:1](0:1))
  File "/usr/lib/python2.6/site-packages/sqlalchemy/orm/query.py", line 1152, in __getitem__
    return list(res)
  File "/usr/lib/python2.6/site-packages/sqlalchemy/orm/query.py", line 1292, in __iter__
    return self._execute_and_instances(context)
  File "/usr/lib/python2.6/site-packages/sqlalchemy/orm/query.py", line 1295, in _execute_and_instances
    result = self.session.execute(querycontext.statement, params=self._params, mapper=self._mapper_zero_or_none())
  File "/usr/lib/python2.6/site-packages/sqlalchemy/orm/session.py", line 755, in execute
    clause, params or {})
  File "/usr/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 824, in execute
    return Connection.executors[c](c)(self, object, multiparams, params)
  File "/usr/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 872, in _execute_clauseelement
    parameters=params
  File "/usr/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 938, in __create_execution_context
    return dialect.execution_ctx_cls(dialect, connection=self, **kwargs)
  File "/usr/lib/python2.6/site-packages/sqlalchemy/engine/default.py", line 167, in __init__
    self.compiled_parameters = [for m in parameters](compiled.construct_params(m))
  File "/usr/lib/python2.6/site-packages/sqlalchemy/sql/compiler.py", line 243, in construct_params
    pd[self.bind_names[bindparam](self.bind_names[bindparam)] = bindparam.value()
  File "/usr/lib/python2.6/site-packages/sqlalchemy/orm/strategies.py", line 398, in <lambda>
    bindparam.value = lambda: mapper._get_committed_attr_by_column(o, bind_to_col[bindparam.key](bindparam.key))
  File "/usr/lib/python2.6/site-packages/sqlalchemy/orm/mapper.py", line 1089, in _get_committed_attr_by_column
    return self._get_committed_state_attr_by_column(state, column)
  File "/usr/lib/python2.6/site-packages/sqlalchemy/orm/mapper.py", line 1092, in _get_committed_state_attr_by_column
    return self._get_col_to_prop(column).getcommitted(state, column, passive=passive)
  File "/usr/lib/python2.6/site-packages/sqlalchemy/orm/properties.py", line 102, in getcommitted
    return state.get_impl(self.key).get_committed_value(state, state.dict, passive=passive)
  File "/usr/lib/python2.6/site-packages/sqlalchemy/orm/state.py", line 92, in get_impl
    return self.manager.get_impl(key)
AttributeError: 'NoneType' object has no attribute 'get_impl

Notice that subsequent requests are handled correctly, leaving me with no clue on the problem.

I start the connection at the module level in a custom middleware I set in Django, the code there is:

engine = sqlalchemy.create_engine(connectionString,    
    pool_recycle=7200,                                 
#    echo=masterSettings.DEBUG,                        
#    echo_pool=masterSettings.DEBUG,                   
    )

elixir.metadata.bind = engine                          
elixir.options_defaults['autosetup']('autosetup') = True

class DatabaseSession(object):                         
    """                                                
    SQLalchemy/Elixir master database session controler
    """                                                
    def process_response(self, request, response):     
        """                                            
        Close database session                         
        """                                            
        try:                                           
            elixir.session.close()                     
        except:                                        
           pass                                        
        return response

Comments (5)

  1. Mike Bayer repo owner

    an InstanceState object can't be created without a manager, and then it's never changed. So the only theory I have here is that you're seeing some kind of concurrency-related abuse on mapped objects as they are loaded or deserialized (in this case the object referenced by "client" is not in its proper state).

    nothing we could fix on this end without a reproducing test case.

  2. Former user Account Deleted

    Thank you for your feedback, you gave me a hint towards the solution!

    That client would be a elixir retrieved object but on rare occasions the object would be loaded from memcached instead, as so it couldn't be used in the get_by method as it probably lacked some of the information of the original elixir object.

    That said, I just specified get_by(client_id=client_id... instead, and it all works!

    Thanks again for the help!

  3. Mike Bayer repo owner

    Using plain pickle.dumps/loads on a mapped instance should return it in the correct state - InstanceState implements __setstate__() which restores the manager. However, if your mappers weren't compiled yet, then it would fail.

    I think I may want to even assert that manager is not None when InstanceState deserializes - since this is a common situation (just never seen it fail in that way before).

  4. Log in to comment