- changed status to closed
Exception when pickle.dumps(DeclarativeObject) with sqlalchemy.ext.mutable.Mutable field
Issue #3552
closed
Hi there,
today we noticed an explosion when we tried to pickle.dumps one of our model objects with an error like this:
pickle.PicklingError: Can't pickle <function remove at 0x107afcb90>: it's not found as weakref.remove
Googling we found this: https://groups.google.com/forum/#!topic/sqlalchemy/b3eoWwhKriI which lead us to reduce the issue to this sample code which seems to be pretty minimal in showing the error:
#!/usr/bin/env python
import json
from sqlalchemy import types
# Basically stolen from
# http://docs.sqlalchemy.org/en/rel_0_8/core/types.html#marshal-json-strings
class JSONEncodedDict(types.TypeDecorator):
"""Represents an immutable structure as a json-encoded string.
Usage::
JSONEncodedDict(255)
Made mutation aware by the MutableDict below!
Defaults to an empty dict if read as NULL from the DB for migration friendliness.
"""
impl = types.VARCHAR
python_type = dict
def process_bind_param(self, value, dialect):
if value is not None:
value = json.dumps(value)
return value
def process_result_value(self, value, dialect):
if value is not None:
value = json.loads(value, object_hook=dict)
else:
value = {}
return value
from sqlalchemy.ext.mutable import Mutable
class MutableDict(Mutable, dict):
@classmethod
def coerce(cls, key, value):
"Convert plain dictionaries to MutableDict."
if not isinstance(value, MutableDict):
if isinstance(value, dict):
return MutableDict(value)
# this call will raise ValueError
return Mutable.coerce(key, value)
else:
return value
def __setitem__(self, key, value):
"Detect dictionary set events and emit change events."
dict.__setitem__(self, key, value)
self.changed()
def __delitem__(self, key):
"Detect dictionary del events and emit change events."
dict.__delitem__(self, key)
self.changed()
MutableDict.associate_with(JSONEncodedDict)
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base(constructor=None)
from sqlalchemy import Column, Integer
class User(Base):
__tablename__ = 'user'
id = Column("iduser", Integer, primary_key=True)
settings = Column(JSONEncodedDict(4096), default=dict)
from sqlalchemy import create_engine
from sqlalchemy.orm import scoped_session, sessionmaker
engine = create_engine('sqlite://')
DBSession = scoped_session(sessionmaker())
DBSession.configure(bind=engine)
Base.metadata.bind = engine
Base.metadata.create_all()
import pickle
user = User()
DBSession.add(user)
DBSession.flush()
pickle.dumps(user)
print('Worked')
This still errors in the latest version from pypi (1.0.8).
What can you make from this? It looks like an error to us - but maybe we're using mutable wrong?
Comments (2)
-
repo owner -
reporter Doh... directly below the documentation of the mutable dict.... Ahem.
Well, thanks!
- Log in to comment
please see http://docs.sqlalchemy.org/en/rel_1_0/orm/extensions/mutable.html#supporting-pickling thanks!