Add MutableSet / MutableList classes to sqlalchemy.ext.mutable
these are basic classes that should be present, they are only missing because we need implementations + unit tests.
original:
Edit: I've solved it in the comment, but I've no idea what I'm doing... Maybe the ticket should be amended to 'Provide canonical implementations for MutableSet and MutableList in ext.mutable'
I've created a MutableSet class, closely following the MutableDict class in ext.mutable.
Unfortunately it doesn't pickle correctly and I can't figure out what the problem is.
from sqlalchemy import *
from sqlalchemy.ext.mutable import Mutable, MutableDict
test_table = Table('test_table, metadata,
Column('test_pk', String(40), primary_key=True),
Column('as_dict', MutableDict.as_mutable(JsonType)),
Column('as_set', MutableSet.as_mutable(JsonType))
my error message after using pickle.dumps on an object that contains the MutableSet column. Note the 'remove' function mentioned in the error is to do with _parents WeakRef rather than MutableSet.remove)
File "/usr/lib/python2.7/pickle.py", line 748, in save_global
(obj, module, name))
PicklingError: Can't pickle <function remove at 0xb48a041c>: it's not found as weakref.remove
And the MutableSet class (and JsonType supporting class - included for completeness)
class MutableSet(Mutable, set):
@classmethod
def coerce(cls, key, value):
"Convert plain sets to MutableSet."
if not isinstance(value, MutableSet):
if isinstance(value, set):
return MutableSet(value)
elif isinstance(value, dict):
return MutableDict(value)
# this call will raise ValueError
return Mutable.coerce(key, value)
else:
return value
def __getstate__(self):
return set(self)
def __setstate__(self, state):
self.update(state)
def update(self, *args):
set.update(self, *args, **kwargs)
self.changed()
def intersection_update(self, *args):
set.intersection_update(self, *args)
self.changed()
def difference_update(self, *args):
set.difference_update(self, *args)
self.changed()
def symmetric_difference_update(self, *args):
set.symmetric_difference_update(self, *args)
self.changed()
def add(self, elem):
set.add(self, elem)
self.changed()
def remove(self, elem):
set.remove(self, elem)
self.changed()
def discard(self, elem):
set.discard(self, elem)
self.changed()
def pop(self):
set.pop(self)
self.changed()
def clear(self):
set.clear(self)
self.changed()
class JsonType(types.TypeDecorator):
impl = types.Unicode
def process_bind_param(self, value, engine):
import jsonpickle
if isinstance(value, MutableList):
value = [v for v in value]
if isinstance(value, MutableDict):
value = dict(value)
if isinstance(value, MutableSet):
value = set(value)
return unicode(jsonpickle.encode(value))
def process_result_value(self, value, engine):
import jsonpickle
if value:
return jsonpickle.decode(value)
else:
return {}
def copy_value(self, value):
return deepcopy(value)
def compare_values(self, x, y):
return x == y
Comments (11)
-
reporter -
reporter - edited description
-
repo owner absolutely these should be provided.
Apparently
__getstate__()
is not effective for sets: http://stackoverflow.com/a/25950163/34549whoops.
-
repo owner - changed milestone to 1.0.xx
- marked as major
-
assigned issue to
- edited description
- changed title to Added MutableSet / MutableList classes to sqlalchemy.ext.mutable
- marked as enhancement
-
repo owner - changed title to Add MutableSet / MutableList classes to sqlalchemy.ext.mutable
-
Can you add a 'changed' event to go along with these? Analogous to a 'set' event for immutable attributes.
-
repo owner new event interface, sure. can you open a separate ticket for that and illustrate what you want the event to look like? thanks.
-
reporter Here's a dump of what I'm currently using in production: https://gist.github.com/eoghanmurray/1360da8635f0a6a6d528 (fixes a bug with **kwargs in set.update)
-
I've added issue
#3303. Thanks. -
repo owner - changed milestone to 1.x.xx
-
repo owner - changed status to resolved
- add changelog, migration, version flags and some extra notes
to the new MutableList and MutableSet classes, fixes
#3297
→ <<cset a99a32d3d166>>
- Log in to comment
I've solved it by adding the following to MutableSet, but I've no idea why this fixes it: