Pickle of Properties object fails
Trying to pickle a Properties object as shown below fails.
import pickle
from sqlalchemy.util import Properties
data = {'hello': 'bye'}
props = Properties(data)
pickle.loads(pickle.dumps(props))
Traceback (most recent call last):
File "test_collections.py", line 14, in <module>
pickle.loads(pickle.dumps(props))
File "/usr/lib64/python2.7/pickle.py", line 1380, in dumps
Pickler(file, protocol).dump(obj)
File "/usr/lib64/python2.7/pickle.py", line 224, in dump
self.save(obj)
File "/usr/lib64/python2.7/pickle.py", line 306, in save
rv = reduce(self.proto)
File "/home/pieter/.virtualenvs/proigia-rest/lib64/python2.7/copy_reg.py", line 84, in _reduce_ex
dict = getstate()
File "/home/pieter/.virtualenvs/proigia-rest/lib/python2.7/site-packages/sqlalchemy/util/_collections.py", line 202, in __getstate__
return {'_data': self.__dict__['_data']}
File "/home/pieter/.virtualenvs/proigia-rest/lib/python2.7/site-packages/sqlalchemy/util/_collections.py", line 211, in __getattr__
raise AttributeError(key)
AttributeError: __dict__
It's clear Properties is __slot__
based and thus has no __dict__
.
SQLAlchemy 1.0.13
Comments (7)
-
repo owner -
repo owner -
reporter I'll try to make a PR.
I do not use the the
Properties
directly: The original exception was returned fromImmutableProperties
which I then traced back to theProperties
object.After a lot of searching I have found I was hitting this by pickling a column in a table which has a colanderalchemy object attached:
import sqlalchemy as sa from sqlalchemy.ext import declarative import colanderalchemy Base = declarative.declarative_base() import pickle class Table1(Base): __tablename__ = 'table1' col1 = sa.Column(sa.String(), primary_key=True) col2 = sa.Column(sa.String()) def main(): colanderalchemy.setup_schema(None, Table1) pickle.dumps(Table1.col1) if __name__ == '__main__': main()
Resulting in:
Traceback (most recent call last): File "sqlalchemy_test2.py", line 22, in <module> main() File "sqlalchemy_test2.py", line 19, in main pickle.dumps(Table1.col1) File "/usr/lib64/python2.7/pickle.py", line 1380, in dumps Pickler(file, protocol).dump(obj) File "/usr/lib64/python2.7/pickle.py", line 224, in dump self.save(obj) File "/usr/lib64/python2.7/pickle.py", line 331, in save self.save_reduce(obj=obj, *rv) File "/usr/lib64/python2.7/pickle.py", line 425, in save_reduce save(state) File "/usr/lib64/python2.7/pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "/usr/lib64/python2.7/pickle.py", line 655, in save_dict self._batch_setitems(obj.iteritems()) File "/usr/lib64/python2.7/pickle.py", line 669, in _batch_setitems save(v) File "/usr/lib64/python2.7/pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "/usr/lib64/python2.7/pickle.py", line 655, in save_dict self._batch_setitems(obj.iteritems()) File "/usr/lib64/python2.7/pickle.py", line 669, in _batch_setitems save(v) File "/usr/lib64/python2.7/pickle.py", line 331, in save self.save_reduce(obj=obj, *rv) File "/usr/lib64/python2.7/pickle.py", line 425, in save_reduce save(state) File "/usr/lib64/python2.7/pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "/usr/lib64/python2.7/pickle.py", line 655, in save_dict self._batch_setitems(obj.iteritems()) File "/usr/lib64/python2.7/pickle.py", line 669, in _batch_setitems save(v) File "/usr/lib64/python2.7/pickle.py", line 306, in save rv = reduce(self.proto) File "/home/pieter/.virtualenvs/proigia-rest/lib64/python2.7/copy_reg.py", line 84, in _reduce_ex dict = getstate() File "/home/pieter/.virtualenvs/proigia-rest/lib/python2.7/site-packages/sqlalchemy/util/_collections.py", line 202, in __getstate__ return {'_data': self.__dict__['_data']} KeyError: '_data'
Commenting out the
colanderalchemy.setup_schema(None, Table1)
line will result in a different pickle exception (which I have not debugged yet as I've decided to take an other route that never tries to pickle a column object). -
repo owner - changed status to resolved
Repair pickling for Properties object
Fixed bug whereby the
__getstate__
/__setstate__
methods for sqlalchemy.util.Properties were non-working due to the transition in the 1.0 series to__slots__
. The issue potentially impacted some third-party applications. Pull request courtesy Pieter Mulder.Fixes:
#3728Change-Id: I01ebd425bbfe145747fea2edd0d2d412c74fd84d Pull-request: https://github.com/zzzeek/sqlalchemy/pull/286 (cherry picked from commit cab57e9bab04fbdea44690c08dff379a29eaab32)→ <<cset fcfff7712817>>
-
repo owner Repair pickling for Properties object
Fixed bug whereby the
__getstate__
/__setstate__
methods for sqlalchemy.util.Properties were non-working due to the transition in the 1.0 series to__slots__
. The issue potentially impacted some third-party applications. Pull request courtesy Pieter Mulder.Fixes:
#3728Change-Id: I01ebd425bbfe145747fea2edd0d2d412c74fd84d Pull-request: https://github.com/zzzeek/sqlalchemy/pull/286 (cherry picked from commit cab57e9bab04fbdea44690c08dff379a29eaab32)→ <<cset 39c18b53aa7e>>
-
repo owner Fix
#3728Pickle of Properties object failsChange-Id: I01ebd425bbfe145747fea2edd0d2d412c74fd84d Pull-request: https://github.com/zzzeek/sqlalchemy/pull/286
→ <<cset 20fe9ad17131>>
-
repo owner Repair pickling for Properties object
Fixed bug whereby the
__getstate__
/__setstate__
methods for sqlalchemy.util.Properties were non-working due to the transition in the 1.0 series to__slots__
. The issue potentially impacted some third-party applications. Pull request courtesy Pieter Mulder.Fixes:
#3728Change-Id: I01ebd425bbfe145747fea2edd0d2d412c74fd84d Pull-request: https://github.com/zzzeek/sqlalchemy/pull/286→ <<cset cab57e9bab04>>
- Log in to comment
Easy to fix (PR w/ test would help), however currently no part of SQLAlchemy seems to make use of Properties directly except for the ColumnCollection -> OrderedProperties chain, for which ColumnCollection provides its own getstate/setstate and that's where the test coverage is. May I ask what your use case for direct use of Properties is ? sqlalchemy.util is not "private" per-se but the constructs in here are intended only to support the library features themselves.