Cant JSON serialize sqlalchemy.util.langhelpers.symbol
Starting with version 0.9.0, simplejson sees util.symbol objects as integers, causing it to generate invalid JSON.
We're trying to serialize stack traces and function arguments as JSON, and although we've defined a custom JSONEncoder default() method to handle non-standard data types, we're getting invalid JSON like:
...
"args": [
"<sqlalchemy.orm.attributes.ScalarAttributeImpl object at 0x4966090>",
"<sqlalchemy.orm.state.InstanceState object at 0x52ca650>",
"{'_data': {'follower_count': None, 'followers': None, 'following': None, 'locale': u'zh_hans', 'region': None, 'timezone': u'Africa/Cairo', 'utc_offset': Decimal('2.0')}, '_modified': {'locale': u'zh_hans'}, '_sa_instance_state': <sqlalchemy.orm.state.InstanceState object at 0x52ca650>, 'locale': u'zh_hans'}",
symbol('PASSIVE_OFF')
],
...
...because while our custom default() method handles data types simplejson doesn't recognize, it thinks "symbol('PASSIVE_OFF')" is an int, and dumps the raw value into the JSON.
To reproduce (using python 2.7.6, sqlalchemy 1.0.4, simplejson 3.6.5):
Python 2.7.6 (default, Sep 9 2014, 15:04:36)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlalchemy as sa
>>> import json
>>> sym = sa.util.symbol("PASSIVE_OFF")
>>> data = ['str1', 123, sym]
>>> print json.JSONEncoder().encode(data)
["str1", 123, symbol('PASSIVE_OFF')]
>>> print sym
symbol('PASSIVE_OFF')
>>> type(sym)
<class 'sqlalchemy.util.langhelpers.symbol'>
>>> isinstance(sym,int)
True
Python 3 handles this slightly differently: rather than "symbol('PASSIVE_OFF')" in the JSON, we see integers like -4343837943729882877 in the output of encode().
Comments (3)
-
repo owner -
reporter Mike,
Though I can't see why "need to be able to JSON serialize the internals of the library" has to be a supported use case.
For context, this came up here at Rollbar, where we're trying capture as much app context information as is possible/practical when an exception's thrown.
We have a hack/workaround for our issue (https://github.com/rollbar/pyrollbar/issues/60)
...and I've created a simplejson issue (https://github.com/simplejson/simplejson/issues/118)
I'm not sure if I can really add anything else.
-
repo owner - changed status to resolved
looks like simplejson accommodated it upstream: https://github.com/simplejson/simplejson/issues/118
- Log in to comment
IMO this is a bug in Python's json library. This is more than just serializing it correctly, these are constants that are compared using "is", so you need to pull them from the module.
However, I'm not able to get it to recognize the int subclass, in the script below "default" fails:
Though I can't see why "need to be able to JSON serialize the internals of the library" has to be a supported use case. If these constants were just simple ints, your program would still be failing, as these need to be the same int value; it's not safe to rely on "1 is 1" in Python, I've seen this fail on Pypy under cloudy circumstances.