- changed milestone to 0.9.10
PostgreSQL non-native Enum fails with unicode
Issue #3354
resolved
sqlalchemy 0.9.9, python 2.7.9, osx, PostgreSQL database with encoding unicode.
# -*- coding: utf8 -*-
from sqlalchemy.types import String, Integer, Enum
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, create_engine
from sqlalchemy.orm import Session
import sys
Base = declarative_base()
class Thing(Base):
__tablename__ = 'thing'
id = Column(Integer, primary_key=True)
name = Column(String(1))
value = Column(Enum('B',u'Ü',name='myvalue', native_enum=False, convert_unicode=None))
engine = create_engine(sys.argv[1], echo=True,
client_encoding='utf8')
Base.metadata.create_all(engine)
sess = Session(engine)
sess.add(Thing(name='A', value=u'Ü'))
sess.commit()
sess.query(Thing).filter_by(name='A').first()
Traceback (most recent call last):
File "./utfbug.py", line 23, in <module>
sess.query(Thing).filter_by(name='A').first()
File "<path>/sqlalchemy/orm/query.py", line 2370, in first
ret = list(self[0:1])
File "<path>/sqlalchemy/orm/query.py", line 2231, in __getitem__
return list(res)
File "<path>/sqlalchemy/orm/loading.py", line 103, in instances
util.raise_from_cause(err)
File "<path>/sqlalchemy/util/compat.py", line 199, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb)
File "<path>/sqlalchemy/orm/loading.py", line 74, in instances
rows = [process[0](row, None) for row in fetch]
File "<path>/sqlalchemy/orm/loading.py", line 459, in _instance
populate_state(state, dict_, row, isnew, only_load_props)
File "<path>/sqlalchemy/orm/loading.py", line 309, in populate_state
populator(state, dict_, row)
File "<path>/sqlalchemy/orm/strategies.py", line 169, in fetch_col
dict_[key] = row[col]
UnicodeEncodeError: 'ascii' codec can't encode character u'\xdc' in position 0: ordinal not in range(128)
if I set convert_unicode=False, it works fine. mysql ist not affected.
Comments (3)
-
repo owner -
repo owner - changed status to resolved
- Fixed a long-standing bug where the :class:
.Enum
type as used with the psycopg2 dialect in conjunction with non-ascii values andnative_enum=False
would fail to decode return results properly. This stemmed from when the PG :class:.postgresql.ENUM
type used to be a standalone type without a "non native" option. fixes#3354 - corrected the assertsql comparison rule to expect a non-ascii SQL string
→ <<cset ecd7b31d5eae>>
-
repo owner - Fixed a long-standing bug where the :class:
.Enum
type as used with the psycopg2 dialect in conjunction with non-ascii values andnative_enum=False
would fail to decode return results properly. This stemmed from when the PG :class:.postgresql.ENUM
type used to be a standalone type without a "non native" option. fixes#3354cherry pick from ecd7b31d5eaed138e699293719f70260da3c978d
→ <<cset 0775c90ce090>>
- Fixed a long-standing bug where the :class:
- Log in to comment
this one is sort of funny because each time I look at it the cause gets more and more intricate. First I thought it was just the "convert_unicode" param being passed to String incorrectly. Then I realized it's this: https://bitbucket.org/zzzeek/sqlalchemy/src/ee40c7afaabe33e2080607de0045a9fd0a4748d8/lib/sqlalchemy/dialects/postgresql/psycopg2.py?at=master#cl-357 and I figured, that must be some old legacy thing, but then I realized, oh, we actually need that for a "native" enum still, and the reason it's not checking for "native_enum" is because that code precedes when we had the generic "Enum" type; it used to be a standalone object.