SystemError: <built-in function utf_8_decode> returned a result with an error set
Issue #3972
closed
Getting this error when a billiard.exceptions.SoftTimeLimitExceeded
exception occurs during a select query:
billiard.exceptions.SoftTimeLimitExceeded:
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/myapp/amqp.py", line 161, in decorated
return fn(*args, **kwargs)
File "/app/myapp/tasks.py", line 1399, in cache_user_summary
dt=current,
File "/app/myapp/utils.py", line 983, in get_summary
use_cache=use_cache)
File "/app/myapp/utils.py", line 601, in get_combined_durations
durations = get_durations(user=user, start=start, end=end, use_cache=use_cache)
File "/app/myapp/utils.py", line 713, in get_durations
heartbeats.extend(query.all())
File "/app/venv/lib/python3.5/site-packages/sqlalchemy/orm/query.py", line 2703, in all
return list(self)
File "/app/venv/lib/python3.5/site-packages/sqlalchemy/orm/loading.py", line 90, in instances
util.raise_from_cause(err)
File "/app/venv/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/app/venv/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
raise value
File "/app/venv/lib/python3.5/site-packages/sqlalchemy/orm/loading.py", line 67, in instances
fetch = cursor.fetchmany(query._yield_per)
File "/app/venv/lib/python3.5/site-packages/sqlalchemy/engine/result.py", line 1154, in fetchmany
self.cursor, self.context)
File "/app/venv/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1405, in _handle_dbapi_exception
util.reraise(*exc_info)
File "/app/venv/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
raise value
File "/app/venv/lib/python3.5/site-packages/sqlalchemy/engine/result.py", line 1147, in fetchmany
l = self.process_rows(self._fetchmany_impl(size))
File "/app/venv/lib/python3.5/site-packages/sqlalchemy/engine/result.py", line 1306, in _fetchmany_impl
row = self._fetchone_impl()
File "/app/venv/lib/python3.5/site-packages/sqlalchemy/engine/result.py", line 1296, in _fetchone_impl
self.__buffer_rows()
File "/app/venv/lib/python3.5/site-packages/sqlalchemy/engine/result.py", line 1283, in __buffer_rows
self.__rowbuffer = collections.deque(self.cursor.fetchmany(size))
File "/app/venv/lib/python3.5/encodings/utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
SystemError: <built-in function utf_8_decode> returned a result with an error set
Using Python 3.5.2, SQLAlchemy 1.1.9 and psycopg2 2.7.1.
Comments (4)
-
repo owner -
reporter I'll get a reproducible testcase. I think it's not related to corrupted data, just related to a long-running query being interrupted when the
SoftTimeLimitExceeded
is raised from Celery interrupting. -
repo owner in any case the stack trace shows the error is within psycopg2 fetchmany(). The method of "interruption" here would be the cause but if it can't be cleanly intercepted by psycopg2, you'd want to report up there.
-
repo owner - changed status to closed
please reopen if you have reproducible conditions to add
- Log in to comment
hi there -
do you plan to attach an mcve here? I have no idea what "billiard" is, nor what kind of database you're using, schema, models, any of that, and test cases must be attached with succinct, single-file scripts that have no external dependencies (read the link for more guidelines). As it stands it seems like you have a database with some corrupt unicode data which does not imply any bug in SQLAlchemy.
If you are having general issues debugging a complex program, please get help on the IRC channel or on the mailing list at https://groups.google.com/forum/#!forum/sqlalchemy.