conversation view not working anymore with MySQL Connector/Python

[Thilo Notz]

Hi,

first, thanks for making this great django app!

I just migrated to django 1.8.3 and postman 3.3.0 and have the following problem now. :

The "by conversation" view now produces an error when using MySQL. The "by message" view still works fine for all folders. So in general my configuration is alright. When I switched to sqlite for testing the problem goes away. However my setup is correct since the view by message and other functionality works. Maybe an incompatible query is generated.

See the output below:

Environment:

Request Method: GET
Request URL: http://localhost:8000/messaging/inbox/

Django Version: 1.8.3
Python Version: 3.4.0

Template error:
In template xxxx/postman/base_folder.html, error at line 17
   too many values to unpack (expected 2)



   14 : {% if invalid_page %}
   15 : <p>{% trans "Sorry, this page number is invalid." %}</p>
   16 : {% else %}
   17 :  {% if pm_messages %} 
   18 : {% block pm_by_modes %}<div id="pm_by_modes">



Traceback:
File "/usr/local/lib/python3.4/dist-packages/django/core/handlers/base.py" in get_response
  164.                 response = response.render()
File "/usr/local/lib/python3.4/dist-packages/django/template/response.py" in render
  158.             self.content = self.rendered_content
File "/usr/local/lib/python3.4/dist-packages/django/template/response.py" in rendered_content
  135.         content = template.render(context, self._request)
File "/usr/local/lib/python3.4/dist-packages/django/template/backends/django.py" in render
  74.         return self.template.render(context)
File "/usr/local/lib/python3.4/dist-packages/django/template/base.py" in render
  209.                     return self._render(context)
File "/usr/local/lib/python3.4/dist-packages/django/template/base.py" in _render
  201.         return self.nodelist.render(context)
File "/usr/local/lib/python3.4/dist-packages/django/template/base.py" in render
  903.                 bit = self.render_node(node, context)
File "/usr/local/lib/python3.4/dist-packages/django/template/debug.py" in render_node
  79.             return node.render(context)
File "/usr/local/lib/python3.4/dist-packages/django/template/loader_tags.py" in render
  135.         return compiled_parent._render(context)
File "/usr/local/lib/python3.4/dist-packages/django/template/base.py" in _render
  201.         return self.nodelist.render(context)
File "/usr/local/lib/python3.4/dist-packages/django/template/base.py" in render
  903.                 bit = self.render_node(node, context)
File "/usr/local/lib/python3.4/dist-packages/django/template/debug.py" in render_node
  79.             return node.render(context)
File "/usr/local/lib/python3.4/dist-packages/django/template/loader_tags.py" in render
  135.         return compiled_parent._render(context)
File "/usr/local/lib/python3.4/dist-packages/django/template/base.py" in _render
  201.         return self.nodelist.render(context)
File "/usr/local/lib/python3.4/dist-packages/django/template/base.py" in render
  903.                 bit = self.render_node(node, context)
File "/usr/local/lib/python3.4/dist-packages/django/template/debug.py" in render_node
  79.             return node.render(context)
File "/usr/local/lib/python3.4/dist-packages/django/template/loader_tags.py" in render
  135.         return compiled_parent._render(context)
File "/usr/local/lib/python3.4/dist-packages/django/template/base.py" in _render
  201.         return self.nodelist.render(context)
File "/usr/local/lib/python3.4/dist-packages/django/template/base.py" in render
  903.                 bit = self.render_node(node, context)
File "/usr/local/lib/python3.4/dist-packages/django/template/debug.py" in render_node
  79.             return node.render(context)
File "/usr/local/lib/python3.4/dist-packages/django/template/loader_tags.py" in render
  65.                 result = block.nodelist.render(context)
File "/usr/local/lib/python3.4/dist-packages/django/template/base.py" in render
  903.                 bit = self.render_node(node, context)
File "/usr/local/lib/python3.4/dist-packages/django/template/debug.py" in render_node
  79.             return node.render(context)
File "/usr/local/lib/python3.4/dist-packages/django/template/defaulttags.py" in render
  329.                 return nodelist.render(context)
File "/usr/local/lib/python3.4/dist-packages/django/template/base.py" in render
  903.                 bit = self.render_node(node, context)
File "/usr/local/lib/python3.4/dist-packages/django/template/debug.py" in render_node
  79.             return node.render(context)
File "/usr/local/lib/python3.4/dist-packages/django/template/defaulttags.py" in render
  328.             if match:
File "/usr/local/lib/python3.4/dist-packages/django/db/models/query.py" in __bool__
  166.         self._fetch_all()
File "/usr/local/lib/python3.4/dist-packages/django/db/models/query.py" in _fetch_all
  965.             self._result_cache = list(self.iterator())
File "/usr/local/lib/python3.4/dist-packages/django/db/models/query.py" in iterator
  238.         results = compiler.execute_sql()
File "/usr/local/lib/python3.4/dist-packages/django/db/models/sql/compiler.py" in execute_sql
  829.             sql, params = self.as_sql()
File "/usr/local/lib/python3.4/dist-packages/postman/query.py" in as_sql
  68.         extra_table, extra_params = self.union(self.query.pm_get_extra())
File "/usr/local/lib/python3.4/dist-packages/postman/query.py" in union
  85.             sql, params = qs.query.sql_with_params()
File "/usr/local/lib/python3.4/dist-packages/django/db/models/sql/query.py" in sql_with_params
  223.         return self.get_compiler(DEFAULT_DB_ALIAS).as_sql()
File "/usr/local/lib/python3.4/dist-packages/django/db/models/sql/compiler.py" in as_sql
  434.                 for _, (o_sql, o_params, _) in order_by:

Exception Type: ValueError at /messaging/inbox/
Exception Value: too many values to unpack (expected 2)

I debugged into the code and I found that the error first appears in line 143 in models.py where the order_by() is called.

[Patrick Samson]

My first guess is that it is not related to mysql, but to the django version. Please do these additional measures if possible:

• With the current 1.8.3, add print(order_by) just above line 434 in django/db/models/sql/compiler.py and report the output.

• For a temporary try, test the view with django 1.8 (https://www.djangoproject.com/download/1.8/tarball/) to see if it's different.

[Patrick Samson]

Thanks. So, it has to do with mysql !

The ['NULL'] is the bad guy.

I think it comes from order_by = self.connection.ops.force_no_ordering() a few lines above. But I don't understand the conversion from django\db\backends\mysql\operations.py which returns a [(None, ("NULL", [], False))].

May be you can investigate this point, if you feel comfortable enough, otherwise don't mind. Meanwhile I have to install mysql on my new PC.

[Thilo Notz]

I am not comfortable enough in that area as I am still new to django. I would leave it to an expert for now...

[Thilo Notz]

For your info the combination django 1.7.1 and postman 3.3.0 works.

[Patrick Samson]

You should have said that you were using the MySQL Connector/Python ('ENGINE': 'mysql.connector.django') instead of the native Django adaptor ('ENGINE': 'django.db.backends.mysql').

The current MySQL version (v2.1.2) doesn't support Django 1.8 but only 1.7. Quoted from https://docs.djangoproject.com/en/1.8/ref/databases/: "It may not support the most recent releases of Django.".

Unless you want to stay in Dj 1.7, you should move to the mysqlclient option, especially as "At the time of this writing, this is the recommended choice for using MySQL with Django.".

I will change the title of the issue to reflect this important fact.

[Thilo Notz]

You are so right. I wasn't aware of that. It works with mysqlclient. Thanks a lot!