Issue #1200 resolved

IndexError, KeyError, LookupError, UnicodeError exceptions aren't propagated from generators

Jim Paris
created an issue
  • With this code:
import cherrypy
class HelloWorld:
    @cherrypy.expose
    def one(self):
        def content():
            yield "One"
        return content()
    @cherrypy.expose
    def two(self):
        def content():
            yield "Two"
            raise ValueError()
        return content()
    @cherrypy.expose
    def three(self):
        def content():
            yield "Three"
            raise KeyError()
        return content()
cherrypy.quickstart(HelloWorld())
  • Requesting http://localhost:8080/one works as expected:
One
  • Requesting http://localhost:8080/two shows the ValueError as expected:
500 Internal Server Error

The server encountered an unexpected condition which prevented it from fulfilling the request.

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 656, in respond
    response.body = self.handler()
  File "/usr/lib/python2.7/dist-packages/cherrypy/lib/encoding.py", line 228, in __call__
    ct.params['charset'] = self.find_acceptable_charset()
  File "/usr/lib/python2.7/dist-packages/cherrypy/lib/encoding.py", line 142, in find_acceptable_charset
    if encoder(self.default_encoding):
  File "/usr/lib/python2.7/dist-packages/cherrypy/lib/encoding.py", line 86, in encode_string
    for chunk in self.body:
  File "test.py", line 13, in content
    raise ValueError()
ValueError
  • But requesting http://localhost:8080/three gives a cryptic and incorrect error about encoding that gives no information about the source of the exception:
500 Internal Server Error

Response body could not be encoded with 'utf-8'.

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 656, in respond
    response.body = self.handler()
  File "/usr/lib/python2.7/dist-packages/cherrypy/lib/encoding.py", line 228, in __call__
    ct.params['charset'] = self.find_acceptable_charset()
  File "/usr/lib/python2.7/dist-packages/cherrypy/lib/encoding.py", line 145, in find_acceptable_charset
    raise cherrypy.HTTPError(500, self.failmsg % self.default_encoding)
HTTPError: (500, "Response body could not be encoded with 'utf-8'.")

This is because ResponseEncoder.encode_string catches the KeyError (as LookupError) and discards the exception info. Patch below and attached (it just moves the try/except around the actual encoding).

diff -urN cherrypy3-3.2.2-orig/cherrypy/lib/encoding.py cherrypy3-3.2.2/cherrypy/lib/encoding.py
--- cherrypy3-3.2.2-orig/cherrypy/lib/encoding.py   2013-01-01 21:53:16.602098603 -0500
+++ cherrypy3-3.2.2/cherrypy/lib/encoding.py    2013-01-01 21:53:47.549896273 -0500
@@ -81,17 +81,16 @@
             return False
         self.attempted_charsets.add(encoding)

-        try:
-            body = []
-            for chunk in self.body:
-                if isinstance(chunk, unicodestr):
+        body = []
+        for chunk in self.body:
+            if isinstance(chunk, unicodestr):
+                try:
                     chunk = chunk.encode(encoding, self.errors)
-                body.append(chunk)
-            self.body = body
-        except (LookupError, UnicodeError):
-            return False
-        else:
-            return True
+                except (LookupError, UnicodeError):
+                    return False
+            body.append(chunk)
+        self.body = body
+        return True

     def find_acceptable_charset(self):
         request = cherrypy.serving.request

Comments (4)

  1. Joel Rivera

    Bugfix. The IndexError, KeyError, LookupError, UnicodeError exceptions were not propagated from generators, the solution was to limit the scope of the try/except block, even if the try/catch block is inside a for loop because of the premise that the exceptions are not that heavy when the except block is not executed in the majority of the cases (which is the expected behavior at that particular part).

    Applying the suggested patch of @jimparis.

    Closes issue #1200.

    → <<cset a70206dec99b>>

  2. Log in to comment