Issue #1068 resolved

File upload crashes when using HTTPS

Anonymous created an issue

CherryPy 3.2.0 contains a fatal error which prevents file upload over HTTPS. The result is a crash in wsgiserver:

{{{ File "cherrypy/wsgiserver/init.py", line 995, in read assert n <= left, "recv(%d) returned %d bytes" % (left, n) AssertionError: recv(10) returned 16384 bytes }}}

Proof-of-Concept code is the same as in Ticket #783. The code works over HTTP. When you remove the comment signs from the server.ssl_certificate/server.ssl_private_key lines, it will use HTTPS and the code will fail.

Tested on Windows 7 with Activestate Python 2.6.6.18 (32bit) and Ubuntu 10.10 (32bit, default installation with Python 2.6.6).

Reported by shypike

Comments (18)

  1. Anonymous

    That seems to happen because an SSL transport might actually return more than you have asked for. Here's a potential fix. I'm using this in my deployment and it now works fine.

    diff -r 701f413a0d64 cherrypy/wsgiserver/wsgiserver2.py
    --- a/cherrypy/wsgiserver/wsgiserver2.py	Thu Nov 17 10:13:42 2011 -0500
    +++ b/cherrypy/wsgiserver/wsgiserver2.py	Tue Dec 13 12:19:07 2011 +0000
    @@ -1049,7 +1049,13 @@
                             buf.write(data)
                             del data  # explicit free
                             break
    -                    assert n <= left, "recv(%d) returned %d bytes" % (left, n)
    +                    if n > left:
    +                        # Could happen with SSL transport. Differ extra data read to the next call
    +                        buf.write(data[:left])
    +                        self._rbuf.write(data[left:])
    +                        del data
    +                        break
    +                    #assert n <= left, "recv(%d) returned %d bytes" % (left, n)
                         buf.write(data)
                         buf_len += n
                         del data  # explicit free
    
    
  2. cbf305

    I am also having this issue. I tried the patch suggested by faisyl and it seems to take care of the problem. I'm using Cherrypy 3.2.2.

  3. Anonymous

    I just ran into the exact same problem. It was working great with cherrypy 3.1.2, but on upgrading to Kubuntu 12.04, which ships with cherrypy 3.2.2, file uploads with HTTPS is a no-go.

  4. Jason R. Coombs

    In an attempt to recreate the bug, I started setting up an SSL CherryPy server, based on the PaperMill example in #783. Here's the example I created, which adds support for generating a test certificate as well.

    import shutil
    import os
    import tempfile
    import subprocess
    
    import cherrypy
    import OpenSSL.crypto
    
    cert_name = 'server.pem'
    
    def generate_cert():
        "Generate a certificate using openssl"
        if os.path.exists(cert_name):
            return
        cmd = [
            "openssl", "req", "-x509", "-nodes",
            "-days", "365",
            "-newkey", "rsa:1024",
            "-keyout", cert_name,
            "-out", cert_name,
            "-subj", "/C=EA/ST=Earth/L=Earth/CN=localhost",
        ]
        subprocess.check_call(cmd)
    
    def check_cert():
        generate_cert()
        with open(cert_name, 'rb') as certfile:
            cert_data = certfile.read()
        cert = OpenSSL.crypto.load_certificate(OpenSSL.crypto.FILETYPE_PEM, cert_data)
        cert.get_serial_number()
    
    class PaperMill(object):
        config = {
          'global': {
            'server.ssl_certificate': cert_name,
            'server.ssl_private_key': cert_name,
          }
        }
    
        def index(self):
            return """
                <html><head></head><body>
                <form id='upload_form' method='post' action='upload' enctype='multipart/form-data'>
                    Filename: <input id="filename_input" type="file" name="myFile"/>
                    <input type="submit" value="Upload!">
                </form>
            </body></html>"""
    
        index.exposed = True
    
        @cherrypy.expose
        def upload(self, myFile):
            out = """<html>
    
            <body>
                myFile length: %s<br>
                myFile filename: %s<br>
                myFile mime-type: %s<br>
            </body>
            </html>"""
    
            size = 0
    
            with tempfile.NamedTemporaryFile(delete=False) as tmp:
                while True:
                    # Read blocks of 8KB at a time
                    data = myFile.file.read(1024 * 8)
                    if not data:
                        break
                    tmp.write(data)
                    size += len(data)
    
            target = os.path.join(os.getcwd(), myFile.filename)
            shutil.move(tmp.name, target)
    
            return out % (size, myFile.filename, myFile.type)
    
    if __name__ == "__main__":
        generate_cert()
        cherrypy.quickstart(PaperMill(), config=PaperMill.config)
    

    However, I ran into issues in that running this code crashes Python. At least on Windows 8 x64 with Python 2.7 64-bit and OpenSSL 1.0.0j 64-bit and pyopenssl 0.13, if I invoke check_cert() from above, Python will fault in ntdll.dll. Because CherryPy makes this same call, I can't start a CherryPy server with SSL. It's quite possible that my build of pyopenssl doesn't work with 64-bit on Windows or that I didn't build it properly (I basically just easy_installed it with a proper dev environment configured). In any case, I'm stumped for now and will have to return to this later.

  5. J Lee

    We have encountered this problem as well. I applied the patch above and it worked perfectly. I was wondering with the release of 3.2.3, and that this has been outstanding for 2 years, why the patch has not been moved to source, and will it ever.

  6. Log in to comment