1. Ian Bicking
  2. WebOb
Issue #21 resolved

Large file upload failure on Linux (Ubuntu)

William Groppe
created an issue

Hey Sergey,

After upgrading to Pyramid 1.2 the application we are building was throwing errors for any large upload. The problem is only occurring on Linux (Ubuntu 11.04). We tracked the error down to WebOb 1.1. We downgraded to WebOb 1.0.2 and the problem was resolved.

The problem is pretty easy to reproduce. Create a blank Pyramid application, and a simple upload function in views.py.

{{{

def upload(request): file = request.params["upload_file"] log.warning(file.filename) log.warning(file) return Response(body="success")

}}}

Now uploading any large (10mb+) file will fail with a DisconnectionError.

I'm going to try to attach a virtualbox instance that reproduces this problem. The login is will/will.

Thanks for your help,

Will

Comments (17)

  1. Sergey Schetinin

    One important piece of information missing from the report is the wsgi server you're using.

    Please try to trap that error and provide full traceback and environ contents.

    Thanks.

  2. William Groppe reporter

    The wsgi server was paster, here's a complete stack trace.

    Starting server in PID 1003.
    serving on 0.0.0.0:6543 view at http://127.0.0.1:6543
    Debug at: http://10.10.10.143:6543/_debug/view/1316145401
    ----------------------------------------
    Exception happened during processing of request from ('10.10.10.149', 49891)
    Traceback (most recent call last):
      File "/usr/local/lib/python2.7/dist-packages/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread
        self.finish_request(request, client_address)
      File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request
        self.RequestHandlerClass(request, client_address, self)
      File "/usr/lib/python2.7/SocketServer.py", line 639, in __init__
        self.handle()
      File "/usr/local/lib/python2.7/dist-packages/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 442, in handle
        BaseHTTPRequestHandler.handle(self)
      File "/usr/lib/python2.7/BaseHTTPServer.py", line 343, in handle
        self.handle_one_request()
      File "/usr/local/lib/python2.7/dist-packages/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 437, in handle_one_request
        self.wsgi_execute()
      File "/usr/local/lib/python2.7/dist-packages/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 287, in wsgi_execute
        self.wsgi_start_response)
      File "/usr/local/lib/python2.7/dist-packages/WebError-0.10.3-py2.7.egg/weberror/evalexception.py", line 235, in __call__
        return self.respond(environ, start_response)
      File "/usr/local/lib/python2.7/dist-packages/WebError-0.10.3-py2.7.egg/weberror/evalexception.py", line 475, in respond
        if self.xmlhttp_key in req.params:
      File "/usr/local/lib/python2.7/dist-packages/WebOb-1.1-py2.7.egg/webob/request.py", line 648, in params
        params = NestedMultiDict(self._str_GET, self._str_POST)
      File "/usr/local/lib/python2.7/dist-packages/WebOb-1.1-py2.7.egg/webob/request.py", line 558, in _str_POST
        keep_blank_values=True)
      File "/usr/lib/python2.7/cgi.py", line 508, in __init__
        self.read_multi(environ, keep_blank_values, strict_parsing)
      File "/usr/lib/python2.7/cgi.py", line 632, in read_multi
        environ, keep_blank_values, strict_parsing)
      File "/usr/lib/python2.7/cgi.py", line 510, in __init__
        self.read_single()
      File "/usr/lib/python2.7/cgi.py", line 647, in read_single
        self.read_lines()
      File "/usr/lib/python2.7/cgi.py", line 669, in read_lines
        self.read_lines_to_outerboundary()
      File "/usr/lib/python2.7/cgi.py", line 697, in read_lines_to_outerboundary
        line = self.fp.readline(1<<16)
      File "/usr/local/lib/python2.7/dist-packages/WebOb-1.1-py2.7.egg/webob/request.py", line 1292, in readline
        self._check_disconnect()
      File "/usr/local/lib/python2.7/dist-packages/WebOb-1.1-py2.7.egg/webob/request.py", line 1311, in _check_disconnect
        + "(%d more bytes were expected)" % self.remaining
    DisconnectionError: The client disconnected while sending the POST/PUT body (11358868 more bytes were expected)
    
    
  3. Sergey Schetinin

    My best guess so far is that the app you've described does throw an exception (but not the DisconnectionError, probably something related to Response not imported for example). Then weberror catches it and tries to work with the request that was already read raising the exception you see.

    To confirm that, try disabling the weberror middleware and tell me what happens.

  4. Log in to comment