WindowsError [Error 6] The handle is invalid in HTTPServer.bind

Jason R. Coombs avatarJason R. Coombs created an issue

With the changes from [2038], I get sporadic errors when the CherryPy HTTPServer starts or reloads. Traceback follows.

Traceback (most recent call last):
  File "c:\users\jaraco\projects\public\cherrypy\cherrypy\process\servers.py", line 175, in _start_http_thread
    self.httpserver.start()
  File "c:\users\jaraco\projects\public\cherrypy\cherrypy\wsgiserver\__init__.py", line 1655, in start
    self.bind(af, socktype, proto)
  File "c:\users\jaraco\projects\public\cherrypy\cherrypy\wsgiserver\__init__.py", line 1685, in bind
    prevent_socket_inheritance(self.socket)
  File "c:\users\jaraco\projects\public\cherrypy\cherrypy\wsgiserver\__init__.py", line 1478, in prevent_socket_inheritance
    raise WinError()
WindowsError: [Error 6] The handle is invalid.

At first, I was only getting the errors during an autoreload, and I could easily disable that or restart the server. With a recent change to my server code, I found that the error occurred on startup (consistently).

After lots of trial and error, I found that the tipping point which causes the error to exhibit was a parameter to a function call in my CherryPy Root object (which is never called during startup). The mere presence of that innocuous parameter, regardless of name, triggers the behavior.

With the parameter in place, I found that other tweaks will prevent the WindowsError. I can run the code with pdb and it doesn't fail. I can run Python with -O and it doesn't fail. I can run the same code on a different PC, and it doesn't fail. I can make this tiny change, and it doesn't fail:

Index: cherrypy/wsgiserver/__init__.py
===================================================================
--- cherrypy/wsgiserver/__init__.py     (revision 2684)
+++ cherrypy/wsgiserver/__init__.py     (working copy)
@@ -1474,7 +1474,8 @@
     else:
         def prevent_socket_inheritance(sock):
             """Mark the given socket fd as non-inheritable (Windows)."""
-            if not windll.kernel32.SetHandleInformation(sock.fileno(), 1, 0):
+            fileno = sock.fileno()
+            if not windll.kernel32.SetHandleInformation(fileno, 1, 0):
                 raise WinError()
 else:
     def prevent_socket_inheritance(sock):

Note that it's not strictly a timing issue. I tried putting a sleep call before the call to SetHandleInformation, but that did not seem to have any effect (other than to slow down the bind operation).

I suspect this may be related to the TODO in #856:

Note that if another thread spawns a process in between the socket creation and the call to prevent_socket_inheritance, we're still out of luck. A solution for Windows is proposed in http://bugs.python.org/issue3006, by directly coding it in C, when holding the Global Interpreter Lock. 

Does "out of luck" mean the handle could be invalid?

For now, I plan to apply the tiny patch above, and see if the problems re-emerge using that code. Any suggestions are more than welcome.

Comments (8)

  1. Anonymous

    For me «fileno» patch worked not, and «python -O» was unstable. But what really did helped was launching «python.exe my_script.py» '''in separate process window''', not inside shells like «cmd.exe» or «Far.exe». Maybe it is somehow related to ''«another thread spawns a process»'' mentioned above, because that is what happens inside shells.

  2. Anonymous

    Now «separate process window» workaround became unstable too. Next thing that worked was no-opping that «prevent_socket_inheritance»:

    # C:\Python26\Lib\site-packages\CherryPy-3.2.0-py2.6.egg\cherrypy\wsgiserver\__init__.py
    # after last «def prevent_socket_inheritance»:
    def prevent_socket_inheritance(sock):
        pass
    

    Note: this is for local development on Windows only, not for production.

  3. Ben Hoyt

    We had this same intermittent issue. It took us a while to find it, but we've found why it happens, and have the solution.

    It happens on 64-bit Windows because prevent_socket_inheritance() doesn't specify the types in the ctypes call to SetHandleInformation(), which is defined in the Windows documentation as:

    BOOL WINAPI SetHandleInformation(HANDLE hObject, DWORD dwMask, DWORD dwFlags);
    

    A HANDLE is a pointer type (void*), so on 32-bit Windows it's a 32-bit value, but on 64-bit Windows, it's a 64-bit value. ctypes defaults to passing integers like sock.fileno() as 32-bit, so sometimes on 64-bit Windows (depending on what else was on the stack, I guess) the handle value would be corrupt, and cause this invalid handle exception.

    You can fix this in CherryPy by changing:

    def prevent_socket_inheritance(sock):
        """Mark the given socket fd as non-inheritable (Windows)."""
        if not windll.kernel32.SetHandleInformation(sock.fileno(), 1, 0):
            raise WinError()
    

    to specify the arguments explictly using wintypes:

    from ctypes import wintypes                    # need to import wintypes as well
    
    _SetHandleInformation = windll.kernel32.SetHandleInformation
    _SetHandleInformation.argtypes = [wintypes.HANDLE, wintypes.DWORD, wintypes.DWORD]
    _SetHandleInformation.restype = wintypes.BOOL  # don't need this, but might as well
    
    def prevent_socket_inheritance(sock):
        """Mark the given socket fd as non-inheritable (Windows)."""
        if not _SetHandleInformation(sock.fileno(), 1, 0):
            raise WinError()
    

    FYI, we use the CherryPy server with web.py to power Oyster.com, and we're running 64-bit Windows on x86.

  4. Jason R. Coombs

    Great catch. Having worked with ctypes, I'm surprised I didn't catch that nuance. The fix looks like exactly the right thing. I've put together a patch and will push it soon.

  5. Log in to comment
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.