Issue #1244 resolved

_cpcompat_subprocess on windows not fully backwards compatible

cdent
created an issue

*Apologies in advance for a bug report heavy on narrative and light on code. I don't have access to windows, so I can't replicate a problem reported in my project, so I've traced it by reading through code and commit logs.

4aae6961428888611471f3891481ed760e1210be copied in Python 2.7.3 subprocess.py. This seems to work well for most situations except on the windows platform with older versions of Python. On windows, subprocess.py imports the _subprocess C module and then imports symbols from it. Some of those symbols (notably CREATE_NEW_PROCESS_GROUP) are only present in newer _subprocess so on older Python versions on windows the import will fail.

As far as I can tell this commit from April 2010 introduced the symbol for the first time.

Including the C module in CherryPy is onerous so either the promised compatibility back to much older versions (on windows) is void or some modification is needed.

As far as I can tell CREATE_NEW_PROCESS_GROUP is not used in cherrypy itself so perhaps just hork that out from lines 449 and 454?

The same issue may be present with CREATE_NEW_CONSOLE.

Comments (11)

  1. Jason R. Coombs

    Good report. Thanks for the investigation and detailed report. I'm inclined to try any hack to keep this hack going. I expect Windows systems to have an easier time upgrading Python versions, so I don't want to spend too much time on this issue, but it's certainly worth cheap fixes.

  2. mar10

    Personally I would not rank this 'minor' since it is currently stopping us from releasing.

    3.2.2 seems to have another problem with SSL uploads which we try to solve by upgrading to 3.2.4. Sadly upgrading from Py26 to Py27 is currently not an option, due to some of our dependencies. So we could deliver with a patched 3.2.2, but this is something I'd like to avoid.

  3. Sylvain Hellegouarch

    I agree this shan't be considered as minor. My concern though is whether the remedy is better than the initial solution.

    Mart10: Did you manage to get this working fine with 2.6? If so, would you have a patch? I believe this would need to be tested against all version that we support on various platforms.

  4. mar10

    A colleague has patched and tested 3.2.4 with two changes:

    _cpcompactsubprocess.py at line 448

    if mswindows:
        from _subprocess import CREATE_NEW_CONSOLE, \ # PATCH: Removed CREATE_NEW_PROCESS_GROUP here
                                 STD_INPUT_HANDLE, STD_OUTPUT_HANDLE, \
                                 STD_ERROR_HANDLE, SW_HIDE, \
                                 STARTF_USESTDHANDLES, STARTF_USESHOWWINDOW
        try:
            from _subprocess import CREATE_NEW_PROCESS_GROUP
        except:
            _subprocess.CREATE_NEW_PROCESS_GROUP = 512
    

    and wsgiserver2.py at line 71 (this should probably go to wsgiserver3 as well?)

    __all__ = ['HTTPRequest', 'HTTPConnection', 'HTTPServer',
               'SizeCheckWrapper', 'KnownLengthRFile', 'ChunkedRFile',
               'CP_fileobject',
               'MaxSizeExceeded', 'NoSSLError', 'FatalSSLAlert',
               'WorkerThread', 'ThreadPool', 'SSLAdapter',
               'CherryPyWSGIServer',
               'Gateway', 'WSGIGateway', 'WSGIGateway_10', 'WSGIGateway_u0',
               'WSGIPathInfoDispatcher', 'get_ssl_adapter_class', 
               "socket_errors_to_ignore"] # PATCH: added this export
    
  5. Log in to comment