Commits

Victor Stinner committed c4e7f27

close-on-exec PEP

Comments (0)

Files changed (1)

python/pep_cloexec.rst

+PEP: xxx
+Title: Add cloexec argument to open(), socket and other functions
+Version: $Revision$
+Last-Modified: $Date$
+Author: Victor Stinner <victor.stinner@gmail.com>
+Status: Draft
+Type: Standards Track
+Content-Type: text/x-rst
+Created: 10-January-2013
+Python-Version: 3.4
+
+
+Abstract
+========
+
+This PEP proposes options to configure how the close-on-exec is set on file
+descriptors created by Python. The value can be changed globally, or explicitly
+when creating a new file descriptors.
+
+
+Rationale
+=========
+
+Rationale:
+
+ * Third party modules may call ``fork()`` + ``exec()``
+ * ``SocketServer``: "address already in use issue" error
+ * ``fcntl()`` with ``FD_CLOEXEC`` flag is not atomic if another thread
+   calls ``fork()``
+ * Leak of file descriptors: security vulnerability
+ * ``os.O_CLOEXEC`` and ``os.pipe2()`` were added to Python 3.3
+
+New functions:
+
+ * ``sys.getdefaultcloexec() -> bool``
+ * ``sys.setdefaultcloexec(cloexec: bool)``
+ * ``os.set_cloexec()``
+
+Add optional ``cloexec`` argument to:
+
+ * ``open()``: ``os.fdopen()`` is indirectly modified
+ * ``socket.socket()``, ``socket.socketpair()``
+ * ``os.dup()``, ``os.dup2()``
+ * ``os.pipe()``
+ * Maybe also: ``os.open()``, ``os.openpty()``
+ * Should be modified later:
+
+   * ``select.devpoll()``
+   * ``select.poll()``
+   * ``select.epoll()``
+   * ``select.kqueue()``
+   * ``socket.socket.recvmsg()``: use ``MSG_CMSG_CLOEXEC``, or ``os.set_cloexec()``
+
+If the ``cloexec`` argument is not set, ``sys.getdefaultcloexec()`` will be used.
+
+Impacted modules:
+
+ * ``socketserver``
+ * ``xmlrpc.server``
+ * ``subprocess``
+ * ``tempfile``
+ * Maybe: ``signal``, ``threading``
+
+File descriptors created by third party modules do not respect
+``sys.setdefaultcloexec()`` if they are not created by functions of the Python
+standard library.
+
+
+Functions
+=========
+
+os.set_cloexec(fd, cloexec)
+---------------------------
+
+Best-effort by definition::
+
+    if os.name == 'nt':
+        def set_cloexec(fd, cloexec):
+            SetHandleInformation(fd, HANDLE_FLAG_INHERIT, int(cloexec))
+    if ioctl is not None and hasattr('FIOCLEX', ioctl):
+        def set_cloexec(fd, cloexec):
+            if cloexec:
+                ioctl.ioctl(fd, ioctl.FIOCLEX)
+            else:
+                ioctl.ioctl(fd, ioctl.FIONCLEX)
+    elif fnctl is not None:
+        def set_cloexec(fd, cloexec):
+            flags = fcntl.fcntl(fd, fcntl.F_GETFD)
+            if cloexec:
+                flags |= FD_CLOEXEC
+            else:
+                flags &= ~FD_CLOEXEC
+            fcntl.fcntl(fd, fcntl.F_SETFD, flags)
+    else:
+        def set_cloexec(fd, cloexec):
+            raise NotImplementedError("close-on-exec flag is not supported on your platform")
+
+open()
+------
+
+ * ``open()`` with ``O_CLOEXEC flag`` [atomic]
+ * Windows: ``open()`` with ``O_NOINHERIT`` flag [atomic]
+ * ``open()`` + ``os.set_cloexec(fd, True)`` [best-effort]
+
+socket.socket()
+---------------
+
+ * ``socket()`` with ``SOCK_CLOEXEC`` flag [atomic]
+ * ``socket()`` + ``os.set_cloexec(fd, True)`` [best-effort]
+
+XXX what is SO_CLOEXEC??? XXX
+
+os.dup()
+--------
+
+ * ``dup3()`` with ``O_CLOEXEC`` flag [atomic]
+ * ``fcntl(fd, F_DUPFD_CLOEXEC)`` [atomic]
+ * ``dup()`` + ``os.set_cloexec(fd, True)`` [best-effort]
+
+os.dup2()
+-------------------
+
+ * ``dup3()`` with ``O_CLOEXEC`` flag [atomic]
+ * ``dup2()`` + ``os.set_cloexec(fd, True)`` [best-effort]
+
+os.pipe()
+---------
+
+ * Windows: ``_pipe()`` with ``O_NOINHERIT`` flag [atomic]
+ * ``pipe2()`` with ``O_CLOEXEC`` flag [atomic]
+ * ``pipe()`` + ``os.set_cloexec(fd, True)`` [best-effort]
+
+
+Backward compatibility
+======================
+
+There is no backward incompatible change. The default behaviour is unchanged:
+close-on-exec is not set by default.
+
+
+Alternatives
+============
+
+open(): add "e" flag to mode
+----------------------------
+
+Since its version 2.7, the GNU libc supports "e" flag for fopen(). It uses
+``O_CLOEXEC`` if available, or use ``fcntl(fd, F_SETFD, flags | FD_CLOEXEC)``.
+
+This API doesn't allow to disable explictly close-on-exec flag if it was
+enabled globally with sys.setdefaultcloexec().
+
+
+Appendix: Operating system support
+==================================
+
+Windows
+-------
+
+Windows has an ``O_NOINHERIT`` flag: "Don't inherit in child processes".  For
+example, it is supported by ``open()`` and ``_pipe()``. The value of the flag
+can be modified on a file descriptor using:
+``SetHandleInformation(fd, HANDLE_FLAG_INHERIT, 1)``.
+
+fcntl
+-----
+
+ * ``fcntl(fd, F_GETFD)``
+ * ``fcntl(fd, F_SETFD, flags | FD_CLOEXEC)``
+
+Available on:
+
+ * AIX 4.2
+ * Digital UNIX 4.0
+ * FreeBSD 8.2
+ * HP-UX 11
+ * IRIX 6.5
+ * Linux 2.0
+ * Mac OS X 10.8
+ * OpenBSD 4.9
+ * Solaris 8
+ * SunOS 4.1.1_U1
+ * Unicos 9.0
+
+Note: OpenBSD older 5.2 does not close the file descriptor if fork() is used
+before exec(), but it works correctly if exec() is called without fork().
+
+ioctl
+-----
+
+ * ``ioctl(fd, FIOCLEX, 0);`` sets close-on-exec flag
+ * ``ioctl(fd, FIONCLEX, 0);`` unsets close-on-exec flag
+
+Availability: Linux, Mac OS X, QNX, NetBSD, OpenBSD, FreeBSD.
+
+
+Atomic flags
+------------
+
+New flags:
+
+ * ``O_CLOEXEC``: available on Linux (2.6.23+), FreeBSD (8.3+). Part of POSIX.1-2008.
+ * ``socket()``: ``SOCK_CLOEXEC`` flag, available on Linux 2.6.27+
+ * ``fcntl()``: ``F_DUPFD_CLOEXEC`` flag, available on Linux ???
+ * ``recvmsg()``: ``MSG_CMSG_CLOEXEC``, available on Linux ???
+
+On Linux older than 2.6.23, ``O_CLOEXEC`` flag is simply ignored. So we have to
+check that the flag is supported by calling ``fcntl()``. If it doesn't work, we
+have to set the flag using ``fcntl()``.
+
+XXX what is the behaviour on Linux older than 2.6.27 with SOCK_CLOEXEC? XXX
+
+New functions:
+
+ * ``dup3()``: available on Linux ???
+ * ``pipe2()``: available on Linux ???
+ * ``accept4()``: available on Linux 2.6.28
+
+If ``accept4()`` is called on Linux older than 2.6.28, ``accept4()`` returns
+``-1`` (fail) and errno is set to ``ENOSYS``.
+
+
+Links
+=====
+
+Links:
+
+ * `Secure File Descriptor Handling
+   <http://udrepper.livejournal.com/20407.html>`_
+ * `win32_support.py of the Tornado project
+   <https://bitbucket.org/pvl/gaeseries-tornado/src/c2671cea1842/tornado/win32_support.py>`_:
+   emulate fcntl(fd, F_SETFD, FD_CLOEXEC) using
+   ``SetHandleInformation(fd, HANDLE_FLAG_INHERIT, 1)``
+
+Python issues:
+
+ * `open() does not able to set flags, such as O_CLOEXEC
+   <http://bugs.python.org/issue12105>`_
+ * `Add "e" mode to open(): close-and-exec (O_CLOEXEC) / O_NOINHERIT
+   <http://bugs.python.org/issue16850>`_
+ * `TCP listening sockets created without FD_CLOEXEC flag
+   <http://bugs.python.org/issue12107>`_
+ * `Use O_CLOEXEC in the tempfile module
+   <http://bugs.python.org/issue16860>`_
+ * `Support accept4() for atomic setting of flags at socket creation
+   <http://bugs.python.org/issue10115>`_
+
+Ruby:
+
+ * `Set FD_CLOEXEC for all fds (except 0, 1, 2)
+   <http://bugs.ruby-lang.org/issues/5041>`_
+ * `O_CLOEXEC flag missing for Kernel::open
+   <http://bugs.ruby-lang.org/issues/1291>`_:
+   `commit reverted <O_CLOEXEC flag missing for Kernel::open>`_ later
+