Commits

Vinay Sajip committed 0a0efb7 Merge

Merged upstream changes.

Comments (0)

Files changed (113)

 a4f75773c0060cee38b0bb651a7aba6f56b0e996 v3.1.3
 32fcb9e94985cb19ce37ba9543f091c0dbe9d7dd v3.1.4rc1
 c918ec9f3a76d6afedfbb5d455004de880443a3d v3.1.4
+ee26aca3219cf4bb0b93352e83edcc9cb28c7802 v3.1.5rc1
 b37b7834757492d009b99cf0ca4d42d2153d7fac v3.2a1
 56d4373cecb73c8b45126ba7b045b3c7b3f94b0b v3.2a2
 da012d9a2c23d144e399d2e01a55b8a83ad94573 v3.2a3
 ac1f7e5c05104d557d5acd922e95625ba5d1fe10 v3.2.1
 c860feaa348d663e598986894ee4680480577e15 v3.2.2rc1
 137e45f15c0bd262c9ad4c032d97425bc0589456 v3.2.2
+7085403daf439adb3f9e70ef13f6bedb1c447376 v3.2.3rc1
    * Stefan Franke
    * Jim Fulton
    * Peter Funk
+   * Ethan Furman
    * Lele Gaifax
    * Matthew Gallagher
    * Gabriel Genellina

Doc/c-api/buffer.rst

 
 .. sectionauthor:: Greg Stein <gstein@lyra.org>
 .. sectionauthor:: Benjamin Peterson
+.. sectionauthor:: Stefan Krah
 
 
 .. index::
 
 While each of these types have their own semantics, they share the common
 characteristic of being backed by a possibly large memory buffer.  It is
-then desireable, in some situations, to access that buffer directly and
+then desirable, in some situations, to access that buffer directly and
 without intermediate copying.
 
 Python provides such a facility at the C level in the form of the *buffer
 resource leaks.
 
 
-The buffer structure
-====================
+.. _buffer-structure:
+
+Buffer structure
+================
 
 Buffer structures (or simply "buffers") are useful as a way to expose the
 binary data from another object to the Python programmer.  They can also be
 
 .. c:type:: Py_buffer
 
-   .. c:member:: void *buf
+   .. c:member:: void \*obj
 
-      A pointer to the start of the memory for the object.
+      A new reference to the exporting object or *NULL*. The reference is owned
+      by the consumer and automatically decremented and set to *NULL* by
+      :c:func:`PyBuffer_Release`.
+
+      For temporary buffers that are wrapped by :c:func:`PyMemoryView_FromBuffer`
+      this field must be *NULL*.
+
+   .. c:member:: void \*buf
+
+      A pointer to the start of the logical structure described by the buffer
+      fields. This can be any location within the underlying physical memory
+      block of the exporter. For example, with negative :c:member:`~Py_buffer.strides`
+      the value may point to the end of the memory block.
+
+      For contiguous arrays, the value points to the beginning of the memory
+      block.
 
    .. c:member:: Py_ssize_t len
-      :noindex:
 
-      The total length of the memory in bytes.
+      ``product(shape) * itemsize``. For contiguous arrays, this is the length
+      of the underlying memory block. For non-contiguous arrays, it is the length
+      that the logical structure would have if it were copied to a contiguous
+      representation.
+
+      Accessing ``((char *)buf)[0] up to ((char *)buf)[len-1]`` is only valid
+      if the buffer has been obtained by a request that guarantees contiguity. In
+      most cases such a request will be :c:macro:`PyBUF_SIMPLE` or :c:macro:`PyBUF_WRITABLE`.
 
    .. c:member:: int readonly
 
-      An indicator of whether the buffer is read only.
+      An indicator of whether the buffer is read-only. This field is controlled
+      by the :c:macro:`PyBUF_WRITABLE` flag.
 
-   .. c:member:: const char *format
-      :noindex:
+   .. c:member:: Py_ssize_t itemsize
 
-      A *NULL* terminated string in :mod:`struct` module style syntax giving
-      the contents of the elements available through the buffer.  If this is
-      *NULL*, ``"B"`` (unsigned bytes) is assumed.
+      Item size in bytes of a single element. Same as the value of :func:`struct.calcsize`
+      called on non-NULL :c:member:`~Py_buffer.format` values.
+
+      Important exception: If a consumer requests a buffer without the
+      :c:macro:`PyBUF_FORMAT` flag, :c:member:`~Py_Buffer.format` will
+      be set to  *NULL*,  but :c:member:`~Py_buffer.itemsize` still has
+      the value for the original format.
+
+      If :c:member:`~Py_Buffer.shape` is present, the equality
+      ``product(shape) * itemsize == len`` still holds and the consumer
+      can use :c:member:`~Py_buffer.itemsize` to navigate the buffer.
+
+      If :c:member:`~Py_Buffer.shape` is *NULL* as a result of a :c:macro:`PyBUF_SIMPLE`
+      or a :c:macro:`PyBUF_WRITABLE` request, the consumer must disregard
+      :c:member:`~Py_buffer.itemsize` and assume ``itemsize == 1``.
+
+   .. c:member:: const char \*format
+
+      A *NUL* terminated string in :mod:`struct` module style syntax describing
+      the contents of a single item. If this is *NULL*, ``"B"`` (unsigned bytes)
+      is assumed.
+
+      This field is controlled by the :c:macro:`PyBUF_FORMAT` flag.
 
    .. c:member:: int ndim
 
-      The number of dimensions the memory represents as a multi-dimensional
-      array.  If it is 0, :c:data:`strides` and :c:data:`suboffsets` must be
-      *NULL*.
+      The number of dimensions the memory represents as an n-dimensional array.
+      If it is 0, :c:member:`~Py_Buffer.buf` points to a single item representing
+      a scalar. In this case, :c:member:`~Py_buffer.shape`, :c:member:`~Py_buffer.strides`
+      and :c:member:`~Py_buffer.suboffsets` MUST be *NULL*.
 
-   .. c:member:: Py_ssize_t *shape
+      The macro :c:macro:`PyBUF_MAX_NDIM` limits the maximum number of dimensions
+      to 64. Exporters MUST respect this limit, consumers of multi-dimensional
+      buffers SHOULD be able to handle up to :c:macro:`PyBUF_MAX_NDIM` dimensions.
 
-      An array of :c:type:`Py_ssize_t`\s the length of :c:data:`ndim` giving the
-      shape of the memory as a multi-dimensional array.  Note that
-      ``((*shape)[0] * ... * (*shape)[ndims-1])*itemsize`` should be equal to
-      :c:data:`len`.
+   .. c:member:: Py_ssize_t \*shape
 
-   .. c:member:: Py_ssize_t *strides
+      An array of :c:type:`Py_ssize_t` of length :c:member:`~Py_buffer.ndim`
+      indicating the shape of the memory as an n-dimensional array. Note that
+      ``shape[0] * ... * shape[ndim-1] * itemsize`` MUST be equal to
+      :c:member:`~Py_buffer.len`.
 
-      An array of :c:type:`Py_ssize_t`\s the length of :c:data:`ndim` giving the
-      number of bytes to skip to get to a new element in each dimension.
+      Shape values are restricted to ``shape[n] >= 0``. The case
+      ``shape[n] == 0`` requires special attention. See `complex arrays`_
+      for further information.
 
-   .. c:member:: Py_ssize_t *suboffsets
+      The shape array is read-only for the consumer.
 
-      An array of :c:type:`Py_ssize_t`\s the length of :c:data:`ndim`.  If these
-      suboffset numbers are greater than or equal to 0, then the value stored
-      along the indicated dimension is a pointer and the suboffset value
-      dictates how many bytes to add to the pointer after de-referencing. A
-      suboffset value that it negative indicates that no de-referencing should
-      occur (striding in a contiguous memory block).
+   .. c:member:: Py_ssize_t \*strides
 
-      Here is a function that returns a pointer to the element in an N-D array
-      pointed to by an N-dimensional index when there are both non-NULL strides
-      and suboffsets::
+      An array of :c:type:`Py_ssize_t` of length :c:member:`~Py_buffer.ndim`
+      giving the number of bytes to skip to get to a new element in each
+      dimension.
 
-          void *get_item_pointer(int ndim, void *buf, Py_ssize_t *strides,
-              Py_ssize_t *suboffsets, Py_ssize_t *indices) {
-              char *pointer = (char*)buf;
-              int i;
-              for (i = 0; i < ndim; i++) {
-                  pointer += strides[i] * indices[i];
-                  if (suboffsets[i] >=0 ) {
-                      pointer = *((char**)pointer) + suboffsets[i];
-                  }
-              }
-              return (void*)pointer;
-           }
+      Stride values can be any integer. For regular arrays, strides are
+      usually positive, but a consumer MUST be able to handle the case
+      ``strides[n] <= 0``. See `complex arrays`_ for further information.
 
+      The strides array is read-only for the consumer.
 
-   .. c:member:: Py_ssize_t itemsize
+   .. c:member:: Py_ssize_t \*suboffsets
 
-      This is a storage for the itemsize (in bytes) of each element of the
-      shared memory. It is technically un-necessary as it can be obtained
-      using :c:func:`PyBuffer_SizeFromFormat`, however an exporter may know
-      this information without parsing the format string and it is necessary
-      to know the itemsize for proper interpretation of striding. Therefore,
-      storing it is more convenient and faster.
+      An array of :c:type:`Py_ssize_t` of length :c:member:`~Py_buffer.ndim`.
+      If ``suboffsets[n] >= 0``, the values stored along the nth dimension are
+      pointers and the suboffset value dictates how many bytes to add to each
+      pointer after de-referencing. A suboffset value that is negative
+      indicates that no de-referencing should occur (striding in a contiguous
+      memory block).
 
-   .. c:member:: void *internal
+      This type of array representation is used by the Python Imaging Library
+      (PIL). See `complex arrays`_ for further information how to access elements
+      of such an array.
+
+      The suboffsets array is read-only for the consumer.
+
+   .. c:member:: void \*internal
 
       This is for use internally by the exporting object. For example, this
       might be re-cast as an integer by the exporter and used to store flags
       about whether or not the shape, strides, and suboffsets arrays must be
-      freed when the buffer is released. The consumer should never alter this
+      freed when the buffer is released. The consumer MUST NOT alter this
       value.
 
+.. _buffer-request-types:
+
+Buffer request types
+====================
+
+Buffers are usually obtained by sending a buffer request to an exporting
+object via :c:func:`PyObject_GetBuffer`. Since the complexity of the logical
+structure of the memory can vary drastically, the consumer uses the *flags*
+argument to specify the exact buffer type it can handle.
+
+All :c:data:`Py_buffer` fields are unambiguously defined by the request
+type.
+
+request-independent fields
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+The following fields are not influenced by *flags* and must always be filled in
+with the correct values: :c:member:`~Py_buffer.obj`, :c:member:`~Py_buffer.buf`,
+:c:member:`~Py_buffer.len`, :c:member:`~Py_buffer.itemsize`, :c:member:`~Py_buffer.ndim`.
+
+
+readonly, format
+~~~~~~~~~~~~~~~~
+
+   .. c:macro:: PyBUF_WRITABLE
+
+      Controls the :c:member:`~Py_buffer.readonly` field. If set, the exporter
+      MUST provide a writable buffer or else report failure. Otherwise, the
+      exporter MAY provide either a read-only or writable buffer, but the choice
+      MUST be consistent for all consumers.
+
+   .. c:macro:: PyBUF_FORMAT
+
+      Controls the :c:member:`~Py_buffer.format` field. If set, this field MUST
+      be filled in correctly. Otherwise, this field MUST be *NULL*.
+
+
+:c:macro:`PyBUF_WRITABLE` can be \|'d to any of the flags in the next section.
+Since :c:macro:`PyBUF_SIMPLE` is defined as 0, :c:macro:`PyBUF_WRITABLE`
+can be used as a stand-alone flag to request a simple writable buffer.
+
+:c:macro:`PyBUF_FORMAT` can be \|'d to any of the flags except :c:macro:`PyBUF_SIMPLE`.
+The latter already implies format ``B`` (unsigned bytes).
+
+
+shape, strides, suboffsets
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The flags that control the logical structure of the memory are listed
+in decreasing order of complexity. Note that each flag contains all bits
+of the flags below it.
+
+
++-----------------------------+-------+---------+------------+
+|  Request                    | shape | strides | suboffsets |
++=============================+=======+=========+============+
+| .. c:macro:: PyBUF_INDIRECT |  yes  |   yes   | if needed  |
++-----------------------------+-------+---------+------------+
+| .. c:macro:: PyBUF_STRIDES  |  yes  |   yes   |    NULL    |
++-----------------------------+-------+---------+------------+
+| .. c:macro:: PyBUF_ND       |  yes  |   NULL  |    NULL    |
++-----------------------------+-------+---------+------------+
+| .. c:macro:: PyBUF_SIMPLE   |  NULL |   NULL  |    NULL    |
++-----------------------------+-------+---------+------------+
+
+
+contiguity requests
+~~~~~~~~~~~~~~~~~~~
+
+C or Fortran contiguity can be explicitly requested, with and without stride
+information. Without stride information, the buffer must be C-contiguous.
+
++-----------------------------------+-------+---------+------------+--------+
+|  Request                          | shape | strides | suboffsets | contig |
++===================================+=======+=========+============+========+
+| .. c:macro:: PyBUF_C_CONTIGUOUS   |  yes  |   yes   |    NULL    |   C    |
++-----------------------------------+-------+---------+------------+--------+
+| .. c:macro:: PyBUF_F_CONTIGUOUS   |  yes  |   yes   |    NULL    |   F    |
++-----------------------------------+-------+---------+------------+--------+
+| .. c:macro:: PyBUF_ANY_CONTIGUOUS |  yes  |   yes   |    NULL    | C or F |
++-----------------------------------+-------+---------+------------+--------+
+| .. c:macro:: PyBUF_ND             |  yes  |   NULL  |    NULL    |   C    |
++-----------------------------------+-------+---------+------------+--------+
+
+
+compound requests
+~~~~~~~~~~~~~~~~~
+
+All possible requests are fully defined by some combination of the flags in
+the previous section. For convenience, the buffer protocol provides frequently
+used combinations as single flags.
+
+In the following table *U* stands for undefined contiguity. The consumer would
+have to call :c:func:`PyBuffer_IsContiguous` to determine contiguity.
+
+
+
++-------------------------------+-------+---------+------------+--------+----------+--------+
+|  Request                      | shape | strides | suboffsets | contig | readonly | format |
++===============================+=======+=========+============+========+==========+========+
+| .. c:macro:: PyBUF_FULL       |  yes  |   yes   | if needed  |   U    |     0    |  yes   |
++-------------------------------+-------+---------+------------+--------+----------+--------+
+| .. c:macro:: PyBUF_FULL_RO    |  yes  |   yes   | if needed  |   U    |  1 or 0  |  yes   |
++-------------------------------+-------+---------+------------+--------+----------+--------+
+| .. c:macro:: PyBUF_RECORDS    |  yes  |   yes   |    NULL    |   U    |     0    |  yes   |
++-------------------------------+-------+---------+------------+--------+----------+--------+
+| .. c:macro:: PyBUF_RECORDS_RO |  yes  |   yes   |    NULL    |   U    |  1 or 0  |  yes   |
++-------------------------------+-------+---------+------------+--------+----------+--------+
+| .. c:macro:: PyBUF_STRIDED    |  yes  |   yes   |    NULL    |   U    |     0    |  NULL  |
++-------------------------------+-------+---------+------------+--------+----------+--------+
+| .. c:macro:: PyBUF_STRIDED_RO |  yes  |   yes   |    NULL    |   U    |  1 or 0  |  NULL  |
++-------------------------------+-------+---------+------------+--------+----------+--------+
+| .. c:macro:: PyBUF_CONTIG     |  yes  |   NULL  |    NULL    |   C    |     0    |  NULL  |
++-------------------------------+-------+---------+------------+--------+----------+--------+
+| .. c:macro:: PyBUF_CONTIG_RO  |  yes  |   NULL  |    NULL    |   C    |  1 or 0  |  NULL  |
++-------------------------------+-------+---------+------------+--------+----------+--------+
+
+
+Complex arrays
+==============
+
+NumPy-style: shape and strides
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The logical structure of NumPy-style arrays is defined by :c:member:`~Py_buffer.itemsize`,
+:c:member:`~Py_buffer.ndim`, :c:member:`~Py_buffer.shape` and :c:member:`~Py_buffer.strides`.
+
+If ``ndim == 0``, the memory location pointed to by :c:member:`~Py_buffer.buf` is
+interpreted as a scalar of size :c:member:`~Py_buffer.itemsize`. In that case,
+both :c:member:`~Py_buffer.shape` and :c:member:`~Py_buffer.strides` are *NULL*.
+
+If :c:member:`~Py_buffer.strides` is *NULL*, the array is interpreted as
+a standard n-dimensional C-array. Otherwise, the consumer must access an
+n-dimensional array as follows:
+
+   ``ptr = (char *)buf + indices[0] * strides[0] + ... + indices[n-1] * strides[n-1]``
+   ``item = *((typeof(item) *)ptr);``
+
+
+As noted above, :c:member:`~Py_buffer.buf` can point to any location within
+the actual memory block. An exporter can check the validity of a buffer with
+this function:
+
+.. code-block:: python
+
+   def verify_structure(memlen, itemsize, ndim, shape, strides, offset):
+       """Verify that the parameters represent a valid array within
+          the bounds of the allocated memory:
+              char *mem: start of the physical memory block
+              memlen: length of the physical memory block
+              offset: (char *)buf - mem
+       """
+       if offset % itemsize:
+           return False
+       if offset < 0 or offset+itemsize > memlen:
+           return False
+       if any(v % itemsize for v in strides):
+           return False
+
+       if ndim <= 0:
+           return ndim == 0 and not shape and not strides
+       if 0 in shape:
+           return True
+
+       imin = sum(strides[j]*(shape[j]-1) for j in range(ndim)
+                  if strides[j] <= 0)
+       imax = sum(strides[j]*(shape[j]-1) for j in range(ndim)
+                  if strides[j] > 0)
+
+       return 0 <= offset+imin and offset+imax+itemsize <= memlen
+
+
+PIL-style: shape, strides and suboffsets
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In addition to the regular items, PIL-style arrays can contain pointers
+that must be followed in order to get to the next element in a dimension.
+For example, the regular three-dimensional C-array ``char v[2][2][3]`` can
+also be viewed as an array of 2 pointers to 2 two-dimensional arrays:
+``char (*v[2])[2][3]``. In suboffsets representation, those two pointers
+can be embedded at the start of :c:member:`~Py_buffer.buf`, pointing
+to two ``char x[2][3]`` arrays that can be located anywhere in memory.
+
+
+Here is a function that returns a pointer to the element in an N-D array
+pointed to by an N-dimensional index when there are both non-NULL strides
+and suboffsets::
+
+   void *get_item_pointer(int ndim, void *buf, Py_ssize_t *strides,
+                          Py_ssize_t *suboffsets, Py_ssize_t *indices) {
+       char *pointer = (char*)buf;
+       int i;
+       for (i = 0; i < ndim; i++) {
+           pointer += strides[i] * indices[i];
+           if (suboffsets[i] >=0 ) {
+               pointer = *((char**)pointer) + suboffsets[i];
+           }
+       }
+       return (void*)pointer;
+   }
+
 
 Buffer-related functions
 ========================
 
-
 .. c:function:: int PyObject_CheckBuffer(PyObject *obj)
 
    Return 1 if *obj* supports the buffer interface otherwise 0.  When 1 is
    succeed.
 
 
-.. c:function:: int PyObject_GetBuffer(PyObject *obj, Py_buffer *view, int flags)
+.. c:function:: int PyObject_GetBuffer(PyObject *exporter, Py_buffer *view, int flags)
 
-      Export a view over some internal data from the target object *obj*.
-      *obj* must not be NULL, and *view* must point to an existing
-      :c:type:`Py_buffer` structure allocated by the caller (most uses of
-      this function will simply declare a local variable of type
-      :c:type:`Py_buffer`).  The *flags* argument is a bit field indicating
-      what kind of buffer is requested.  The buffer interface allows
-      for complicated memory layout possibilities; however, some callers
-      won't want to handle all the complexity and instead request a simple
-      view of the target object (using :c:macro:`PyBUF_SIMPLE` for a read-only
-      view and :c:macro:`PyBUF_WRITABLE` for a read-write view).
+   Send a request to *exporter* to fill in *view* as specified by  *flags*.
+   If the exporter cannot provide a buffer of the exact type, it MUST raise
+   :c:data:`PyExc_BufferError`, set :c:member:`view->obj` to *NULL* and
+   return -1.
 
-      Some exporters may not be able to share memory in every possible way and
-      may need to raise errors to signal to some consumers that something is
-      just not possible. These errors should be a :exc:`BufferError` unless
-      there is another error that is actually causing the problem. The
-      exporter can use flags information to simplify how much of the
-      :c:data:`Py_buffer` structure is filled in with non-default values and/or
-      raise an error if the object can't support a simpler view of its memory.
+   On success, fill in *view*, set :c:member:`view->obj` to a new reference
+   to *exporter* and return 0.
 
-      On success, 0 is returned and the *view* structure is filled with useful
-      values.  On error, -1 is returned and an exception is raised; the *view*
-      is left in an undefined state.
-
-      The following are the possible values to the *flags* arguments.
-
-      .. c:macro:: PyBUF_SIMPLE
-
-         This is the default flag.  The returned buffer exposes a read-only
-         memory area.  The format of data is assumed to be raw unsigned bytes,
-         without any particular structure.  This is a "stand-alone" flag
-         constant.  It never needs to be '|'d to the others.  The exporter will
-         raise an error if it cannot provide such a contiguous buffer of bytes.
-
-      .. c:macro:: PyBUF_WRITABLE
-
-         Like :c:macro:`PyBUF_SIMPLE`, but the returned buffer is writable.  If
-         the exporter doesn't support writable buffers, an error is raised.
-
-      .. c:macro:: PyBUF_STRIDES
-
-         This implies :c:macro:`PyBUF_ND`.  The returned buffer must provide
-         strides information (i.e. the strides cannot be NULL).  This would be
-         used when the consumer can handle strided, discontiguous arrays.
-         Handling strides automatically assumes you can handle shape.  The
-         exporter can raise an error if a strided representation of the data is
-         not possible (i.e. without the suboffsets).
-
-      .. c:macro:: PyBUF_ND
-
-         The returned buffer must provide shape information.  The memory will be
-         assumed C-style contiguous (last dimension varies the fastest).  The
-         exporter may raise an error if it cannot provide this kind of
-         contiguous buffer.  If this is not given then shape will be *NULL*.
-
-      .. c:macro:: PyBUF_C_CONTIGUOUS
-                  PyBUF_F_CONTIGUOUS
-                  PyBUF_ANY_CONTIGUOUS
-
-         These flags indicate that the contiguity returned buffer must be
-         respectively, C-contiguous (last dimension varies the fastest), Fortran
-         contiguous (first dimension varies the fastest) or either one.  All of
-         these flags imply :c:macro:`PyBUF_STRIDES` and guarantee that the
-         strides buffer info structure will be filled in correctly.
-
-      .. c:macro:: PyBUF_INDIRECT
-
-         This flag indicates the returned buffer must have suboffsets
-         information (which can be NULL if no suboffsets are needed).  This can
-         be used when the consumer can handle indirect array referencing implied
-         by these suboffsets. This implies :c:macro:`PyBUF_STRIDES`.
-
-      .. c:macro:: PyBUF_FORMAT
-
-         The returned buffer must have true format information if this flag is
-         provided.  This would be used when the consumer is going to be checking
-         for what 'kind' of data is actually stored.  An exporter should always
-         be able to provide this information if requested.  If format is not
-         explicitly requested then the format must be returned as *NULL* (which
-         means ``'B'``, or unsigned bytes).
-
-      .. c:macro:: PyBUF_STRIDED
-
-         This is equivalent to ``(PyBUF_STRIDES | PyBUF_WRITABLE)``.
-
-      .. c:macro:: PyBUF_STRIDED_RO
-
-         This is equivalent to ``(PyBUF_STRIDES)``.
-
-      .. c:macro:: PyBUF_RECORDS
-
-         This is equivalent to ``(PyBUF_STRIDES | PyBUF_FORMAT |
-         PyBUF_WRITABLE)``.
-
-      .. c:macro:: PyBUF_RECORDS_RO
-
-         This is equivalent to ``(PyBUF_STRIDES | PyBUF_FORMAT)``.
-
-      .. c:macro:: PyBUF_FULL
-
-         This is equivalent to ``(PyBUF_INDIRECT | PyBUF_FORMAT |
-         PyBUF_WRITABLE)``.
-
-      .. c:macro:: PyBUF_FULL_RO
-
-         This is equivalent to ``(PyBUF_INDIRECT | PyBUF_FORMAT)``.
-
-      .. c:macro:: PyBUF_CONTIG
-
-         This is equivalent to ``(PyBUF_ND | PyBUF_WRITABLE)``.
-
-      .. c:macro:: PyBUF_CONTIG_RO
-
-         This is equivalent to ``(PyBUF_ND)``.
+   Successful calls to :c:func:`PyObject_GetBuffer` must be paired with calls
+   to :c:func:`PyBuffer_Release`, similar to :c:func:`malloc` and :c:func:`free`.
+   Thus, after the consumer is done with the buffer, :c:func:`PyBuffer_Release`
+   must be called exactly once.
 
 
 .. c:function:: void PyBuffer_Release(Py_buffer *view)
 
-   Release the buffer *view*.  This should be called when the buffer is no
-   longer being used as it may free memory from it.
+   Release the buffer *view* and decrement the reference count for
+   :c:member:`view->obj`. This function MUST be called when the buffer
+   is no longer being used, otherwise reference leaks may occur.
+
+   It is an error to call this function on a buffer that was not obtained via
+   :c:func:`PyObject_GetBuffer`.
 
 
 .. c:function:: Py_ssize_t PyBuffer_SizeFromFormat(const char *)
 
-   Return the implied :c:data:`~Py_buffer.itemsize` from the struct-stype
-   :c:data:`~Py_buffer.format`.
+   Return the implied :c:data:`~Py_buffer.itemsize` from :c:data:`~Py_buffer.format`.
+   This function is not yet implemented.
 
 
-.. c:function:: int PyBuffer_IsContiguous(Py_buffer *view, char fortran)
+.. c:function:: int PyBuffer_IsContiguous(Py_buffer *view, char order)
 
-   Return 1 if the memory defined by the *view* is C-style (*fortran* is
-   ``'C'``) or Fortran-style (*fortran* is ``'F'``) contiguous or either one
-   (*fortran* is ``'A'``).  Return 0 otherwise.
+   Return 1 if the memory defined by the *view* is C-style (*order* is
+   ``'C'``) or Fortran-style (*order* is ``'F'``) contiguous or either one
+   (*order* is ``'A'``).  Return 0 otherwise.
 
 
-.. c:function:: void PyBuffer_FillContiguousStrides(int ndim, Py_ssize_t *shape, Py_ssize_t *strides, Py_ssize_t itemsize, char fortran)
+.. c:function:: void PyBuffer_FillContiguousStrides(int ndim, Py_ssize_t *shape, Py_ssize_t *strides, Py_ssize_t itemsize, char order)
 
    Fill the *strides* array with byte-strides of a contiguous (C-style if
-   *fortran* is ``'C'`` or Fortran-style if *fortran* is ``'F'``) array of the
+   *order* is ``'C'`` or Fortran-style if *order* is ``'F'``) array of the
    given shape with the given number of bytes per element.
 
 
-.. c:function:: int PyBuffer_FillInfo(Py_buffer *view, PyObject *obj, void *buf, Py_ssize_t len, int readonly, int infoflags)
+.. c:function:: int PyBuffer_FillInfo(Py_buffer *view, PyObject *exporter, void *buf, Py_ssize_t len, int readonly, int flags)
 
-   Fill in a buffer-info structure, *view*, correctly for an exporter that can
-   only share a contiguous chunk of memory of "unsigned bytes" of the given
-   length.  Return 0 on success and -1 (with raising an error) on error.
+   Handle buffer requests for an exporter that wants to expose *buf* of size *len*
+   with writability set according to *readonly*. *buf* is interpreted as a sequence
+   of unsigned bytes.
 
+   The *flags* argument indicates the request type. This function always fills in
+   *view* as specified by flags, unless *buf* has been designated as read-only
+   and :c:macro:`PyBUF_WRITABLE` is set in *flags*.
+
+   On success, set :c:member:`view->obj` to a new reference to *exporter* and
+   return 0. Otherwise, raise :c:data:`PyExc_BufferError`, set
+   :c:member:`view->obj` to *NULL* and return -1;
+
+   If this function is used as part of a :ref:`getbufferproc <buffer-structs>`,
+   *exporter* MUST be set to the exporting object. Otherwise, *exporter* MUST
+   be NULL.
+
+
+

Doc/c-api/exceptions.rst

 
 .. c:function:: PyObject* PyException_GetCause(PyObject *ex)
 
-   Return the cause (another exception instance set by ``raise ... from ...``)
-   associated with the exception as a new reference, as accessible from Python
-   through :attr:`__cause__`.  If there is no cause associated, this returns
-   *NULL*.
+   Return the cause (either an exception instance, or :const:`None`,
+   set by ``raise ... from ...``) associated with the exception as a new
+   reference, as accessible from Python through :attr:`__cause__`.
+
+   If there is no cause associated, this returns *NULL* (from Python
+   ``__cause__ is Ellipsis``).  If the cause is :const:`None`, the default
+   exception display routines stop showing the context chain.
 
 
 .. c:function:: void PyException_SetCause(PyObject *ex, PyObject *ctx)
 
    Set the cause associated with the exception to *ctx*.  Use *NULL* to clear
-   it.  There is no type check to make sure that *ctx* is an exception instance.
-   This steals a reference to *ctx*.
+   it.  There is no type check to make sure that *ctx* is either an exception
+   instance or :const:`None`.  This steals a reference to *ctx*.
+
+   If the cause is set to :const:`None` the default exception display
+   routines will not display this exception's context, and will not follow the
+   chain any further.
 
 
 .. _unicodeexceptions:

Doc/c-api/memoryview.rst

 
    Create a memoryview object from an object that provides the buffer interface.
    If *obj* supports writable buffer exports, the memoryview object will be
-   readable and writable, otherwise it will be read-only.
+   read/write, otherwise it may be either read-only or read/write at the
+   discretion of the exporter.
 
+.. c:function:: PyObject *PyMemoryView_FromMemory(char *mem, Py_ssize_t size, int flags)
+
+   Create a memoryview object using *mem* as the underlying buffer.
+   *flags* can be one of :c:macro:`PyBUF_READ` or :c:macro:`PyBUF_WRITE`.
 
 .. c:function:: PyObject *PyMemoryView_FromBuffer(Py_buffer *view)
 
    Create a memoryview object wrapping the given buffer structure *view*.
-   The memoryview object then owns the buffer represented by *view*, which
-   means you shouldn't try to call :c:func:`PyBuffer_Release` yourself: it
-   will be done on deallocation of the memoryview object.
-
+   For simple byte buffers, :c:func:`PyMemoryView_FromMemory` is the preferred
+   function.
 
 .. c:function:: PyObject *PyMemoryView_GetContiguous(PyObject *obj, int buffertype, char order)
 
    currently allowed to create subclasses of :class:`memoryview`.
 
 
-.. c:function:: Py_buffer *PyMemoryView_GET_BUFFER(PyObject *obj)
+.. c:function:: Py_buffer *PyMemoryView_GET_BUFFER(PyObject *mview)
 
-   Return a pointer to the buffer structure wrapped by the given
-   memoryview object.  The object **must** be a memoryview instance;
-   this macro doesn't check its type, you must do it yourself or you
-   will risk crashes.
+   Return a pointer to the memoryview's private copy of the exporter's buffer.
+   *mview* **must** be a memoryview instance; this macro doesn't check its type,
+   you must do it yourself or you will risk crashes.
 
+.. c:function:: Py_buffer *PyMemoryView_GET_BASE(PyObject *mview)
+
+   Return either a pointer to the exporting object that the memoryview is based
+   on or *NULL* if the memoryview has been created by one of the functions
+   :c:func:`PyMemoryView_FromMemory` or :c:func:`PyMemoryView_FromBuffer`.
+   *mview* **must** be a memoryview instance.
+

Doc/c-api/type.rst

 
 .. c:function:: PyObject* PyType_GenericNew(PyTypeObject *type, PyObject *args, PyObject *kwds)
 
-   XXX: Document.
+   Generic handler for the :attr:`tp_new` slot of a type object.  Initialize
+   all instance variables to *NULL*.
 
 .. c:function:: int PyType_Ready(PyTypeObject *type)
 

Doc/c-api/typeobj.rst

 
 .. sectionauthor:: Greg J. Stein <greg@lyra.org>
 .. sectionauthor:: Benjamin Peterson
-
-
-The :ref:`buffer interface <bufferobjects>` exports a model where an object can expose its internal
-data.
-
-If an object does not export the buffer interface, then its :attr:`tp_as_buffer`
-member in the :c:type:`PyTypeObject` structure should be *NULL*.  Otherwise, the
-:attr:`tp_as_buffer` will point to a :c:type:`PyBufferProcs` structure.
-
+.. sectionauthor:: Stefan Krah
 
 .. c:type:: PyBufferProcs
 
-   Structure used to hold the function pointers which define an implementation of
-   the buffer protocol.
+   This structure holds pointers to the functions required by the
+   :ref:`Buffer protocol <bufferobjects>`. The protocol defines how
+   an exporter object can expose its internal data to consumer objects.
 
-   .. c:member:: getbufferproc bf_getbuffer
+.. c:member:: getbufferproc PyBufferProcs.bf_getbuffer
 
-      This should fill a :c:type:`Py_buffer` with the necessary data for
-      exporting the type.  The signature of :data:`getbufferproc` is ``int
-      (PyObject *obj, Py_buffer *view, int flags)``.  *obj* is the object to
-      export, *view* is the :c:type:`Py_buffer` struct to fill, and *flags* gives
-      the conditions the caller wants the memory under.  (See
-      :c:func:`PyObject_GetBuffer` for all flags.)  :c:member:`bf_getbuffer` is
-      responsible for filling *view* with the appropriate information.
-      (:c:func:`PyBuffer_FillView` can be used in simple cases.)  See
-      :c:type:`Py_buffer`\s docs for what needs to be filled in.
+   The signature of this function is::
 
+       int (PyObject *exporter, Py_buffer *view, int flags);
 
-   .. c:member:: releasebufferproc bf_releasebuffer
+   Handle a request to *exporter* to fill in *view* as specified by *flags*.
+   A standard implementation of this function will take these steps:
 
-      This should release the resources of the buffer.  The signature of
-      :c:data:`releasebufferproc` is ``void (PyObject *obj, Py_buffer *view)``.
-      If the :c:data:`bf_releasebuffer` function is not provided (i.e. it is
-      *NULL*), then it does not ever need to be called.
+   - Check if the request can be met. If not, raise :c:data:`PyExc_BufferError`,
+     set :c:data:`view->obj` to *NULL* and return -1.
 
-      The exporter of the buffer interface must make sure that any memory
-      pointed to in the :c:type:`Py_buffer` structure remains valid until
-      releasebuffer is called.  Exporters will need to define a
-      :c:data:`bf_releasebuffer` function if they can re-allocate their memory,
-      strides, shape, suboffsets, or format variables which they might share
-      through the struct bufferinfo.
+   - Fill in the requested fields.
 
-      See :c:func:`PyBuffer_Release`.
+   - Increment an internal counter for the number of exports.
+
+   - Set :c:data:`view->obj` to *exporter* and increment :c:data:`view->obj`.
+
+   - Return 0.
+
+   The individual fields of *view* are described in section
+   :ref:`Buffer structure <buffer-structure>`, the rules how an exporter
+   must react to specific requests are in section
+   :ref:`Buffer request types <buffer-request-types>`.
+
+   All memory pointed to in the :c:type:`Py_buffer` structure belongs to
+   the exporter and must remain valid until there are no consumers left.
+   :c:member:`~Py_buffer.shape`, :c:member:`~Py_buffer.strides`,
+   :c:member:`~Py_buffer.suboffsets` and :c:member:`~Py_buffer.internal`
+   are read-only for the consumer.
+
+   :c:func:`PyBuffer_FillInfo` provides an easy way of exposing a simple
+   bytes buffer while dealing correctly with all request types.
+
+   :c:func:`PyObject_GetBuffer` is the interface for the consumer that
+   wraps this function.
+
+.. c:member:: releasebufferproc PyBufferProcs.bf_releasebuffer
+
+   The signature of this function is::
+
+       void (PyObject *exporter, Py_buffer *view);
+
+   Handle a request to release the resources of the buffer. If no resources
+   need to be released, this field may be *NULL*. A standard implementation
+   of this function will take these steps:
+
+   - Decrement an internal counter for the number of exports.
+
+   - If the counter is 0, free all memory associated with *view*.
+
+   The exporter MUST use the :c:member:`~Py_buffer.internal` field to keep
+   track of buffer-specific resources (if present). This field is guaranteed
+   to remain constant, while a consumer MAY pass a copy of the original buffer
+   as the *view* argument.
+
+
+   This function MUST NOT decrement :c:data:`view->obj`, since that is
+   done automatically in :c:func:`PyBuffer_Release`.
+
+
+   :c:func:`PyBuffer_Release` is the interface for the consumer that
+   wraps this function.

Doc/extending/extending.rst

           imports it. */
        PyImport_ImportModule("spam");
 
-An example may be found in the file :file:`Demo/embed/demo.c` in the Python
-source distribution.
-
 .. note::
 
    Removing entries from ``sys.modules`` or importing compiled modules into

Doc/extending/newtypes.rst

 ==========
 
 The Python runtime sees all Python objects as variables of type
-:c:type:`PyObject\*`.  A :c:type:`PyObject` is not a very magnificent object - it
-just contains the refcount and a pointer to the object's "type object".  This is
-where the action is; the type object determines which (C) functions get called
-when, for instance, an attribute gets looked up on an object or it is multiplied
-by another object.  These C functions are called "type methods".
+:c:type:`PyObject\*`, which serves as a "base type" for all Python objects.
+:c:type:`PyObject` itself only contains the refcount and a pointer to the
+object's "type object".  This is where the action is; the type object determines
+which (C) functions get called when, for instance, an attribute gets looked
+up on an object or it is multiplied by another object.  These C functions
+are called "type methods".
 
 So, if you want to define a new object type, you need to create a new type
 object.
        PyObject_HEAD
    } noddy_NoddyObject;
 
-This is what a Noddy object will contain---in this case, nothing more than every
-Python object contains, namely a refcount and a pointer to a type object.  These
-are the fields the ``PyObject_HEAD`` macro brings in.  The reason for the macro
-is to standardize the layout and to enable special debugging fields in debug
-builds.  Note that there is no semicolon after the ``PyObject_HEAD`` macro; one
-is included in the macro definition.  Be wary of adding one by accident; it's
-easy to do from habit, and your compiler might not complain, but someone else's
-probably will!  (On Windows, MSVC is known to call this an error and refuse to
-compile the code.)
+This is what a Noddy object will contain---in this case, nothing more than what
+every Python object contains---a refcount and a pointer to a type object.
+These are the fields the ``PyObject_HEAD`` macro brings in.  The reason for the
+macro is to standardize the layout and to enable special debugging fields in
+debug builds.  Note that there is no semicolon after the ``PyObject_HEAD``
+macro; one is included in the macro definition.  Be wary of adding one by
+accident; it's easy to do from habit, and your compiler might not complain,
+but someone else's probably will!  (On Windows, MSVC is known to call this an
+error and refuse to compile the code.)
 
 For contrast, let's take a look at the corresponding definition for standard
 Python floats::
 Adding data and methods to the Basic example
 --------------------------------------------
 
-Let's expend the basic example to add some data and methods.  Let's also make
+Let's extend the basic example to add some data and methods.  Let's also make
 the type usable as a base class. We'll create a new module, :mod:`noddy2` that
 adds these capabilities:
 
 created. New methods always accept positional and keyword arguments, but they
 often ignore the arguments, leaving the argument handling to initializer
 methods. Note that if the type supports subclassing, the type passed may not be
-the type being defined.  The new method calls the tp_alloc slot to allocate
-memory. We don't fill the :attr:`tp_alloc` slot ourselves. Rather
+the type being defined.  The new method calls the :attr:`tp_alloc` slot to
+allocate memory. We don't fill the :attr:`tp_alloc` slot ourselves. Rather
 :c:func:`PyType_Ready` fills it for us by inheriting it from our base class,
 which is :class:`object` by default.  Most types use the default allocation.
 
    static PyObject *
    Noddy_name(Noddy* self)
    {
-       static PyObject *format = NULL;
-       PyObject *args, *result;
-
-       if (format == NULL) {
-           format = PyString_FromString("%s %s");
-           if (format == NULL)
-               return NULL;
-       }
-
        if (self->first == NULL) {
            PyErr_SetString(PyExc_AttributeError, "first");
            return NULL;
            return NULL;
        }
 
-       args = Py_BuildValue("OO", self->first, self->last);
-       if (args == NULL)
-           return NULL;
-
-       result = PyString_Format(format, args);
-       Py_DECREF(args);
-
-       return result;
+       return PyUnicode_FromFormat("%S %S", self->first, self->last);
    }
 
 The method is implemented as a C function that takes a :class:`Noddy` (or
 :class:`Noddy` subclass) instance as the first argument.  Methods always take an
 instance as the first argument. Methods often take positional and keyword
-arguments as well, but in this cased we don't take any and don't need to accept
+arguments as well, but in this case we don't take any and don't need to accept
 a positional argument tuple or keyword argument dictionary. This method is
 equivalent to the Python method::
 
 at the end; it is a sentinel that marks the end of the array.  The
 :attr:`ml_name` field of the sentinel must be *NULL*.
 
-XXX Need to refer to some unified discussion of the structure fields, shared
-with the next section.
-
 The second table is used to define attributes which map directly to data stored
 in the instance.  A variety of primitive C types are supported, and access may
 be read-only or read-write.  The structures in the table are defined as::
 convert Python values to and from C values.  The :attr:`flags` field is used to
 store flags which control how the attribute can be accessed.
 
-XXX Need to move some of this to a shared section!
-
 The following flag constants are defined in :file:`structmember.h`; they may be
 combined using bitwise-OR.
 
        return result;
    }
 
-XXX some fields need to be added here... ::
+::
 
    /* Iterators */
    getiterfunc tp_iter;

Doc/howto/cporting.rst

 
 .. _cporting-howto:
 
-********************************
-Porting Extension Modules to 3.0
-********************************
+*************************************
+Porting Extension Modules to Python 3
+*************************************
 
 :author: Benjamin Peterson
 
 
 .. topic:: Abstract
 
-   Although changing the C-API was not one of Python 3.0's objectives, the many
-   Python level changes made leaving 2.x's API intact impossible.  In fact, some
-   changes such as :func:`int` and :func:`long` unification are more obvious on
-   the C level.  This document endeavors to document incompatibilities and how
-   they can be worked around.
+   Although changing the C-API was not one of Python 3's objectives,
+   the many Python-level changes made leaving Python 2's API intact
+   impossible.  In fact, some changes such as :func:`int` and
+   :func:`long` unification are more obvious on the C level.  This
+   document endeavors to document incompatibilities and how they can
+   be worked around.
 
 
 Conditional compilation
 =======================
 
-The easiest way to compile only some code for 3.0 is to check if
-:c:macro:`PY_MAJOR_VERSION` is greater than or equal to 3. ::
+The easiest way to compile only some code for Python 3 is to check
+if :c:macro:`PY_MAJOR_VERSION` is greater than or equal to 3. ::
 
    #if PY_MAJOR_VERSION >= 3
    #define IS_PY3K
 Changes to Object APIs
 ======================
 
-Python 3.0 merged together some types with similar functions while cleanly
+Python 3 merged together some types with similar functions while cleanly
 separating others.
 
 
 -----------------------
 
 
-Python 3.0's :func:`str` (``PyString_*`` functions in C) type is equivalent to
-2.x's :func:`unicode` (``PyUnicode_*``).  The old 8-bit string type has become
-:func:`bytes`.  Python 2.6 and later provide a compatibility header,
+Python 3's :func:`str` (``PyString_*`` functions in C) type is equivalent to
+Python 2's :func:`unicode` (``PyUnicode_*``).  The old 8-bit string type has
+become :func:`bytes`.  Python 2.6 and later provide a compatibility header,
 :file:`bytesobject.h`, mapping ``PyBytes`` names to ``PyString`` ones.  For best
-compatibility with 3.0, :c:type:`PyUnicode` should be used for textual data and
+compatibility with Python 3, :c:type:`PyUnicode` should be used for textual data and
 :c:type:`PyBytes` for binary data.  It's also important to remember that
-:c:type:`PyBytes` and :c:type:`PyUnicode` in 3.0 are not interchangeable like
-:c:type:`PyString` and :c:type:`PyUnicode` are in 2.x.  The following example
+:c:type:`PyBytes` and :c:type:`PyUnicode` in Python 3 are not interchangeable like
+:c:type:`PyString` and :c:type:`PyUnicode` are in Python 2.  The following example
 shows best practices with regards to :c:type:`PyUnicode`, :c:type:`PyString`,
 and :c:type:`PyBytes`. ::
 
 long/int Unification
 --------------------
 
-In Python 3.0, there is only one integer type.  It is called :func:`int` on the
-Python level, but actually corresponds to 2.x's :func:`long` type.  In the
-C-API, ``PyInt_*`` functions are replaced by their ``PyLong_*`` neighbors.  The
-best course of action here is using the ``PyInt_*`` functions aliased to
+Python 3 has only one integer type, :func:`int`.  But it actually
+corresponds to Python 2's :func:`long` type--the :func:`int` type
+used in Python 2 was removed.  In the C-API, ``PyInt_*`` functions
+are replaced by their ``PyLong_*`` equivalents.
+
+The best course of action here is using the ``PyInt_*`` functions aliased to
 ``PyLong_*`` found in :file:`intobject.h`.  The abstract ``PyNumber_*`` APIs
 can also be used in some cases. ::
 
 Module initialization and state
 ===============================
 
-Python 3.0 has a revamped extension module initialization system.  (See
-:pep:`3121`.)  Instead of storing module state in globals, they should be stored
-in an interpreter specific structure.  Creating modules that act correctly in
-both 2.x and 3.0 is tricky.  The following simple example demonstrates how. ::
+Python 3 has a revamped extension module initialization system.  (See
+:pep:`3121`.)  Instead of storing module state in globals, they should
+be stored in an interpreter specific structure.  Creating modules that
+act correctly in both Python 2 and Python 3 is tricky.  The following
+simple example demonstrates how. ::
 
    #include "Python.h"
 
    }
 
 
+CObject replaced with Capsule
+=============================
+
+The :c:type:`Capsule` object was introduced in Python 3.1 and 2.7 to replace
+:c:type:`CObject`.  CObjects were useful,
+but the :c:type:`CObject` API was problematic: it didn't permit distinguishing
+between valid CObjects, which allowed mismatched CObjects to crash the
+interpreter, and some of its APIs relied on undefined behavior in C.
+(For further reading on the rationale behind Capsules, please see :issue:`5630`.)
+
+If you're currently using CObjects, and you want to migrate to 3.1 or newer,
+you'll need to switch to Capsules.
+:c:type:`CObject` was deprecated in 3.1 and 2.7 and completely removed in
+Python 3.2.  If you only support 2.7, or 3.1 and above, you
+can simply switch to :c:type:`Capsule`.  If you need to support Python 3.0,
+or versions of Python earlier than 2.7,
+you'll have to support both CObjects and Capsules.
+(Note that Python 3.0 is no longer supported, and it is not recommended
+for production use.)
+
+The following example header file :file:`capsulethunk.h` may
+solve the problem for you.  Simply write your code against the
+:c:type:`Capsule` API and include this header file after
+:file:`Python.h`.  Your code will automatically use Capsules
+in versions of Python with Capsules, and switch to CObjects
+when Capsules are unavailable.
+
+:file:`capsulethunk.h` simulates Capsules using CObjects.  However,
+:c:type:`CObject` provides no place to store the capsule's "name".  As a
+result the simulated :c:type:`Capsule` objects created by :file:`capsulethunk.h`
+behave slightly differently from real Capsules.  Specifically:
+
+  * The name parameter passed in to :c:func:`PyCapsule_New` is ignored.
+
+  * The name parameter passed in to :c:func:`PyCapsule_IsValid` and
+    :c:func:`PyCapsule_GetPointer` is ignored, and no error checking
+    of the name is performed.
+
+  * :c:func:`PyCapsule_GetName` always returns NULL.
+
+  * :c:func:`PyCapsule_SetName` always throws an exception and
+    returns failure.  (Since there's no way to store a name
+    in a CObject, noisy failure of :c:func:`PyCapsule_SetName`
+    was deemed preferable to silent failure here.  If this is
+    inconveient, feel free to modify your local
+    copy as you see fit.)
+
+You can find :file:`capsulethunk.h` in the Python source distribution
+in the :file:`Doc/includes` directory.  We also include it here for
+your reference; here is :file:`capsulethunk.h`:
+
+.. literalinclude:: ../includes/capsulethunk.h
+
+
+
 Other options
 =============
 
 If you are writing a new extension module, you might consider `Cython
 <http://www.cython.org>`_.  It translates a Python-like language to C.  The
-extension modules it creates are compatible with Python 3.x and 2.x.
+extension modules it creates are compatible with Python 3 and Python 2.
 

Doc/howto/logging-cookbook.rst

 .. currentmodule:: logging.handlers
 
 Sometimes you have to get your logging handlers to do their work without
-blocking the thread youre logging from. This is common in Web applications,
+blocking the thread you're logging from. This is common in Web applications,
 though of course it also occurs in other scenarios.
 
 A common culprit which demonstrates sluggish behaviour is the
 :class:`SMTPHandler`: sending emails can take a long time, for a
-number of reasons outside the developers control (for example, a poorly
+number of reasons outside the developer's control (for example, a poorly
 performing mail or network infrastructure). But almost any network-based
 handler can block: Even a :class:`SocketHandler` operation may do a
 DNS query under the hood which is too slow (and this query can be deep in the
 
 The second part of the solution is :class:`QueueListener`, which has been
 designed as the counterpart to :class:`QueueHandler`.  A
-:class:`QueueListener` is very simple: its passed a queue and some handlers,
+:class:`QueueListener` is very simple: it's passed a queue and some handlers,
 and it fires up an internal thread which listens to its queue for LogRecords
 sent from ``QueueHandlers`` (or any other source of ``LogRecords``, for that
 matter). The ``LogRecords`` are removed from the queue and passed to the
                 raise
             except:
                 import sys, traceback
-                print >> sys.stderr, 'Whoops! Problem:'
+                print('Whoops! Problem:', file=sys.stderr)
                 traceback.print_exc(file=sys.stderr)
 
     # Arrays used for random selections in this demo
 Obviously this example sets the log length much too small as an extreme
 example.  You would want to set *maxBytes* to an appropriate value.
 
+.. _format-styles:
+
+Use of alternative formatting styles
+------------------------------------
+
+When logging was added to the Python standard library, the only way of
+formatting messages with variable content was to use the %-formatting
+method. Since then, Python has gained two new formatting approaches:
+:class:`string.Template` (added in Python 2.4) and :meth:`str.format`
+(added in Python 2.6).
+
+Logging (as of 3.2) provides improved support for these two additional
+formatting styles. The :class:`Formatter` class been enhanced to take an
+additional, optional keyword parameter named ``style``. This defaults to
+``'%'``, but other possible values are ``'{'`` and ``'$'``, which correspond
+to the other two formatting styles. Backwards compatibility is maintained by
+default (as you would expect), but by explicitly specifying a style parameter,
+you get the ability to specify format strings which work with
+:meth:`str.format` or :class:`string.Template`. Here's an example console
+session to show the possibilities:
+
+.. code-block:: pycon
+
+    >>> import logging
+    >>> root = logging.getLogger()
+    >>> root.setLevel(logging.DEBUG)
+    >>> handler = logging.StreamHandler()
+    >>> bf = logging.Formatter('{asctime} {name} {levelname:8s} {message}',
+    ...                        style='{')
+    >>> handler.setFormatter(bf)
+    >>> root.addHandler(handler)
+    >>> logger = logging.getLogger('foo.bar')
+    >>> logger.debug('This is a DEBUG message')
+    2010-10-28 15:11:55,341 foo.bar DEBUG    This is a DEBUG message
+    >>> logger.critical('This is a CRITICAL message')
+    2010-10-28 15:12:11,526 foo.bar CRITICAL This is a CRITICAL message
+    >>> df = logging.Formatter('$asctime $name ${levelname} $message',
+    ...                        style='$')
+    >>> handler.setFormatter(df)
+    >>> logger.debug('This is a DEBUG message')
+    2010-10-28 15:13:06,924 foo.bar DEBUG This is a DEBUG message
+    >>> logger.critical('This is a CRITICAL message')
+    2010-10-28 15:13:11,494 foo.bar CRITICAL This is a CRITICAL message
+    >>>
+
+Note that the formatting of logging messages for final output to logs is
+completely independent of how an individual logging message is constructed.
+That can still use %-formatting, as shown here::
+
+    >>> logger.error('This is an%s %s %s', 'other,', 'ERROR,', 'message')
+    2010-10-28 15:19:29,833 foo.bar ERROR This is another, ERROR, message
+    >>>
+
+Logging calls (``logger.debug()``, ``logger.info()`` etc.) only take
+positional parameters for the actual logging message itself, with keyword
+parameters used only for determining options for how to handle the actual
+logging call (e.g. the ``exc_info`` keyword parameter to indicate that
+traceback information should be logged, or the ``extra`` keyword parameter
+to indicate additional contextual information to be added to the log). So
+you cannot directly make logging calls using :meth:`str.format` or
+:class:`string.Template` syntax, because internally the logging package
+uses %-formatting to merge the format string and the variable arguments.
+There would no changing this while preserving backward compatibility, since
+all logging calls which are out there in existing code will be using %-format
+strings.
+
+There is, however, a way that you can use {}- and $- formatting to construct
+your individual log messages. Recall that for a message you can use an
+arbitrary object as a message format string, and that the logging package will
+call ``str()`` on that object to get the actual format string. Consider the
+following two classes::
+
+    class BraceMessage(object):
+        def __init__(self, fmt, *args, **kwargs):
+            self.fmt = fmt
+            self.args = args
+            self.kwargs = kwargs
+
+        def __str__(self):
+            return self.fmt.format(*self.args, **self.kwargs)
+
+    class DollarMessage(object):
+        def __init__(self, fmt, **kwargs):
+            self.fmt = fmt
+            self.kwargs = kwargs
+
+        def __str__(self):
+            from string import Template
+            return Template(self.fmt).substitute(**self.kwargs)
+
+Either of these can be used in place of a format string, to allow {}- or
+$-formatting to be used to build the actual "message" part which appears in the
+formatted log output in place of "%(message)s" or "{message}" or "$message".
+It's a little unwieldy to use the class names whenever you want to log
+something, but it's quite palatable if you use an alias such as __ (double
+underscore – not to be confused with _, the single underscore used as a
+synonym/alias for :func:`gettext.gettext` or its brethren).
+
+The above classes are not included in Python, though they're easy enough to
+copy and paste into your own code. They can be used as follows (assuming that
+they're declared in a module called ``wherever``):
+
+.. code-block:: pycon
+
+    >>> from wherever import BraceMessage as __
+    >>> print(__('Message with {0} {name}', 2, name='placeholders'))
+    Message with 2 placeholders
+    >>> class Point: pass
+    ...
+    >>> p = Point()
+    >>> p.x = 0.5
+    >>> p.y = 0.5
+    >>> print(__('Message with coordinates: ({point.x:.2f}, {point.y:.2f})',
+    ...       point=p))
+    Message with coordinates: (0.50, 0.50)
+    >>> from wherever import DollarMessage as __
+    >>> print(__('Message with $num $what', num=2, what='placeholders'))
+    Message with 2 placeholders
+    >>>
+
+While the above examples use ``print()`` to show how the formatting works, you
+would of course use ``logger.debug()`` or similar to actually log using this
+approach.
+
+One thing to note is that you pay no significant performance penalty with this
+approach: the actual formatting happens not when you make the logging call, but
+when (and if) the logged message is actually about to be output to a log by a
+handler. So the only slightly unusual thing which might trip you up is that the
+parentheses go around the format string and the arguments, not just the format
+string. That's because the __ notation is just syntax sugar for a constructor
+call to one of the XXXMessage classes.
+
+
+.. currentmodule:: logging
+
+.. _custom-logrecord:
+
+Customising ``LogRecord``
+-------------------------
+
+Every logging event is represented by a :class:`LogRecord` instance.
+When an event is logged and not filtered out by a logger's level, a
+:class:`LogRecord` is created, populated with information about the event and
+then passed to the handlers for that logger (and its ancestors, up to and
+including the logger where further propagation up the hierarchy is disabled).
+Before Python 3.2, there were only two places where this creation was done:
+
+* :meth:`Logger.makeRecord`, which is called in the normal process of
+  logging an event. This invoked :class:`LogRecord` directly to create an
+  instance.
+* :func:`makeLogRecord`, which is called with a dictionary containing
+  attributes to be added to the LogRecord. This is typically invoked when a
+  suitable dictionary has been received over the network (e.g. in pickle form
+  via a :class:`~handlers.SocketHandler`, or in JSON form via an
+  :class:`~handlers.HTTPHandler`).
+
+This has usually meant that if you need to do anything special with a
+:class:`LogRecord`, you've had to do one of the following.
+
+* Create your own :class:`Logger` subclass, which overrides
+  :meth:`Logger.makeRecord`, and set it using :func:`~logging.setLoggerClass`
+  before any loggers that you care about are instantiated.
+* Add a :class:`Filter` to a logger or handler, which does the
+  necessary special manipulation you need when its
+  :meth:`~Filter.filter` method is called.
+
+The first approach would be a little unwieldy in the scenario where (say)
+several different libraries wanted to do different things. Each would attempt
+to set its own :class:`Logger` subclass, and the one which did this last would
+win.
+
+The second approach works reasonably well for many cases, but does not allow
+you to e.g. use a specialized subclass of :class:`LogRecord`. Library
+developers can set a suitable filter on their loggers, but they would have to
+remember to do this every time they introduced a new logger (which they would
+do simply by adding new packages or modules and doing ::
+
+   logger = logging.getLogger(__name__)
+
+at module level). It's probably one too many things to think about. Developers
+could also add the filter to a :class:`~logging.NullHandler` attached to their
+top-level logger, but this would not be invoked if an application developer
+attached a handler to a lower-level library logger – so output from that
+handler would not reflect the intentions of the library developer.
+
+In Python 3.2 and later, :class:`~logging.LogRecord` creation is done through a
+factory, which you can specify. The factory is just a callable you can set with
+:func:`~logging.setLogRecordFactory`, and interrogate with
+:func:`~logging.getLogRecordFactory`. The factory is invoked with the same
+signature as the :class:`~logging.LogRecord` constructor, as :class:`LogRecord`
+is the default setting for the factory.
+
+This approach allows a custom factory to control all aspects of LogRecord
+creation. For example, you could return a subclass, or just add some additional
+attributes to the record once created, using a pattern similar to this::
+
+    old_factory = logging.getLogRecordFactory()
+
+    def record_factory(*args, **kwargs):
+        record = old_factory(*args, **kwargs)
+        record.custom_attribute = 0xdecafbad
+        return record
+
+    logging.setLogRecordFactory(record_factory)
+
+This pattern allows different libraries to chain factories together, and as
+long as they don't overwrite each other's attributes or unintentionally
+overwrite the attributes provided as standard, there should be no surprises.
+However, it should be borne in mind that each link in the chain adds run-time
+overhead to all logging operations, and the technique should only be used when
+the use of a :class:`Filter` does not provide the desired result.
+
+
 .. _zeromq-handlers:
 
 Subclassing QueueHandler - a ZeroMQ example

Doc/includes/capsulethunk.h

+#ifndef __CAPSULETHUNK_H
+#define __CAPSULETHUNK_H
+
+#if (    (PY_VERSION_HEX <  0x02070000) \
+     || ((PY_VERSION_HEX >= 0x03000000) \
+      && (PY_VERSION_HEX <  0x03010000)) )
+
+#define __PyCapsule_GetField(capsule, field, default_value) \
+    ( PyCapsule_CheckExact(capsule) \
+        ? (((PyCObject *)capsule)->field) \
+        : (default_value) \
+    ) \
+
+#define __PyCapsule_SetField(capsule, field, value) \
+    ( PyCapsule_CheckExact(capsule) \
+        ? (((PyCObject *)capsule)->field = value), 1 \
+        : 0 \
+    ) \
+
+
+#define PyCapsule_Type PyCObject_Type
+
+#define PyCapsule_CheckExact(capsule) (PyCObject_Check(capsule))
+#define PyCapsule_IsValid(capsule, name) (PyCObject_Check(capsule))
+
+
+#define PyCapsule_New(pointer, name, destructor) \
+    (PyCObject_FromVoidPtr(pointer, destructor))
+
+
+#define PyCapsule_GetPointer(capsule, name) \
+    (PyCObject_AsVoidPtr(capsule))
+
+/* Don't call PyCObject_SetPointer here, it fails if there's a destructor */
+#define PyCapsule_SetPointer(capsule, pointer) \
+    __PyCapsule_SetField(capsule, cobject, pointer)
+
+
+#define PyCapsule_GetDestructor(capsule) \
+    __PyCapsule_GetField(capsule, destructor)
+
+#define PyCapsule_SetDestructor(capsule, dtor) \
+    __PyCapsule_SetField(capsule, destructor, dtor)
+
+
+/*
+ * Sorry, there's simply no place
+ * to store a Capsule "name" in a CObject.
+ */
+#define PyCapsule_GetName(capsule) NULL
+
+static int
+PyCapsule_SetName(PyObject *capsule, const char *unused)
+{
+    unused = unused;
+    PyErr_SetString(PyExc_NotImplementedError,
+        "can't use PyCapsule_SetName with CObjects");
+    return 1;
+}
+
+
+
+#define PyCapsule_GetContext(capsule) \
+    __PyCapsule_GetField(capsule, descr)
+
+#define PyCapsule_SetContext(capsule, context) \
+    __PyCapsule_SetField(capsule, descr, context)
+
+
+static void *
+PyCapsule_Import(const char *name, int no_block)
+{
+    PyObject *object = NULL;
+    void *return_value = NULL;
+    char *trace;
+    size_t name_length = (strlen(name) + 1) * sizeof(char);
+    char *name_dup = (char *)PyMem_MALLOC(name_length);
+
+    if (!name_dup) {
+        return NULL;
+    }
+
+    memcpy(name_dup, name, name_length);
+
+    trace = name_dup;
+    while (trace) {
+        char *dot = strchr(trace, '.');
+        if (dot) {
+            *dot++ = '\0';
+        }
+
+        if (object == NULL) {
+            if (no_block) {
+                object = PyImport_ImportModuleNoBlock(trace);
+            } else {
+                object = PyImport_ImportModule(trace);
+                if (!object) {
+                    PyErr_Format(PyExc_ImportError,
+                        "PyCapsule_Import could not "
+                        "import module \"%s\"", trace);
+                }
+            }
+        } else {
+            PyObject *object2 = PyObject_GetAttrString(object, trace);
+            Py_DECREF(object);
+            object = object2;
+        }
+        if (!object) {
+            goto EXIT;
+        }
+
+        trace = dot;
+    }
+
+    if (PyCObject_Check(object)) {
+        PyCObject *cobject = (PyCObject *)object;
+        return_value = cobject->cobject;
+    } else {
+        PyErr_Format(PyExc_AttributeError,
+            "PyCapsule_Import \"%s\" is not valid",
+            name);
+    }
+
+EXIT:
+    Py_XDECREF(object);
+    if (name_dup) {
+        PyMem_FREE(name_dup);
+    }
+    return return_value;
+}
+
+#endif /* #if PY_VERSION_HEX < 0x02070000 */
+
+#endif /* __CAPSULETHUNK_H */

Doc/library/exceptions.rst

 defining exceptions is available in the Python Tutorial under
 :ref:`tut-userexceptions`.
 
+When raising (or re-raising) an exception in an :keyword:`except` clause
+:attr:`__context__` is automatically set to the last exception caught; if the
+new exception is not handled the traceback that is eventually displayed will
+include the originating exception(s) and the final exception.
+
+This implicit exception chain can be made explicit by using :keyword:`from`
+with :keyword:`raise`.  The single argument to :keyword:`from` must be an
+exception or :const:`None`, and it will be set as :attr:`__cause__` on the
+raised exception.  If :attr:`__cause__` is an exception it will be displayed
+instead of :attr:`__context__`; if :attr:`__cause__` is None,
+:attr:`__context__` will not be displayed by the default exception handling
+code.  (Note:  the default value for :attr:`__context__` is :const:`None`,
+while the default value for :attr:`__cause__` is :const:`Ellipsis`.)
+
+In either case, the default exception handling code will not display
+any of the remaining links in the :attr:`__context__` chain if
+:attr:`__cause__` has been set.
+
 
 Base classes
 ------------

Doc/library/importlib.rst

 
 .. function:: invalidate_caches()
 
-   Invalidate importlib's internal caches.  Calling this function may be
-   needed if some modules are installed while your program is running and
-   you expect the program to notice the changes.
+   Invalidate the internal caches of the finders stored at
+   :data:`sys.path_importer_cache`. If a finder implements
+   :meth:`abc.Finder.invalidate_caches()` then it will be called to perform the
+   invalidation.  This function may be needed if some modules are installed
+   while your program is running and you expect the program to notice the
+   changes.
 
    .. versionadded:: 3.3
 
         be the value of :attr:`__path__` from the parent package. If a loader
         cannot be found, ``None`` is returned.
 
+   .. method:: invalidate_caches()
+
+        An optional method which, when called, should invalidate any internal
+        cache used by the finder. Used by :func:`invalidate_caches()` when
+        invalidating the caches of all cached finders.
+
 
 .. class:: Loader
 

Doc/library/logging.rst

    If *capture* is ``True``, warnings issued by the :mod:`warnings` module will
    be redirected to the logging system. Specifically, a warning will be
    formatted using :func:`warnings.formatwarning` and the resulting string
-   logged to a logger named ``'py.warnings'`` with a severity of ``'WARNING'``.
+   logged to a logger named ``'py.warnings'`` with a severity of :const:`WARNING`.
 
    If *capture* is ``False``, the redirection of warnings to the logging system
    will stop, and warnings will be redirected to their original destinations

Doc/library/operator.rst

 +-----------------------+-------------------------+---------------------------------------+
 | Containment Test      | ``obj in seq``          | ``contains(seq, obj)``                |
 +-----------------------+-------------------------+---------------------------------------+
-| Division              | ``a / b``               | ``div(a, b)``                         |
+| Division              | ``a / b``               | ``truediv(a, b)``                     |
 +-----------------------+-------------------------+---------------------------------------+
 | Division              | ``a // b``              | ``floordiv(a, b)``                    |
 +-----------------------+-------------------------+---------------------------------------+

Doc/library/re.rst

    Matches the empty string, but only at the beginning or end of a word.
    A word is defined as a sequence of Unicode alphanumeric or underscore
    characters, so the end of a word is indicated by whitespace or a
-   non-alphanumeric, non-underscore Unicode character. Note that
-   formally, ``\b`` is defined as the boundary between a ``\w`` and a
-   ``\W`` character (or vice versa). By default Unicode alphanumerics
-   are the ones used, but this can be changed by using the :const:`ASCII`
-   flag.  Inside a character range, ``\b`` represents the backspace
-   character, for compatibility with Python's string literals.
+   non-alphanumeric, non-underscore Unicode character.  Note that formally,
+   ``\b`` is defined as the boundary between a ``\w`` and a ``\W`` character
+   (or vice versa), or between ``\w`` and the beginning/end of the string.
+   This means that ``r'\bfoo\b'`` matches ``'foo'``, ``'foo.'``, ``'(foo)'``,
+   ``'bar foo baz'`` but not ``'foobar'`` or ``'foo3'``.
+
+   By default Unicode alphanumerics are the ones used, but this can  be changed
+   by using the :const:`ASCII` flag.  Inside a character range, ``\b``
+   represents the backspace character, for compatibility with Python's string
+   literals.
 
 ``\B``
-   Matches the empty string, but only when it is *not* at the beginning or end of a
-   word.  This is just the opposite of ``\b``, so word characters are
+   Matches the empty string, but only when it is *not* at the beginning or end
+   of a word.  This means that ``r'py\B'`` matches ``'python'``, ``'py3'``,
+   ``'py2'``, but not ``'py'``, ``'py.'``, or ``'py!'``.
+   ``\B`` is just the opposite of ``\b``, so word characters are
    Unicode alphanumerics or the underscore, although this can be changed
    by using the :const:`ASCII` flag.
 

Doc/library/stdtypes.rst

    two empty strings, followed by the string itself.
 
 
-.. method:: str.rsplit([sep[, maxsplit]])
+.. method:: str.rsplit(sep=None, maxsplit=-1)
 
    Return a list of the words in the string, using *sep* as the delimiter string.
    If *maxsplit* is given, at most *maxsplit* splits are done, the *rightmost*
       'mississ'
 
 
-.. method:: str.split([sep[, maxsplit]])
+.. method:: str.split(sep=None, maxsplit=-1)
 
    Return a list of the words in the string, using *sep* as the delimiter
    string.  If *maxsplit* is given, at most *maxsplit* splits are done (thus,
 
 .. note::
 
-   The formatting operations described here are obsolete and may go away in future
-   versions of Python.  Use the new :ref:`string-formatting` in new code.
+   The formatting operations described here are modelled on C's printf()
+   syntax.  They only support formatting of certain builtin types.  The
+   use of a binary operator means that care may be needed in order to
+   format tuples and dictionaries correctly.  As the new
+   :ref:`string-formatting` syntax is more flexible and handles tuples and
+   dictionaries naturally, it is recommended for new code.  However, there
+   are no current plans to deprecate printf-style formatting.
 
 String objects have one unique built-in operation: the ``%`` operator (modulo).
 This is also known as the string *formatting* or *interpolation* operator.
 
 :class:`memoryview` objects allow Python code to access the internal data
 of an object that supports the :ref:`buffer protocol <bufferobjects>` without
-copying.  Memory is generally interpreted as simple bytes.
+copying.
 
 .. class:: memoryview(obj)
 
    is a single byte, but other types such as :class:`array.array` may have
    bigger elements.
 
-   ``len(view)`` returns the total number of elements in the memoryview,
-   *view*.  The :class:`~memoryview.itemsize` attribute will give you the
+   ``len(view)`` is equal to the length of :class:`~memoryview.tolist`.
+   If ``view.ndim = 0``, the length is 1. If ``view.ndim = 1``, the length
+   is equal to the number of elements in the view. For higher dimensions,
+   the length is equal to the length of the nested list representation of
+   the view. The :class:`~memoryview.itemsize` attribute will give you the
    number of bytes in a single element.
 
-   A :class:`memoryview` supports slicing to expose its data.  Taking a single
-   index will return a single element as a :class:`bytes` object.  Full
-   slicing will result in a subview::
-
-      >>> v = memoryview(b'abcefg')
-      >>> v[1]
-      b'b'
-      >>> v[-1]
-      b'g'
-      >>> v[1:4]
-      <memory at 0x77ab28>
-      >>> bytes(v[1:4])
-      b'bce'
-
-   If the object the memoryview is over supports changing its data, the
-   memoryview supports slice assignment::
+   A :class:`memoryview` supports slicing to expose its data. If
+   :class:`~memoryview.format` is one of the native format specifiers
+   from the :mod:`struct` module, indexing will return a single element
+   with the correct type. Full slicing will result in a subview::
+
+    >>> v = memoryview(b'abcefg')
+    >>> v[1]
+    98
+    >>> v[-1]
+    103
+    >>> v[1:4]
+    <memory at 0x7f3ddc9f4350>
+    >>> bytes(v[1:4])
+    b'bce'
+
+   Other native formats::
+
+      >>> import array
+      >>> a = array.array('l', [-11111111, 22222222, -33333333, 44444444])
+      >>> a[0]
+      -11111111
+      >>> a[-1]
+      44444444
+      >>> a[2:3].tolist()
+      [-33333333]
+      >>> a[::2].tolist()
+      [-11111111, -33333333]
+      >>> a[::-1].tolist()
+      [44444444, -33333333, 22222222, -11111111]
+
+   .. versionadded:: 3.3
+
+   If the underlying object is writable, the memoryview supports slice
+   assignment. Resizing is not allowed::
 
       >>> data = bytearray(b'abcefg')
       >>> v = memoryview(data)
       >>> v.readonly
       False
-      >>> v[0] = b'z'
+      >>> v[0] = ord(b'z')
       >>> data
       bytearray(b'zbcefg')
       >>> v[1:4] = b'123'
       >>> data
       bytearray(b'z123fg')
-      >>> v[2] = b'spam'
+      >>> v[2:3] = b'spam'
       Traceback (most recent call last):
-        File "<stdin>", line 1, in <module>
-      ValueError: cannot modify size of memoryview object
-
-   Notice how the size of the memoryview object cannot be changed.
-
-   Memoryviews of hashable (read-only) types are also hashable and their
-   hash value matches the corresponding bytes object::
+      File "<stdin>", line 1, in <module>
+      ValueError: memoryview assignment: lvalue and rvalue have different structures
+      >>> v[2:6] = b'spam'
+      >>> data
+      bytearray(b'z1spam')
+
+   Memoryviews of hashable (read-only) types are also hashable. The hash
+   is defined as ``hash(m) == hash(m.tobytes())``::
 
       >>> v = memoryview(b'abcefg')
       >>> hash(v) == hash(b'abcefg')
       True
       >>> hash(v[2:4]) == hash(b'ce')
       True
+      >>> hash(v[::-2]) == hash(b'abcefg'[::-2])
+      True
+
+   Hashing of multi-dimensional objects is supported::
+
+      >>> buf = bytes(list(range(12)))
+      >>> x = memoryview(buf)
+      >>> y = x.cast('B', shape=[2,2,3])
+      >>> x.tolist()
+      [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
+      >>> y.tolist()
+      [[[0, 1, 2], [3, 4, 5]], [[6, 7, 8], [9, 10, 11]]]
+      >>> hash(x) == hash(y) == hash(y.tobytes())
+      True
 
    .. versionchanged:: 3.3
       Memoryview objects are now hashable.
          >>> bytes(m)
          b'abc'
 
+      For non-contiguous arrays the result is equal to the flattened list
+      representation with all elements converted to bytes.
+
    .. method:: tolist()
 
-      Return the data in the buffer as a list of integers. ::
+      Return the data in the buffer as a list of elements. ::
 
          >>> memoryview(b'abc').tolist()
          [97, 98, 99]
+         >>> import array
+         >>> a = array.array('d', [1.1, 2.2, 3.3])
+         >>> m = memoryview(a)
+         >>> m.tolist()
+         [1.1, 2.2, 3.3]
 
    .. method:: release()
 
          >>> with memoryview(b'abc') as m:
          ...     m[0]
          ...
-         b'a'
+         97
          >>> m[0]
          Traceback (most recent call last):
            File "<stdin>", line 1, in <module>
 
       .. versionadded:: 3.2
 
+   .. method:: cast(format[, shape])
+
+      Cast a memoryview to a new format or shape. *shape* defaults to
+      ``[byte_length//new_itemsize]``, which means that the result view
+      will be one-dimensional. The return value is a new memoryview, but
+      the buffer itself is not copied. Supported casts are 1D -> C-contiguous
+      and C-contiguous -> 1D. One of the formats must be a byte format
+      ('B', 'b' or 'c'). The byte length of the result must be the same
+      as the original length.
+
+      Cast 1D/long to 1D/unsigned bytes::
+
+         >>> import array
+         >>> a = array.array('l', [1,2,3])
+         >>> x = memoryview(a)
+         >>> x.format
+         'l'
+         >>> x.itemsize
+         8
+         >>> len(x)
+         3
+         >>> x.nbytes
+         24
+         >>> y = x.cast('B')
+         >>> y.format
+         'B'
+         >>> y.itemsize
+         1
+         >>> len(y)
+         24
+         >>> y.nbytes
+         24
+
+      Cast 1D/unsigned bytes to 1D/char::
+
+         >>> b = bytearray(b'zyz')
+         >>> x = memoryview(b)
+         >>> x[0] = b'a'
+         Traceback (most recent call last):
+           File "<stdin>", line 1, in <module>
+         ValueError: memoryview: invalid value for format "B"
+         >>> y = x.cast('c')
+         >>> y[0] = b'a'
+         >>> b
+         bytearray(b'ayz')
+
+      Cast 1D/bytes to 3D/ints to 1D/signed char::
+
+         >>> import struct
+         >>> buf = struct.pack("i"*12, *list(range(12)))
+         >>> x = memoryview(buf)
+         >>> y = x.cast('i', shape=[2,2,3])
+         >>> y.tolist()
+         [[[0, 1, 2], [3, 4, 5]], [[6, 7, 8], [9, 10, 11]]]
+         >>> y.format
+         'i'
+         >>> y.itemsize
+         4
+         >>> len(y)
+         2
+         >>> y.nbytes
+         48
+         >>> z = y.cast('b')
+         >>> z.format
+         'b'
+         >>> z.itemsize
+         1
+         >>> len(z)
+         48
+         >>> z.nbytes
+         48
+
+      Cast 1D/unsigned char to to 2D/unsigned long::
+
+         >>> buf = struct.pack("L"*6, *list(range(6)))
+         >>> x = memoryview(buf)
+         >>> y = x.cast('L', shape=[2,3])
+         >>> len(y)
+         2
+         >>> y.nbytes
+         48
+         >>> y.tolist()
+         [[0, 1, 2], [3, 4, 5]]
+
+      .. versionadded:: 3.3