Source

python-peps / pep-0362.txt

The default branch has multiple heads

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
PEP: 362
Title: Function Signature Object
Version: $Revision$
Last-Modified: $Date$
Author: Brett Cannon <brett@python.org>, Jiwon Seo <seojiwon@gmail.com>,
        Yury Selivanov <yselivanov@sprymix.com>, Larry Hastings <larry@hastings.org>
Status: Final
Type: Standards Track
Content-Type: text/x-rst
Created: 21-Aug-2006
Python-Version: 3.3
Post-History: 04-Jun-2012
Resolution: http://mail.python.org/pipermail/python-dev/2012-June/120682.html


Abstract
========

Python has always supported powerful introspection capabilities,
including introspecting functions and methods (for the rest of
this PEP, "function" refers to both functions and methods).  By
examining a function object you can fully reconstruct the function's
signature.  Unfortunately this information is stored in an inconvenient
manner, and is spread across a half-dozen deeply nested attributes.

This PEP proposes a new representation for function signatures.
The new representation contains all necessary information about a function
and its parameters, and makes introspection easy and straightforward.

However, this object does not replace the existing function
metadata, which is used by Python itself to execute those
functions.  The new metadata object is intended solely to make
function introspection easier for Python programmers.


Signature Object
================

A Signature object represents the call signature of a function and
its return annotation.  For each parameter accepted by the function
it stores a `Parameter object`_ in its ``parameters`` collection.

A Signature object has the following public attributes and methods:

* return_annotation : object
    The "return" annotation for the function. If the function
    has no "return" annotation, this attribute is set to
    ``Signature.empty``.

* parameters : OrderedDict
    An ordered mapping of parameters' names to the corresponding
    Parameter objects.

* bind(\*args, \*\*kwargs) -> BoundArguments
    Creates a mapping from positional and keyword arguments to
    parameters.  Raises a ``TypeError`` if the passed arguments do
    not match the signature.

* bind_partial(\*args, \*\*kwargs) -> BoundArguments
    Works the same way as ``bind()``, but allows the omission
    of some required arguments (mimics ``functools.partial``
    behavior.)  Raises a ``TypeError`` if the passed arguments do
    not match the signature.

* replace(parameters=<optional>, \*, return_annotation=<optional>) -> Signature
    Creates a new Signature instance based on the instance
    ``replace`` was invoked on.  It is possible to pass different
    ``parameters`` and/or ``return_annotation`` to override the
    corresponding properties of the base signature.  To remove
    ``return_annotation`` from the copied ``Signature``, pass in
    ``Signature.empty``.

    Note that the '=<optional>' notation, means that the argument is
    optional.  This notation applies to the rest of this PEP.

Signature objects are immutable.  Use ``Signature.replace()`` to
make a modified copy:
::

    >>> def foo() -> None:
    ...     pass
    >>> sig = signature(foo)

    >>> new_sig = sig.replace(return_annotation="new return annotation")
    >>> new_sig is not sig
    True
    >>> new_sig.return_annotation != sig.return_annotation
    True
    >>> new_sig.parameters == sig.parameters
    True

    >>> new_sig = new_sig.replace(return_annotation=new_sig.empty)
    >>> new_sig.return_annotation is Signature.empty
    True

There are two ways to instantiate a Signature class:

* Signature(parameters=<optional>, \*, return_annotation=Signature.empty)
    Default Signature constructor.  Accepts an optional sequence
    of ``Parameter`` objects, and an optional ``return_annotation``.
    Parameters sequence is validated to check that there are no
    parameters with duplicate names, and that the parameters
    are in the right order, i.e. positional-only first, then
    positional-or-keyword, etc.
* Signature.from_function(function)
    Returns a Signature object reflecting the signature of the
    function passed in.

It's possible to test Signatures for equality.  Two signatures are
equal when their parameters are equal, their positional and
positional-only parameters appear in the same order, and they
have equal return annotations.

Changes to the Signature object, or to any of its data members,
do not affect the function itself.

Signature also implements ``__str__``:
::

    >>> str(Signature.from_function((lambda *args: None)))
    '(*args)'

    >>> str(Signature())
    '()'


Parameter Object
================

Python's expressive syntax means functions can accept many different
kinds of parameters with many subtle semantic differences.  We
propose a rich Parameter object designed to represent any possible
function parameter.

A Parameter object has the following public attributes and methods:

* name : str
    The name of the parameter as a string.  Must be a valid
    python identifier name (with the exception of ``POSITIONAL_ONLY``
    parameters, which can have it set to ``None``.)

* default : object
    The default value for the parameter.  If the parameter has no
    default value, this attribute is set to ``Parameter.empty``.

* annotation : object
    The annotation for the parameter.  If the parameter has no
    annotation, this attribute is set to ``Parameter.empty``.

* kind
    Describes how argument values are bound to the parameter.
    Possible values:

       * ``Parameter.POSITIONAL_ONLY`` - value must be supplied
         as a positional argument.

         Python has no explicit syntax for defining positional-only
         parameters, but many built-in and extension module functions
         (especially those that accept only one or two parameters)
         accept them.

       * ``Parameter.POSITIONAL_OR_KEYWORD`` - value may be
         supplied as either a keyword or positional argument
         (this is the standard binding behaviour for functions
         implemented in Python.)

       * ``Parameter.KEYWORD_ONLY`` - value must be supplied
         as a keyword argument.  Keyword only parameters are those
         which appear after a "*" or "\*args" entry in a Python
         function definition.

       * ``Parameter.VAR_POSITIONAL`` - a tuple of positional
         arguments that aren't bound to any other parameter.
         This corresponds to a "\*args" parameter in a Python
         function definition.

       * ``Parameter.VAR_KEYWORD`` - a dict of keyword arguments
         that aren't bound to any other parameter. This corresponds
         to a "\*\*kwargs" parameter in a Python function definition.

    Always use ``Parameter.*`` constants for setting and checking
    value of the ``kind`` attribute.

* replace(\*, name=<optional>, kind=<optional>, default=<optional>, annotation=<optional>) -> Parameter
    Creates a new Parameter instance based on the instance
    ``replaced`` was invoked on.  To override a Parameter
    attribute, pass the corresponding argument.  To remove
    an attribute from a ``Parameter``, pass ``Parameter.empty``.


Parameter constructor:

* Parameter(name, kind, \*, annotation=Parameter.empty, default=Parameter.empty)
   Instantiates a Parameter object. ``name`` and ``kind`` are required,
   while ``annotation`` and ``default`` are optional.

Two parameters are equal when they have equal names, kinds, defaults,
and annotations.

Parameter objects are immutable.  Instead of modifying a Parameter object,
you can use ``Parameter.replace()`` to create a modified copy like so:
::

    >>> param = Parameter('foo', Parameter.KEYWORD_ONLY, default=42)
    >>> str(param)
    'foo=42'

    >>> str(param.replace())
    'foo=42'

    >>> str(param.replace(default=Parameter.empty, annotation='spam'))
    "foo:'spam'"


BoundArguments Object
=====================

Result of a ``Signature.bind`` call.  Holds the mapping of arguments
to the function's parameters.

Has the following public attributes:

* arguments : OrderedDict
    An ordered, mutable mapping of parameters' names to arguments' values.
    Contains only explicitly bound arguments.  Arguments for
    which ``bind()`` relied on a default value are skipped.
* args : tuple
    Tuple of positional arguments values.  Dynamically computed from
    the 'arguments' attribute.
* kwargs : dict
    Dict of keyword arguments values. Dynamically computed from
    the 'arguments' attribute.

The ``arguments`` attribute should be used in conjunction with
``Signature.parameters`` for any arguments processing purposes.

``args`` and ``kwargs`` properties can be used to invoke functions:
::

    def test(a, *, b):
        ...

    sig = signature(test)
    ba = sig.bind(10, b=20)
    test(*ba.args, **ba.kwargs)

Arguments which could be passed as part of either ``*args`` or ``**kwargs``
will be included only in the ``BoundArguments.args`` attribute.  Consider the
following example:
::

    def test(a=1, b=2, c=3):
        pass

    sig = signature(test)
    ba = sig.bind(a=10, c=13)

    >>> ba.args
    (10,)

    >>> ba.kwargs:
    {'c': 13}


Implementation
==============

The implementation adds a new function ``signature()`` to the ``inspect``
module.  The function is the preferred way of getting a ``Signature`` for
a callable object.

The function implements the following algorithm:

    - If the object is not callable - raise a TypeError

    - If the object has a ``__signature__`` attribute and if it
      is not ``None`` - return it

    - If it has a ``__wrapped__`` attribute, return
      ``signature(object.__wrapped__)``

    - If the object is a an instance of ``FunctionType`` construct
      and return a new ``Signature`` for it

    - If the object is a bound method, construct and return a new ``Signature``
      object, with its first parameter (usually ``self`` or ``cls``)
      removed.  (``classmethod`` and ``staticmethod`` are supported
      too.  Since both are descriptors, the former returns a bound method,
      and the latter returns its wrapped function.)

    - If the object is an instance of ``functools.partial``, construct
      a new ``Signature`` from its ``partial.func`` attribute, and
      account for already bound ``partial.args`` and ``partial.kwargs``

    - If the object is a class or metaclass:

        - If the object's type has a ``__call__`` method defined in
          its MRO, return a Signature for it

        - If the object has a ``__new__`` method defined in its MRO,
          return a Signature object for it

        - If the object has a ``__init__`` method defined in its MRO,
          return a Signature object for it

    - Return ``signature(object.__call__)``

Note that the ``Signature`` object is created in a lazy manner, and
is not automatically cached.  However, the user can manually cache a
Signature by storing it in the ``__signature__`` attribute.

An implementation for Python 3.3 can be found at [#impl]_.
The python issue tracking the patch is [#issue]_.


Design Considerations
=====================

No implicit caching of Signature objects
----------------------------------------

The first PEP design had a provision for implicit caching of ``Signature``
objects in the ``inspect.signature()`` function.  However, this has the
following downsides:

  * If the ``Signature`` object is cached then any changes to the function
    it describes will not be reflected in it.  However, If the caching is
    needed, it can be always done manually and explicitly

  * It is better to reserve the ``__signature__`` attribute for the cases
    when there is a need to explicitly set to a ``Signature`` object that
    is different from the actual one


Some functions may not be introspectable
----------------------------------------

Some functions may not be introspectable in certain implementations of
Python.  For example, in CPython, built-in functions defined in C provide
no metadata about their arguments.  Adding support for them is out of
scope for this PEP.


Signature and Parameter equivalence
-----------------------------------

We assume that parameter names have semantic significance--two
signatures are equal only when their corresponding parameters are equal
and have the exact same names.  Users who want looser equivalence tests,
perhaps ignoring names of VAR_KEYWORD or VAR_POSITIONAL parameters, will
need to implement those themselves.


Examples
========

Visualizing Callable Objects' Signature
---------------------------------------

Let's define some classes and functions:

::

    from inspect import signature
    from functools import partial, wraps


    class FooMeta(type):
        def __new__(mcls, name, bases, dct, *, bar:bool=False):
            return super().__new__(mcls, name, bases, dct)

        def __init__(cls, name, bases, dct, **kwargs):
            return super().__init__(name, bases, dct)


    class Foo(metaclass=FooMeta):
        def __init__(self, spam:int=42):
            self.spam = spam

        def __call__(self, a, b, *, c) -> tuple:
            return a, b, c

        @classmethod
        def spam(cls, a):
            return a


    def shared_vars(*shared_args):
        """Decorator factory that defines shared variables that are
           passed to every invocation of the function"""

        def decorator(f):
            @wraps(f)
            def wrapper(*args, **kwargs):
                full_args = shared_args + args
                return f(*full_args, **kwargs)

            # Override signature
            sig = signature(f)
            sig = sig.replace(tuple(sig.parameters.values())[1:])
            wrapper.__signature__ = sig

            return wrapper
        return decorator


    @shared_vars({})
    def example(_state, a, b, c):
        return _state, a, b, c


    def format_signature(obj):
        return str(signature(obj))


Now, in the python REPL:

::

    >>> format_signature(FooMeta)
    '(name, bases, dct, *, bar:bool=False)'

    >>> format_signature(Foo)
    '(spam:int=42)'

    >>> format_signature(Foo.__call__)
    '(self, a, b, *, c) -> tuple'

    >>> format_signature(Foo().__call__)
    '(a, b, *, c) -> tuple'

    >>> format_signature(Foo.spam)
    '(a)'

    >>> format_signature(partial(Foo().__call__, 1, c=3))
    '(b, *, c=3) -> tuple'

    >>> format_signature(partial(partial(Foo().__call__, 1, c=3), 2, c=20))
    '(*, c=20) -> tuple'

    >>> format_signature(example)
    '(a, b, c)'

    >>> format_signature(partial(example, 1, 2))
    '(c)'

    >>> format_signature(partial(partial(example, 1, b=2), c=3))
    '(b=2, c=3)'


Annotation Checker
------------------
::

    import inspect
    import functools

    def checktypes(func):
        '''Decorator to verify arguments and return types

        Example:

            >>> @checktypes
            ... def test(a:int, b:str) -> int:
            ...     return int(a * b)

            >>> test(10, '1')
            1111111111

            >>> test(10, 1)
            Traceback (most recent call last):
              ...
            ValueError: foo: wrong type of 'b' argument, 'str' expected, got 'int'
        '''

        sig = inspect.signature(func)

        types = {}
        for param in sig.parameters.values():
            # Iterate through function's parameters and build the list of
            # arguments types
            type_ = param.annotation
            if type_ is param.empty or not inspect.isclass(type_):
                # Missing annotation or not a type, skip it
                continue

            types[param.name] = type_

            # If the argument has a type specified, let's check that its
            # default value (if present) conforms with the type.
            if param.default is not param.empty and not isinstance(param.default, type_):
                raise ValueError("{func}: wrong type of a default value for {arg!r}". \
                                 format(func=func.__qualname__, arg=param.name))

        def check_type(sig, arg_name, arg_type, arg_value):
            # Internal function that encapsulates arguments type checking
            if not isinstance(arg_value, arg_type):
                raise ValueError("{func}: wrong type of {arg!r} argument, " \
                                 "{exp!r} expected, got {got!r}". \
                                 format(func=func.__qualname__, arg=arg_name,
                                        exp=arg_type.__name__, got=type(arg_value).__name__))

        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            # Let's bind the arguments
            ba = sig.bind(*args, **kwargs)
            for arg_name, arg in ba.arguments.items():
                # And iterate through the bound arguments
                try:
                    type_ = types[arg_name]
                except KeyError:
                    continue
                else:
                    # OK, we have a type for the argument, lets get the corresponding
                    # parameter description from the signature object
                    param = sig.parameters[arg_name]
                    if param.kind == param.VAR_POSITIONAL:
                        # If this parameter is a variable-argument parameter,
                        # then we need to check each of its values
                        for value in arg:
                            check_type(sig, arg_name, type_, value)
                    elif param.kind == param.VAR_KEYWORD:
                        # If this parameter is a variable-keyword-argument parameter:
                        for subname, value in arg.items():
                            check_type(sig, arg_name + ':' + subname, type_, value)
                    else:
                        # And, finally, if this parameter a regular one:
                        check_type(sig, arg_name, type_, arg)

            result = func(*ba.args, **ba.kwargs)

            # The last bit - let's check that the result is correct
            return_type = sig.return_annotation
            if (return_type is not sig._empty and
                    isinstance(return_type, type) and
                    not isinstance(result, return_type)):

                raise ValueError('{func}: wrong return type, {exp} expected, got {got}'. \
                                 format(func=func.__qualname__, exp=return_type.__name__,
                                        got=type(result).__name__))
            return result

        return wrapper

Acceptance
==========

PEP 362 was accepted by Guido, Friday, June 22, 2012 [#accepted]_ .
The reference implementation was committed to trunk later that day.


References
==========

.. [#impl] pep362 branch (https://bitbucket.org/1st1/cpython/overview)
.. [#issue] issue 15008 (http://bugs.python.org/issue15008)
.. [#accepted] "A Desperate Plea For Introspection (aka: BDFAP Needed)" (http://mail.python.org/pipermail/python-dev/2012-June/120682.html)


Copyright
=========

This document has been placed in the public domain.


..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End:
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.