Source

python-peps / pep-0231.txt

The default branch has multiple heads

Full commit
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
PEP: 231
Title: __findattr__()
Version: $Revision$
Last-Modified: $Date$
Author: barry@python.org (Barry Warsaw)
Status: Rejected
Type: Standards Track
Created: 30-Nov-2000
Python-Version: 2.1
Post-History:


Introduction

    This PEP describes an extension to instance attribute lookup and
    modification machinery, which allows pure-Python implementations
    of many interesting programming models.  This PEP tracks the
    status and ownership of this feature.  It contains a description
    of the feature and outlines changes necessary to support the
    feature.  This PEP summarizes discussions held in mailing list
    forums, and provides URLs for further information, where
    appropriate.  The CVS revision history of this file contains the
    definitive historical record.


Background

    The semantics for Python instances allow the programmer to
    customize some aspects of attribute lookup and attribute
    modification, through the special methods __getattr__() and
    __setattr__() [1].

    However, because of certain restrictions imposed by these methods,
    there are useful programming techniques that can not be written in
    Python alone, e.g. strict Java Bean-like[2] interfaces and Zope
    style acquisitions[3].  In the latter case, Zope solves this by
    including a C extension called ExtensionClass[5] which modifies
    the standard class semantics, and uses a metaclass hook in
    Python's class model called alternatively the "Don Beaudry Hook"
    or "Don Beaudry Hack"[6].

    While Zope's approach works, it has several disadvantages.  First,
    it requires a C extension.  Second it employs a very arcane, but
    truck-sized loophole in the Python machinery.  Third, it can be
    difficult for other programmers to use and understand (the
    metaclass has well-known brain exploding properties).  And fourth,
    because ExtensionClass instances aren't "real" Python instances,
    some aspects of the Python runtime system don't work with
    ExtensionClass instances.

    Proposals for fixing this problem have often been lumped under the
    rubric of fixing the "class/type dichotomy"; that is, eliminating
    the difference between built-in types and classes[7].  While a
    laudable goal itself, repairing this rift is not necessary in
    order to achieve the types of programming constructs described
    above.  This proposal provides an 80% solution with a minimum of
    modification to Python's class and instance objects.  It does
    nothing to address the type/class dichotomy.


Proposal

    This proposal adds a new special method called __findattr__() with
    the following semantics:

    * If defined in a class, it will be called on all instance
      attribute resolutions instead of __getattr__() and
      __setattr__().

    * __findattr__() is never called recursively.  That is, when a
      specific instance's __findattr__() is on the call stack, further
      attribute accesses for that instance will use the standard
      __getattr__() and __setattr__() methods.

    * __findattr__() is called for both attribute access (`getting')
      and attribute modification (`setting').  It is not called for
      attribute deletion.

    * When called for getting, it is passed a single argument (not
      counting `self'): the name of the attribute being accessed.

    * When called for setting, it is called with third argument, which
      is the value to set the attribute to.

    * __findattr__() methods have the same caching semantics as
      __getattr__() and __setattr__(); i.e. if they are present in the
      class at class definition time, they are used, but if they are
      subsequently added to a class later they are not.


Key Differences with the Existing Protocol

    __findattr__()'s semantics are different from the existing
    protocol in key ways:

    First, __getattr__() is never called if the attribute is found in
    the instance's __dict__.  This is done for efficiency reasons, and
    because otherwise, __setattr__() would have no way to get to the
    instance's attributes.

    Second, __setattr__() cannot use "normal" syntax for setting
    instance attributes, e.g. "self.name = foo" because that would
    cause recursive calls to __setattr__().

    __findattr__() is always called regardless of whether the
    attribute is in __dict__ or not, and a flag in the instance object
    prevents recursive calls to __findattr__().  This gives the class
    a chance to perform some action for every attribute access.  And
    because it is called for both gets and sets, it is easy to write
    similar policy for all attribute access.  Further, efficiency is
    not a problem because it is only paid when the extended mechanism
    is used.


Related Work

    PEP 213 [9] describes a different approach to hooking into
    attribute access and modification.  The semantics proposed in PEP
    213 can be implemented using the __findattr__() hook described
    here, with one caveat.  The current reference implementation of
    __findattr__() does not support hooking on attribute deletion.
    This could be added if it's found desirable.  See example below.


Examples

    One programming style that this proposal allows is a Java
    Bean-like interface to objects, where unadorned attribute access
    and modification is transparently mapped to a functional
    interface.  E.g.

        class Bean:
            def __init__(self, x):
                self.__myfoo = x

            def __findattr__(self, name, *args):
                if name.startswith('_'):
                    # Private names
                    if args: setattr(self, name, args[0])
                    else:    return getattr(self, name)
                else:
                    # Public names
                    if args: name = '_set_' + name
                    else:    name = '_get_' + name
                    return getattr(self, name)(*args)

            def _set_foo(self, x):
                self.__myfoo = x

            def _get_foo(self):
                return self.__myfoo


        b = Bean(3)
        print b.foo
        b.foo = 9
        print b.foo
    

    A second, more elaborate example is the implementation of both
    implicit and explicit acquisition in pure Python:

        import types

        class MethodWrapper:
            def __init__(self, container, method):
                self.__container = container
                self.__method = method

            def __call__(self, *args, **kws):
                return self.__method.im_func(self.__container, *args, **kws)


        class WrapperImplicit:
            def __init__(self, contained, container):
                self.__contained = contained
                self.__container = container

            def __repr__(self):
                return '<Wrapper: [%s | %s]>' % (self.__container,
                                                 self.__contained)

            def __findattr__(self, name, *args):
                # Some things are our own
                if name.startswith('_WrapperImplicit__'):
                    if args: return setattr(self, name, *args)
                    else:    return getattr(self, name)
                # setattr stores the name on the contained object directly
                if args:
                    return setattr(self.__contained, name, args[0])
                # Other special names
                if name == 'aq_parent':
                    return self.__container
                elif name == 'aq_self':
                    return self.__contained
                elif name == 'aq_base':
                    base = self.__contained
                    try:
                        while 1:
                            base = base.aq_self
                    except AttributeError:
                        return base
                # no acquisition for _ names
                if name.startswith('_'):
                    return getattr(self.__contained, name)
                # Everything else gets wrapped
                missing = []
                which = self.__contained
                obj = getattr(which, name, missing)
                if obj is missing:
                    which = self.__container
                    obj = getattr(which, name, missing)
                    if obj is missing:
                        raise AttributeError, name
                of = getattr(obj, '__of__', missing)
                if of is not missing:
                    return of(self)
                elif type(obj) == types.MethodType:
                    return MethodWrapper(self, obj)
                return obj


        class WrapperExplicit:
            def __init__(self, contained, container):
                self.__contained = contained
                self.__container = container

            def __repr__(self):
                return '<Wrapper: [%s | %s]>' % (self.__container,
                                                 self.__contained)

            def __findattr__(self, name, *args):
                # Some things are our own
                if name.startswith('_WrapperExplicit__'):
                    if args: return setattr(self, name, *args)
                    else:    return getattr(self, name)
                # setattr stores the name on the contained object directly
                if args:
                    return setattr(self.__contained, name, args[0])
                # Other special names
                if name == 'aq_parent':
                    return self.__container
                elif name == 'aq_self':
                    return self.__contained
                elif name == 'aq_base':
                    base = self.__contained
                    try:
                        while 1:
                            base = base.aq_self
                    except AttributeError:
                        return base
                elif name == 'aq_acquire':
                    return self.aq_acquire
                # explicit acquisition only
                obj = getattr(self.__contained, name)
                if type(obj) == types.MethodType:
                    return MethodWrapper(self, obj)
                return obj

            def aq_acquire(self, name):
                # Everything else gets wrapped
                missing = []
                which = self.__contained
                obj = getattr(which, name, missing)
                if obj is missing:
                    which = self.__container
                    obj = getattr(which, name, missing)
                    if obj is missing:
                        raise AttributeError, name
                of = getattr(obj, '__of__', missing)
                if of is not missing:
                    return of(self)
                elif type(obj) == types.MethodType:
                    return MethodWrapper(self, obj)
                return obj


        class Implicit:
            def __of__(self, container):
                return WrapperImplicit(self, container)

            def __findattr__(self, name, *args):
                # ignore setattrs
                if args:
                    return setattr(self, name, args[0])
                obj = getattr(self, name)
                missing = []
                of = getattr(obj, '__of__', missing)
                if of is not missing:
                    return of(self)
                return obj


        class Explicit(Implicit):
            def __of__(self, container):
                return WrapperExplicit(self, container)


        # tests
        class C(Implicit):
            color = 'red'

        class A(Implicit):
            def report(self):
                return self.color

        # simple implicit acquisition
        c = C()
        a = A()
        c.a = a
        assert c.a.report() == 'red'

        d = C()
        d.color = 'green'
        d.a = a
        assert d.a.report() == 'green'

        try:
            a.report()
        except AttributeError:
            pass
        else:
            assert 0, 'AttributeError expected'


        # special names
        assert c.a.aq_parent is c
        assert c.a.aq_self is a

        c.a.d = d
        assert c.a.d.aq_base is d
        assert c.a is not a


        # no acquisiton on _ names
        class E(Implicit):
            _color = 'purple'

        class F(Implicit):
            def report(self):
                return self._color

        e = E()
        f = F()
        e.f = f
        try:
            e.f.report()
        except AttributeError:
            pass
        else:
            assert 0, 'AttributeError expected'


        # explicit
        class G(Explicit):
            color = 'pink'

        class H(Explicit):
            def report(self):
                return self.aq_acquire('color')

            def barf(self):
                return self.color

        g = G()
        h = H()
        g.h = h
        assert g.h.report() == 'pink'

        i = G()
        i.color = 'cyan'
        i.h = h
        assert i.h.report() == 'cyan'

        try:
            g.i.barf()
        except AttributeError:
            pass
        else:
            assert 0, 'AttributeError expected'
    

    C++-like access control can also be accomplished, although less
    cleanly because of the difficulty of figuring out what method is
    being called from the runtime call stack:

        import sys
        import types

        PUBLIC = 0
        PROTECTED = 1
        PRIVATE = 2

        try:
            getframe = sys._getframe
        except ImportError:
            def getframe(n):
                try: raise Exception
                except Exception:
                    frame = sys.exc_info()[2].tb_frame
                while n > 0:
                    frame = frame.f_back
                    if frame is None:
                        raise ValueError, 'call stack is not deep enough'
                return frame


        class AccessViolation(Exception):
            pass


        class Access:
            def __findattr__(self, name, *args):
                methcache = self.__dict__.setdefault('__cache__', {})
                missing = []
                obj = getattr(self, name, missing)
                # if obj is missing we better be doing a setattr for
                # the first time
                if obj is not missing and type(obj) == types.MethodType:
                    # Digusting hack because there's no way to
                    # dynamically figure out what the method being
                    # called is from the stack frame.
                    methcache[obj.im_func.func_code] = obj.im_class
                #
                # What's the access permissions for this name?
                access, klass = getattr(self, '__access__', {}).get(
                    name, (PUBLIC, 0))
                if access is not PUBLIC:
                    # Now try to see which method is calling us
                    frame = getframe(0).f_back
                    if frame is None:
                        raise AccessViolation
                    # Get the class of the method that's accessing
                    # this attribute, by using the code object cache
                    if frame.f_code.co_name == '__init__':
                        # There aren't entries in the cache for ctors,
                        # because the calling mechanism doesn't go
                        # through __findattr__().  Are there other
                        # methods that might have the same behavior?
                        # Since we can't know who's __init__ we're in,
                        # for now we'll assume that only protected and
                        # public attrs can be accessed.
                        if access is PRIVATE:
                            raise AccessViolation
                    else:
                        methclass = self.__cache__.get(frame.f_code)
                        if not methclass:
                            raise AccessViolation
                        if access is PRIVATE and methclass is not klass:
                            raise AccessViolation
                        if access is PROTECTED and not issubclass(methclass,
                                                                  klass):
                            raise AccessViolation
                # If we got here, it must be okay to access the attribute
                if args:
                    return setattr(self, name, *args)
                return obj

        # tests
        class A(Access):
            def __init__(self, foo=0, name='A'):
                self._foo = foo
                # can't set private names in __init__
                self.__initprivate(name)

            def __initprivate(self, name):
                self._name = name

            def getfoo(self):
                return self._foo

            def setfoo(self, newfoo):
                self._foo = newfoo

            def getname(self):
                return self._name

        A.__access__ = {'_foo'      : (PROTECTED, A),
                        '_name'     : (PRIVATE, A),
                        '__dict__'  : (PRIVATE, A),
                        '__access__': (PRIVATE, A),
                        }

        class B(A):
            def setfoo(self, newfoo):
                self._foo = newfoo + 3

            def setname(self, name):
                self._name = name

        b = B(1)
        b.getfoo()

        a = A(1)
        assert a.getfoo() == 1
        a.setfoo(2)
        assert a.getfoo() == 2

        try:
            a._foo
        except AccessViolation:
            pass
        else:
            assert 0, 'AccessViolation expected'

        try:
            a._foo = 3
        except AccessViolation:
            pass
        else:
            assert 0, 'AccessViolation expected'

        try:
            a.__dict__['_foo']
        except AccessViolation:
            pass
        else:
            assert 0, 'AccessViolation expected'


        b = B()
        assert b.getfoo() == 0
        b.setfoo(2)
        assert b.getfoo() == 5
        try:
            b.setname('B')
        except AccessViolation:
            pass
        else:
            assert 0, 'AccessViolation expected'

        assert b.getname() == 'A'


    Here's an implementation of the attribute hook described in PEP
    213 (except that hooking on attribute deletion isn't supported by
    the current reference implementation).

        class Pep213:
            def __findattr__(self, name, *args):
                hookname = '__attr_%s__' % name
                if args:
                    op = 'set'
                else:
                    op = 'get'
                # XXX: op = 'del' currently not supported
                missing = []
                meth = getattr(self, hookname, missing)
                if meth is missing:
                    if op == 'set':
                        return setattr(self, name, *args)
                    else:
                        return getattr(self, name)
                else:
                    return meth(op, *args)


        def computation(i):
            print 'doing computation:', i
            return i + 3


        def rev_computation(i):
            print 'doing rev_computation:', i
            return i - 3


        class X(Pep213):
            def __init__(self, foo=0):
                self.__foo = foo

            def __attr_foo__(self, op, val=None):
                if op == 'get':
                    return computation(self.__foo)
                elif op == 'set':
                    self.__foo = rev_computation(val)
                # XXX: 'del' not yet supported

        x = X()
        fooval = x.foo
        print fooval
        x.foo = fooval + 5
        print x.foo
        # del x.foo


Reference Implementation

   The reference implementation, as a patch to the Python core, can be
   found at this URL:

   http://sourceforge.net/patch/?func=detailpatch&patch_id=102613&group_id=5470


References

    [1] http://docs.python.org/reference/datamodel.html#customizing-attribute-access
    [2] http://www.javasoft.com/products/javabeans/
    [3] http://www.digicool.com/releases/ExtensionClass/Acquisition.html
    [5] http://www.digicool.com/releases/ExtensionClass
    [6] http://www.python.org/doc/essays/metaclasses/
    [7] http://www.foretec.com/python/workshops/1998-11/dd-ascher-sum.html
    [8] http://docs.python.org/howto/regex.html
    [9] PEP 213, Attribute Access Handlers, Prescod
        http://www.python.org/dev/peps/pep-0213/


Rejection

    There are serious problems with the recursion-protection feature.
    As described here it's not thread-safe, and a thread-safe solution
    has other problems.  In general, it's not clear how helpful the
    recursion-protection feature is; it makes it hard to write code
    that needs to be callable inside __findattr__ as well as outside
    it.  But without the recursion-protection, it's hard to implement
    __findattr__ at all (since __findattr__ would invoke itself
    recursively for every attribute it tries to access).  There seems
    to be no good solution here.

    It's also dubious how useful it is to support __findattr__ both
    for getting and for setting attributes -- __setattr__ gets called
    in all cases alrady.

    The examples can all be implemented using __getattr__ if care is
    taken not to store instance variables under their own names.


Copyright

    This document has been placed in the Public Domain.



Local Variables:
mode: indented-text
indent-tabs-mode: nil
End: