Source

JythonBook / chapter5.rst

Full commit
Josh Juneau d9b2bb8 





















































































































































































































































































































































































































































































































































































































































































































































































































































































































  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
Chapter 5:  Object Oriented Jython
==================================

This chapter is going to cover the basics of object oriented
programming.  If you're familiar with the concepts.  I'll start with
covering the basic reasons for why you would want to write object
oriented code in the first place, then cover all the basic syntax, and
finally I'll show you a non-trivial example.

Object oriented programming is a method of programming where you
package your code up into bundles of data and behaviour.  In Jython,
you can define a template for this bundle with a class definition
With this first class written, you can then instantiate copies of your
object and have them act upon each other.  This helps you organize
your code into smaller more manageable bundles 

Throughout this chapter, I interchangably use Python and Jython - for
regular object oriented programming - the two dialects of Python are
so similar that there are no meaningful differences between the two
languages.  Enough introduction text though - let's take a look at
some basic syntax to see what this is all about.

Basic Syntax
------------

Writing a class is really simple.  It is fundamentally about managing
some kind of 'state' and exposing some functions to manipulate that
state.  In object jargon - we just call those functions 'methods'.

Let's start by creating a car class.  The goal is to create an object
that will manage it's own location on a two dimensional plane.  We
want to be able to tell it to turn, move forward, and we want to be
able to interrogate the object to find out where it's current location
is. ::

    class Car(object):

        NORTH = 0 
        EAST = 1
        SOUTH = 2 
        WEST = 3

        def __init__(self, x=0, y=0):
            self.x = x
            self.y = y
            self.direction = 0

        def turn_right(self):
            self.direction += 1
            self.direction = self.direction % 4

        def turn_left(self):
            self.direction -= 1
            self.direction = self.direction % 4

        def move(self, distance):
            if self.direction == self.NORTH: 
                self.y += distance
            elif self.direction == self.SOUTH:
                self.y -= distance
            elif self.direction == self.EAST:
                self.x += distance
            else:
                self.x -= distance

        def position(self):
            return (self.x, self.y)

We'll go over that class definition in detail but right now, let's
just see how to create a car, move it around and ask the car where it
is. ::

    from car import Car

    def test_car():
        c = Car()
        c.turn_right()
        c.move(5)
        assert (5, 0) ==  c.position()

        c.turn_left()
        c.move(3)

        assert (5, 3) == c.position()

    if __name__ == '__main__':
        test_car()

The best way to think of a class is to think of it like a special kind of
function that acts like a factory that generates object instances. For
each call to the class - you are creating a new discrete copy of your
object.

Once we've created the car instance, we can simply call functions that
are attached to the car class and the object will manage it's own
location.  From the point of view of our test code - we do not need to
manage the location of the car - nor do we need to manage the
direction that the car is pointing in.  We just tell it to move - and
it does the right thing.

Let's go over the syntax in detail to see exactly what's going on
here.

In Line 1, we declare that our Car object is a subclass of the root
"object" class.  Python, like many object oriented languages has a
'root' object that all other objects are based off of.  This 'object'
class defines basic behavior that all classes can reuse.

Python actually has two kinds of classes - 'newstyle' and old style.
The old way of declaring classes didn't require you to type 'object' -
you'll occassionally see the old-style class usage in some Python
code, but it's not consider a good practice   Just subclass 'object'
for any of your base classes and your life will be simpler[1]_.

Lines 3 to 6 declare class attributes for the direction that any car
can point  to.  These are *class* attributes so they can be shared
across all object instances of the car object.

Now for the good stuff.

Line 8-11 declares the object initializer.  In some languages, you
might be familiar with a constructor - in Jython, we have an
initializer which lets us pass values into an object at the time of
creation.  

In our initializer, we are setting the initial position of the car to
(0, 0) on a 2 dimensional plane and then the direction of the car is
initialized to pointing north.  Fairly straight forward so far.

The function signature uses Python's default argument list feature so
we don't have to explicitly set the initial location to (0,0), but
there's a new argument introduced called 'self'. This is a reference
to the current object.

Remember - your class definition is creating instances of objects.
Once your object is created, it has it's own set of internal variables
to manage.  Your object will inevitably need to access these as well
as any of the classes internal methods.  Python will pass a reference
to the current object as the first argument to all your instance
methods.

If you're coming from some other object oriented language, you're
probably familiar with the 'this' variable. Unlike C++ or Java, Python
doesn't magically introduce the reference into the namespace of
accessible variables, but this is consistent with Python's philosophy
of making things explicit for clarity.

When we want to assign the initial x,y position, we just need to
assign values on to the name 'x', and 'y' on the object.  Binding
the values of x and y to self makes the position values accessible to
any code that has access to self - namely the other methods of the
object. One minor detail here - in Python, you can technically
name the arguments however you want.  There's nothing stopping you
from calling the first argument 'this' instead of 'self', but the
community standard is to use 'self' [2]_.

Line 13 to 19 declare two methods to turn the vehicle in different
directions.  Notice how the direction is never directly manipulated by
the caller of the Car object.  We just asked the car to turn, and the
car changed it's own internal 'direction' state.

Line 21 to 29 define where the car should move to when we move the car
forward.  The internal direction variable informs the car how it
should manipulate the x and y position.  Notice how the caller of the
car object never needs to know precisely what direction the car is
pointing in.  The caller only needs to tell the object to turn and
move forward.  The particular details of how that message is used is
abstracted away.

That's not too bad for a couple dozen lines of code.

This concept of hiding internal details is called encapsulation.  This
is a core concept in object oriented programming.  As you can see from
even this simple example - it allows you to structure your code so
that you can provide a simplified interface to the users of your code.

Having a simplified interface means that we could have all kinds of
behaviour happening behind the function calls to turn and move - but
the caller can ignore all those details and concentrate on *using* the
car instead of managing the car. 

As long as the method signatures don't change, the caller really
doesn't need to care about any of that.  We can easily add
persistence to this class - so we can save and load the car's state 
to disk.

First, pull in the pickle module - pickle will let us convert python
objects into byte strings that can be restored to full objects later.

    import pickle

Now, just add two new methods to load and save the state of the object. ::

    def save(self):
        state = (self.direction, self.x, self.y)
        pickle.dump(state, open('mycar.pickle','wb'))

    def load(self):
        state = pickle.load(open('mycar.pickle','rb'))
        (self.direction, self.x, self.y) = state

Simply add calls to save() at the end of the turn and move methods,
and the object will automatically save all the relevant internal
values to disk.

People who use the car object don't even need to know that it's saving
to disk, because the Car object handles it behind the scenes. ::

        def turn_right(self):
            self.direction += 1
            self.direction = self.direction % 4
            self.save()

        def turn_left(self):
            self.direction -= 1
            self.direction = self.direction % 4
            self.save()

        def move(self, distance):
            if self.direction == self.NORTH: 
                self.y += distance
            elif self.direction == self.SOUTH:
                self.y -= distance
            elif self.direction == self.EAST:
                self.x += distance
            else:
                self.x -= distance
            self.save()

Now, when you call the turn, or move methods, the car will
automatically save itself to disk.  If you want to reconstruct the car
object's state from a previously saved pickle file, you can simply
call the load() method.

Object Attribute Lookups
------------------------

If you've beeen paying attention, you're probably wondering how the
NORTH, SOUTH, EAST and WEST variables got bound to self.  We never
actually assigned them to the self variable during object
initialization - so what's going on when we call move()?  How is
Jythpon actually resolving the value of those four variables?

Now seems like a good time to show how Jython resolves name lookups.

The direction names actually got bound to the Car class.  The Jython
object system does a little bit of magic when you try accessing any
*name* against an object, it first searches for anything that was
bound to 'self'.  If python can't resolve any attribute on self with
that name, it goes up the object graph to the class definition.  The
direction attributes NORTH, SOUTH, EAST, WEST were bound to the class
definition - so the name resolution succeeds and we get the value of
the class attribute.

An very short example will help clarify this  ::

    >>> class Foobar(object):
    ...   def __init__(self):
    ...     self.somevar = 42
    ...   class_attr = 99
    ... 
    >>> 
    >>> obj = Foobar()
    >>> obj.somevar
    42
    >>> obj.class_attr
    99
    >>> obj.not_there
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    AttributeError: 'Foobar' object has no attribute 'not_there'
    >>> 

So the key difference here is *what* you bind a value to.  The values
you bind to self are available only to a single object.  Values you
bind to the class definition are available to all instances of the
class.  The sharing of class attributes among all instances is a
critical distinction because mutating a class attribute will affect
all instances.  This may cause unintended side effects if you're not
paying attention as a variable may change value on you when you aren't
expecting it to. ::

    >>> other = Foobar()
    >>> other.somevar
    42
    >>> other.class_attr
    99
    >>> # obj and other will have different values for somevar
    >>> obj.somevar = 77
    >>> obj.somevar         
    77
    >>> other.somevar
    42
    >>> # Now show that we have the same copy of class_attr
    >>> other.class_attr = 66
    >>> other.class_attr
    66
    >>> obj.class_attr
    66

I think it's important to stress just how transparent Python's object
system really is.  Object attributes are just stored in a plain python
dictionary.  You can directly access this dictionary by looking at the
__dict__ attribute. ::

    >>> obj = Foobar()
    >>> obj.__dict__
    {'somevar': 42}

Notice that there are no references to the methods of the class, or
the class attribute.  I'll reiterate it again - Python is going to
just go up your inheritance graph - and go to the class definition to
look for the methods of Foobar and the class attributes of foobar.

The same trick can be used to inspect all the attributes of the class,
just look into the __dict__ attribute of the class definition and
you'll find your class attributes and all the methods that are
attached to your class definition ::

    >>> Foobar.__dict__
    {'__module__': '__main__', 
        'class_attr': 99, 
        '__dict__': <attribute '__dict__' of 'Foobar' objects>, 
        '__init__': <function __init__ at 1>}

This transparency can be leveraged with dynamic programming techniques
using closures and binding new functions into your class definition at
runtime.  We'll revisit this later in the chapter when we look at
generating function dynamically and finally with a short introduction
to metaprogramming.

Inheritance and Overloading
---------------------------

In the car example, we subclass from the root object type.  You can
also subclass your own classes to specialize the behaviour of your
objects.  You may want to do this if you notice that your code
naturally has a structure where you have many different classes that
all share some common behaviour.

With objects, you can write one class, and then reuse it using
inheritance to automatically gain access to the pre-existing behavior
and attributes of the parent class.  Your 'base' objects will inherit
behaviour from the root 'object' class, but any subsequent subclasses
will inherit from your own classes.

Let's take a simple example of using some animal classes to see how
this works. Define a module "animals.py" with the following code:

    class Animal(object):
        def sound(self):
            return "I don't make any sounds"

    class Goat(Animal):
        def sound(self):
            return "Bleeattt!"

    class Rabbit(Animal):
        def jump(self):
            return "hippity hop hippity hop"

    class Jackalope(Goat, Rabbit):
        pass

Now you should be able to explore that module with the jython
interpreter:

    >>> from animals import *
    >>> animal = Animal()
    >>> goat = Goat()
    >>> rabbit = Rabbit()
    >>> jack = Jackalope()

    >>> animal.sound()
    "I don't make any sounds"
    >>> animal.jump()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    AttributeError: 'Animal' object has no attribute 'jump'

    >>> rabbit.sound()
    "I don't make any sounds"
    >>> rabbit.jump()
    'hippity hop hippity hop'

    >>> goat.sound()
    'Bleeattt!'
    >>> goat.jump()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    AttributeError: 'Goat' object has no attribute 'jump'

    >>> jack.jump()
    'hippity hop hippity hop'
    >>> jack.sound()
    'Bleeattt!'

Inheritance is a very simple concept, when you declare your class, you
simply specify which parent classes you would like to reuse.  Your new
class can then automatically access all the methods and attributes of
the super class.  Notice how the goat couldn't jump and the rabbit
couldn't make any sound, but the Jackalope had access to methods from
both the rabbit and the goat.

With single inheritance - when your class simply inherits from one
parent class - the rules for resolving where to find an attribute or a
method are very straight forward.  Jython just looks up to the parent if
the current object doesn't have a matching attribute.  

It's important to point out now that the Rabbit class is a type of
Animal - the Python runtime can tell you that programmatically by
using the isinstance function ::

    >>> isinstance(bunny, Rabbit)
    True
    >>> isinstance(bunny, Animal)
    True
    >>> isinstance(bunny, Goat)
    False

For many classes, you may want to extend the behavior of the parent
class instead of just completley overriding it.  For this, you'll want
to use the super().  Let's specialize the Rabbit class like this. ::

    class EasterBunny(Rabbit): 
        def sound(self): 
            orig = super(EasterBunny, self).sound() 
            return "%s - but I have eggs!" % orig 

If you now try making this rabbit speak, it will extend the original
sound() method from the base Rabbit class ::

    >>> bunny = EasterBunny() 
    >>> bunny.sound()
    "I don't make any sounds - but I have eggs!"

That wasn't so bad.  For these examples, I only demonstrated that
inherited methods can be invoked, but you can do exactly the same
thing with attributes that are bound to the self.

For multiple inheritance, things get very tricky.  In fact, the rules
for resolving how attributes are looked up would easily fill an entire
chapter (look up "The Python 2.3 Method Resolution Order" on Google if
you don't believe me). There's not enough space in this chapter to
properly cover the topic which should be a good indication to you that
you really don't want to use multiple inheritance.

More advanced abstraction
-------------------------

Abstraction using plain classes is wonderful and all, but it's even
better if your code seems to naturally fit into the syntax of the
language.  Python supports a variety of underscore methods - methods
that start and end with double "_" signs that let you overload the
behaviour of your objects.  This means that your objects will seem to
integrate more tightly with the language itself.

With the underscore methods, you can give you objects behaviour for
logical and mathematical operations.  You can even make your objects
behave more like standard builtin types like lists, sets or
dictionaries.

    from __future__ import with_statement
    from contextlib import closing

    with closing(open('simplefile','w')) as fout:
        fout.writelines(["blah"])

    with closing(open('simplefile','r')) as fin:
        print fin.readlines()

The above snippet of code just opens a file, writes a little bit of
text and then we just read the contents out.  Not terriblly exciting.
Most objects in Python are serializable to strings using the pickle
module.  We can leverage pickle to write out full blown objects to
disk.  Let's see the functional version of this: ::

    from __future__ import with_statement
    from contextlib import closing
    from pickle import dumps, loads

    def write_object(fout, obj):
        data = dumps(obj)
        fout.write("%020d" % len(data))
        fout.write(data)

    def read_object(fin):
        length = int(fin.read(20))
        obj = loads(fin.read(length))
        return obj

    class Simple(object):
        def __init__(self, value):
            self.value = value
        def __unicode__(self):
            return "Simple[%s]" % self.value

    with closing(open('simplefile','wb')) as fout:
        for i in range(10):
            obj = Simple(i)
            write_object(fout, obj)

    print "Loading objects from disk!"
    print '=' * 20

    with closing(open('simplefile','rb')) as fin:
        for i in range(10):
            print read_object(fin)

This should output something like this: ::

    Loading objects from disk!
    ====================
    Simple[0]
    Simple[1]
    Simple[2]
    Simple[3]
    Simple[4]
    Simple[5]
    Simple[6]
    Simple[7]
    Simple[8]
    Simple[9]

So now we're doing something interesting.  Let's look at exactly what
happening here.

First, you'll notice that the Simple object is rendering a nice - the
Simple object can render itself using the __unicode__ method.  This is
clearly an improvement over the earlier rendering of the object with angle
brackets and a hex code.

The write_object function is fairly straight forward, we're just
converting our objects into strings using the pickle module, computing
the length of the string and then writing the length and the actual
serialized object to disk.

This is fine, but the read side is a bit clunky. We don't really know
when to stop reading.  We can fix this using the iteration protocol.
Which bring us to one of my favourite reasons to use objects at all in
Python.  

Protocols
---------

In Python, we have 'duck typing'.  If it sounds like a duck, quacks
like a duck and looks like a duck - well - it's a duck. This is in
stark contrast to more rigid languagse like C# or Java which have
formal interface definitions.  One of the nice benefits of having duck
typing is that Python has the notion of object 'protocols'.

If you happen to implement the right methods - python will recognize
your object as a certain type of 'thing'.

Iterators are objects that look like lists that let you read the next
object.  Implementing an iterator protocol is straight forward - just
implement a next() method and a __iter__ method and you're ready to
rock and roll.  Let's see this in action: ::

    class PickleStream(object):
        """
        This stream can be used to stream objects off of a raw file stream
        """
        def __init__(self, file):
            self.file = file

        def write(self, obj):
            data = dumps(obj)
            length = len(data)
            self.file.write("%020d" % length)
            self.file.write(data)

        def __iter__(self):
            return self

        def next(self):
            data = self.file.read(20)
            if len(data) == 0:
                raise StopIteration
            length = int(data)well
            return loads(self.file.read(length))

        def close(self):
            self.file.close()

The above class will let you wrap a simple file object and you can now
send it raw python objects to write to a file, or you can read objects
out as if the stream was just a list of objects.  Writing and reading
becomes much simpler ::

    with closing(PickleStream(open('simplefile','wb'))) as stream:
        for i in range(10):
            obj = Simple(i)
            stream.write(obj)

    with closing(PickleStream(open('simplefile','rb'))) as stream:
        for obj in stream:
            print obj

Abstracting out the details of serialization into the PickleStream
lets us 'forget' about the details of how we are writing to disk.  All
we care about is that the object will do the right thing when we call
the write() method.

The iteration protocol can be used for much more advanced uses, but
even with this example, it should be obvious how useful it is.  While
you could implement the reading behaviour with a read() mo loethod, just
using the stream as something you can loop over makes the code much
easier to understand.

An aside a common problem that everyone seems to have
-----------------------------------------------------

One particular snag that seems to catch every python programmer is
when you use default values in a method signature. ::

    >>> class Tricky(object):
    ...   def mutate(self, x=[]):
    ...     x.append(1)
    ...     return x
    ... 
    >>> obj = Tricky()
    >>> obj.mutate()
    [1]
    >>> obj.mutate()
    [1, 1]
    >>> obj.mutate()
    [1, 1, 1]

What's happening here is that the instance method 'mutate' is an
object.  The method object stores the default value for 'x' in an
attribute *inside* the method object.  So when you go and mutate the
list, you're actually changing the value of an attribute of the method
itself.   Remember - this happens because when you invoke the mutate
method, you're just accessing a callable attribute on the Tricky
object.

Runtime binding of methods
--------------------------

One interesting feature in Python is that instance methods are
actually just attributes hanging off of the class defintion - the
functions are just attributes like any other variable, except that
they happen to be 'callable'.

It's even possible to create and bind in functions to a class
definition at runtime using the new module to create instance methods.
In the following example, you can see that it's possible to define a
class with nothing in it, and then bind methods to the class
definition at runtime. ::

    >>> def some_func(self, x, y):
    ...   print "I'm in object: %s" % self
    ...   return x * y
    ... 
    >>> import new
    >>> class Foo(object): pass
    ... 
    >>> f = Foo()
    >>> f
    <__main__.Foo object at 0x1>
    >>> Foo.mymethod = new.instancemethod(some_func, f, Foo)
    >>> f.mymethod(6,3)
    I'm in object: <__main__.Foo object at 0x1>
    18

When you invoke the 'mymethod' method, the same attribute lookup
machinery is being invoked.  Python looks up the name against the
'self' object.  When it can't find anything there, it goes to the
class definition.  When it finds it there, the instancemethod object
is returned.  The function is then caled with two arguments and you
get to see the final result. 

This kind of dynamism in Jython is extremely powerful.  You can write
code that generates functions at program runtime and then bind those
functions to objects. You can do all of this because in Jython,
classes are what are known as 'first class objects'.  The class
definition itself is an actual object  - just like any other object.
Manipulating classes is as easy as manipulating any other object.

Closures and Passing Objects
----------------------------

Python supports the notion of nested scopes - this can be used by to
preserve some state information inside of another function.  This
technique isn't all that common outside of dynamic languages, so you
may have never seen this before.  Let's look at a simple example ::

    def adder(x):
        def inner(y):
            return x + y
        return inner

    >>> func = adder(5)
    >>> func
    <function inner at 0x7adf0>
    >>> func(8)
    13

This is pretty cool - we can actually create functions from templates of other
functions.  If you can think of a way to parameterize the behavior of a
function, it becomes possible to create new functions dynamically.
You can think of currying as yet another way of creating templates -
this time you are creating a template for new functions.

This is a tremendously powerful tool once you gain some experience
with it.  Remember - everything in python is an object - even
functions are first class objects in Python so you can pass those in
as arguments as well.  A practical use of this is to partially
construct new functions from 'base' functions with some basic known
behavior.

Let's take the previous adder closure and convert it to a more general
form ::

    def arith(math_func, x):
        def inner(y):
            return math_func(x, y)
        return inner

    def adder(x, y):
        return x + y

    >>> func = arith(adder, 91)
    >>> func(5)
    96

This technique is called currying - you're now creating new function
objects based on previous functions. The most common use for this is
to create decorators.  In Python, you can define special kinds of
objects that wrap up your methods and add extra behavior.  Some
decorators are builtin already like 'property', 'classmethod' and
'staticmethod'.  Once you have a decorator, you can sprinkle it on to
of another function to add new behavior.

Decorator syntax looks something like this ::

    @decorator_func_name(arg1, arg2, arg3, ...)
    def some_functions(x, y, z, ...):
        # Do something useful here
        pass

Suppose we have some method that requires intensive computational
resoures to run, but the results do not vary much over time.  Wouldn't
it be nice if we could cache the results so that the computation
wouldn't have to run each and every time? 

Here's our class with a slow computation method ::

    import time
    class Foobar(object): 
        def slow_compute(self, *args, **kwargs): 
            time.sleep(1) 
            return args, kwargs, 42 

Now let's cache the value using a decorator function.  Our strategy
is that for any function named X with some argument list, we want to
create a unique name and save the final computed value to that name.
We want our cached value to have a human readable name, we we want to
reuse the original function name, as well as the arguments that were
passed in the first time.

Let's get to some code! ::

    import hashlib
    def cache(func):
        """ 
        This decorator will add a _cache_functionName_HEXDIGEST
        attribute after the first invocation of an instance method to
        store cached values.
        """
        # Obtain the function's name
        func_name = func.func_name 
        # Compute a unique value for the unnamed and named arguments
        arghash = hashlib.sha1(str(args) + str(kwargs)).hexdigest()
        cache_name = '_cache_%s_%s' % (func_name, arghash)
        def inner(self, *args, **kwargs):
            if hasattr(self, cache_name):
                # If we have a cached value, just use it
                print "Fetching cached value from : %s" % cache_name
                return getattr(self, cache_name)
            result = func(self, *args, **kwargs)
            setattr(self, cache_name, result)
            return result
        return inner

There are only two new tricks that are in this code.

1) I'm using the hashlib module to convert the arguments to the
   function into a unique single string.
2) The use of getattr, hasattr and setattr to manipulate the cached
   value on the instance object.

Now, if we want to cache the slow method, we just throw on a @cache
line above the method declaration. ::

    @cache
    def slow_compute(self, *args, **kwargs): 
        time.sleep(1) 
        return args, kwargs, 42 

Fantastic - we can reuse this cache decorator for any method we want
now.  Let's suppose now that we want our cache to invalidate itself after
every 3 calls.  This practical use of currying is only a slight
modification to the original caching code. ::

    import hashlib
    def cache(loop_iter):
        def function_closure(func):
            func_name = func.func_name
            def closure(self, loop_iter, *args, **kwargs):
                arghash = hashlib.sha1(str(args) + str(kwargs)).hexdigest()
                cache_name = '_cache_%s_%s' % (func_name, arghash)
                counter_name = '_counter_%s_%s' % (func_name, arghash)
                if hasattr(self, cache_name):
                    # If we have a cached value, just use it
                    print "Fetching cached value from : %s" % cache_name
                    loop_iter -= 1
                    setattr(self, counter_name, loop_iter)
                    result = getattr(self, cache_name)
                    if loop_iter == 0:
                        delattr(self, counter_name)
                        delattr(self, cache_name)
                        print "Cleared cached value"
                    return result
                result = func(self, *args, **kwargs)
                setattr(self, cache_name, result)
                setattr(self, counter_name, loop_iter)
                return result
            return closure
        return function_closure

Now we're free to use @cache for any slow method and caching will
come in for free - including automatic invalidation of the cached
value.  Just use it like this ::

    @cache(10)
    def slow_compute(self, *args, **kwargs): 
        # TODO: stuff goes here...
        pass

Review - and a taste of how we could fit all of this together
-------------------------------------------------------------

Now - I'm going to ask you to use your imagination a litte.  We've
covered quite a bit of ground really quickly.  

We can :

 * look up attributes in an object (use the __dict__ attribute).  
 * check if an object belongs to a particular class hierarchy (use the isinstance function).  
 * build functions out of other functions using currying.and even bind those functions to arbitrary names

This is fantastic - we now have all the basic building blocks we need
to generate complex methods based on the attributes of our class.
Imagine a simplified addressbook application with a simple contact. ::

    class Contact(object):
        first_name = str
        last_name = str
        date_of_birth = datetime.Date

Assuming we know how to save and load to a database, we can use the
function generation techniques to automatically generate load() and
save() methods and bind them into our Contact class.  We can use our
introspection techniques to determine what attributes need to be saved
to our database.  We could even grow special methods onto our Contact
class so that we could iterate over all of the class attributes and
magically grow 'searchby_first_name' and 'searchby_last_name' methods.

See how powerful this can be?  We can write extremly minimal code, and
we could code generate all of our required specialized behavior for
saving, loading and searching for records in a database.  Since we do
all of that programmatically - we can dramatically reduce the amount
of code that we have to write by hand and by doing so - we can redue
the chance that we introduce bugs into our system.

We're going to do exactly that in a later chapter. Build a simple
database abstraction layer to demonstrate how to create your own
object system that will automatically know how to read and write to a
database. 

.. Footnotes

.. [1] New style classes provide a large number of useful features that simply aren't available to old-style classes.  If you end up mixing old and new style classes together, you'll usually get unexpected behaviour that will surprise you - and not in the good way.  It'll surprise you in the kind of way that will keep you up late at night wondering why your code doesn't work and you'll curse the fact that both styles of classes exist at all.

.. [2] One of Python's strengths is legibility - of your code and other code.  Community standards help the legibility of code tremendously.