JythonBook / sandbox / functions.rst

Josh Juneau d9b2bb8 


































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































   1
   2
   3
   4
   5
   6
   7
   8
   9
  10
  11
  12
  13
  14
  15
  16
  17
  18
  19
  20
  21
  22
  23
  24
  25
  26
  27
  28
  29
  30
  31
  32
  33
  34
  35
  36
  37
  38
  39
  40
  41
  42
  43
  44
  45
  46
  47
  48
  49
  50
  51
  52
  53
  54
  55
  56
  57
  58
  59
  60
  61
  62
  63
  64
  65
  66
  67
  68
  69
  70
  71
  72
  73
  74
  75
  76
  77
  78
  79
  80
  81
  82
  83
  84
  85
  86
  87
  88
  89
  90
  91
  92
  93
  94
  95
  96
  97
  98
  99
 100
 101
 102
 103
 104
 105
 106
 107
 108
 109
 110
 111
 112
 113
 114
 115
 116
 117
 118
 119
 120
 121
 122
 123
 124
 125
 126
 127
 128
 129
 130
 131
 132
 133
 134
 135
 136
 137
 138
 139
 140
 141
 142
 143
 144
 145
 146
 147
 148
 149
 150
 151
 152
 153
 154
 155
 156
 157
 158
 159
 160
 161
 162
 163
 164
 165
 166
 167
 168
 169
 170
 171
 172
 173
 174
 175
 176
 177
 178
 179
 180
 181
 182
 183
 184
 185
 186
 187
 188
 189
 190
 191
 192
 193
 194
 195
 196
 197
 198
 199
 200
 201
 202
 203
 204
 205
 206
 207
 208
 209
 210
 211
 212
 213
 214
 215
 216
 217
 218
 219
 220
 221
 222
 223
 224
 225
 226
 227
 228
 229
 230
 231
 232
 233
 234
 235
 236
 237
 238
 239
 240
 241
 242
 243
 244
 245
 246
 247
 248
 249
 250
 251
 252
 253
 254
 255
 256
 257
 258
 259
 260
 261
 262
 263
 264
 265
 266
 267
 268
 269
 270
 271
 272
 273
 274
 275
 276
 277
 278
 279
 280
 281
 282
 283
 284
 285
 286
 287
 288
 289
 290
 291
 292
 293
 294
 295
 296
 297
 298
 299
 300
 301
 302
 303
 304
 305
 306
 307
 308
 309
 310
 311
 312
 313
 314
 315
 316
 317
 318
 319
 320
 321
 322
 323
 324
 325
 326
 327
 328
 329
 330
 331
 332
 333
 334
 335
 336
 337
 338
 339
 340
 341
 342
 343
 344
 345
 346
 347
 348
 349
 350
 351
 352
 353
 354
 355
 356
 357
 358
 359
 360
 361
 362
 363
 364
 365
 366
 367
 368
 369
 370
 371
 372
 373
 374
 375
 376
 377
 378
 379
 380
 381
 382
 383
 384
 385
 386
 387
 388
 389
 390
 391
 392
 393
 394
 395
 396
 397
 398
 399
 400
 401
 402
 403
 404
 405
 406
 407
 408
 409
 410
 411
 412
 413
 414
 415
 416
 417
 418
 419
 420
 421
 422
 423
 424
 425
 426
 427
 428
 429
 430
 431
 432
 433
 434
 435
 436
 437
 438
 439
 440
 441
 442
 443
 444
 445
 446
 447
 448
 449
 450
 451
 452
 453
 454
 455
 456
 457
 458
 459
 460
 461
 462
 463
 464
 465
 466
 467
 468
 469
 470
 471
 472
 473
 474
 475
 476
 477
 478
 479
 480
 481
 482
 483
 484
 485
 486
 487
 488
 489
 490
 491
 492
 493
 494
 495
 496
 497
 498
 499
 500
 501
 502
 503
 504
 505
 506
 507
 508
 509
 510
 511
 512
 513
 514
 515
 516
 517
 518
 519
 520
 521
 522
 523
 524
 525
 526
 527
 528
 529
 530
 531
 532
 533
 534
 535
 536
 537
 538
 539
 540
 541
 542
 543
 544
 545
 546
 547
 548
 549
 550
 551
 552
 553
 554
 555
 556
 557
 558
 559
 560
 561
 562
 563
 564
 565
 566
 567
 568
 569
 570
 571
 572
 573
 574
 575
 576
 577
 578
 579
 580
 581
 582
 583
 584
 585
 586
 587
 588
 589
 590
 591
 592
 593
 594
 595
 596
 597
 598
 599
 600
 601
 602
 603
 604
 605
 606
 607
 608
 609
 610
 611
 612
 613
 614
 615
 616
 617
 618
 619
 620
 621
 622
 623
 624
 625
 626
 627
 628
 629
 630
 631
 632
 633
 634
 635
 636
 637
 638
 639
 640
 641
 642
 643
 644
 645
 646
 647
 648
 649
 650
 651
 652
 653
 654
 655
 656
 657
 658
 659
 660
 661
 662
 663
 664
 665
 666
 667
 668
 669
 670
 671
 672
 673
 674
 675
 676
 677
 678
 679
 680
 681
 682
 683
 684
 685
 686
 687
 688
 689
 690
 691
 692
 693
 694
 695
 696
 697
 698
 699
 700
 701
 702
 703
 704
 705
 706
 707
 708
 709
 710
 711
 712
 713
 714
 715
 716
 717
 718
 719
 720
 721
 722
 723
 724
 725
 726
 727
 728
 729
 730
 731
 732
 733
 734
 735
 736
 737
 738
 739
 740
 741
 742
 743
 744
 745
 746
 747
 748
 749
 750
 751
 752
 753
 754
 755
 756
 757
 758
 759
 760
 761
 762
 763
 764
 765
 766
 767
 768
 769
 770
 771
 772
 773
 774
 775
 776
 777
 778
 779
 780
 781
 782
 783
 784
 785
 786
 787
 788
 789
 790
 791
 792
 793
 794
 795
 796
 797
 798
 799
 800
 801
 802
 803
 804
 805
 806
 807
 808
 809
 810
 811
 812
 813
 814
 815
 816
 817
 818
 819
 820
 821
 822
 823
 824
 825
 826
 827
 828
 829
 830
 831
 832
 833
 834
 835
 836
 837
 838
 839
 840
 841
 842
 843
 844
 845
 846
 847
 848
 849
 850
 851
 852
 853
 854
 855
 856
 857
 858
 859
 860
 861
 862
 863
 864
 865
 866
 867
 868
 869
 870
 871
 872
 873
 874
 875
 876
 877
 878
 879
 880
 881
 882
 883
 884
 885
 886
 887
 888
 889
 890
 891
 892
 893
 894
 895
 896
 897
 898
 899
 900
 901
 902
 903
 904
 905
 906
 907
 908
 909
 910
 911
 912
 913
 914
 915
 916
 917
 918
 919
 920
 921
 922
 923
 924
 925
 926
 927
 928
 929
 930
 931
 932
 933
 934
 935
 936
 937
 938
 939
 940
 941
 942
 943
 944
 945
 946
 947
 948
 949
 950
 951
 952
 953
 954
 955
 956
 957
 958
 959
 960
 961
 962
 963
 964
 965
 966
 967
 968
 969
 970
 971
 972
 973
 974
 975
 976
 977
 978
 979
 980
 981
 982
 983
 984
 985
 986
 987
 988
 989
 990
 991
 992
 993
 994
 995
 996
 997
 998
 999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
XXX Link into the standard docs

XXX ensure PSF license is incorporated in our book, much of the text of this chapter is the description of the builtin functions

Defining Functions and Using Built-Ins
======================================

Introduction
------------

Functions are the fundamental unit of work in Python. In this chapter,
we will start with the basics of functions. Then we look at using the
builtin functions. These are the core functions that are always
available, meaning they don't require an explicit import into your
namespace.

Next we will look at some alternative ways of defining functions, such as
lambdas and classes. We will also look at more advanced types of
functions, namely closures and generator functions.

As you will see, functions are very easy to define and use. Python
encourages an incremental style of development that you can leverage
when writing functions.

So how does this work out in practice? Often when writing a function
it may make sense to start with a sequence of statements and just try
it out in a console. Or maybe just write a short script in an
editor. The idea is to just to prove a path and answer such questions
as, "Does this API work in the way I expect?"  Because top-level code in
a console or script works just like it does in a function, it's easy
to later isolate this code in a function body and then package it as a
function, maybe in a libary, or as a method as part of a class. The
ease of doing this style of development is one aspect that makes
Python such a joy to program in. And of course in the Jython
implementation, it's easy to do that within the context of any Java
library.

.. note:: 

  Perhaps the only tricky part is to keep the whitespace consistent as
  you change the identation level. The key then is to use a good editor
  that supports Python.
  
XX - JJ:  This note sounds a bit negative on the whitespace that Python uses.  In
          my chapters I've been building the whitespace as a great feature of the language.
          As Mark Ramm had mentioned, our minds are wired just like Python is
          written...the whitespace is natural and allows for consistency and easy
          management.  

And nearly everything else is in terms of functions, even what are
typically declarations in other languages like Java. For example, a
class definition or module import is just syntax around the underlying
functions, which you can call yourself if you need to do
so. (Incidentally, these functions are ``type`` and ``__import__``
respectively, and you will be learning more about them later in the
sections on builtins.)

XX - JJ:  These topics are probably a bit advanced for a beginning programmer.  I agree
          that they are useful and should be included in the chapter...but perhaps
          not mentioned yet.  An example right of the start would be great!
          

XXX Functions are first-class objects XXX incorporate



Function Syntax and Basics
--------------------------

Functions are usually defined by using the :keyword:`def` keyword, the name
of the function, its parameters (if any), and the body of code. We
will start by looking at this example function::

  def times2(n):
      return n * 2
      
XX - JJ:  I like the times2 function as a starter...good choice here.

Normal usage can treat function definitions as being very simple. But
there's subtle power in every piece of the function definition, due to
the fact that Python is a dynamic language. We look at these pieces
from both a simple (the more typical case) and a more advanced
perspective.

We will also look at some alternative ways of creating functions in a
later section.


The :keyword:`def` Keyword
~~~~~~~~~~~~~~~~~~~~~~~~~~

Using :keyword:`def` for *define* seems simple enough, and this keyword
certainly can be used to declare a function just like you would in a
static language. You should write most code that way in fact.

However, the more advanced usage is that a function definition can
occur at any level in your code and be introduced at any time. Unlike
the case in a language like C or Java, function definitions are not
declarations. Instead they are *executable statements*. You can nest
functions, and we'll describe that more when we talk about nested
scopes. And you can do things like conditionally define them.

This means it's perfectly valid to write code like this::

    if variant:
        def f():
            print "One way"
     else:
        def f():
            print "or another"

Please note, regardless of when and where the definition occurs,
including its variants as above, the function definition will be
compiled into a function object at the same time as the rest of the
module or script that the function is defined in.


Naming the Function
~~~~~~~~~~~~~~~~~~~

We will describe this more in a later section, but the ``dir`` builtin
function will tell us about the names defined in a given namespace,
defaulting to the module, script, or console environment we are
working in. With this new ``times2`` function defined above, we now
see the following (at least) in the console namespace::

  >>> dir()
  ['__doc__', '__name__', 'times2']

We can also just look at what is bound to that name::

  >>> times2
  <function times2 at 0x1>

(This object is further introspectable. Try ``dir(times2)`` and go
from there.)

We can also redefine a function at any time::

  >>> def f(): print "Hello, world"
  ... 
  >>> def f(): print "Hi, world"
  ... 
  >>> f()
  Hi, world

This is true not just of running it from the console, but any module
or script. The original version of the function object will persist
until it's no longer referenced, at which point it will be ultimately
be garbage collected. In this case, the only reference was the name
``f``, so it became available for GC immediately upon rebind.

What's important here is that we simply rebound the name.  First it
pointed to one function object, then another. We can see that in
action by simply setting another name (equivalently, a variable) to
``times2``::

  >>> t2 = times2
  >>> t2(5)
  10

This makes passing a function as a parameter very easy, for a callback
for example. But first, we need to look at function parameters in more
detail.

.. sidebar:: Function Metaprogramming

  A given name can only be associated with one function at a time, so
  can't overload a function with multiple definitions. If you were to
  define two or more functions with the same name, the last one
  defined is used, as we saw.

  However, it is possible to overload a function, or otherwise
  genericize it. You simply need to create a dispatcher function that
  then dispatches to your set of corresponding functions.


Function Parameters and Calling Functions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

When defining a function, you specify the parameters it
takes. Typically you will see something like the following. The syntax
is familar::

  XXX def f(a, b, c)
  
XX JJ:  Maybe use the tip_calc function for explaining multiple parameters:
    def tip_calc(amt, pct) or something similar?  Although your example gets the point accross

Often defaults are specified::

  XXXX def f(a, b=1, c=None)

With this being the general form of what it take::

  XXX what's a clear way to describe this? probably from the python tutorial or ref
  def f(param1[=default1], *args, **kwargs)
  
  It is oftentimes nice to include default values when defining a function.  This
helps reduce the chance of errors when calling and using the function.  For instance,
if the tip_calc function were to be called without passing any arguments then
no error would be raised if defaults were provided.  However, if we were to try
and do the same without any defaults thn an expected error would be raised.  Let's
take a look at some examples to see how useful defaults can be.

First, let's define the simple *tip_calc* function:

::
    
    def tip_calc(amt, pct=.18):
        return amt * pct

Next, we'll try to use the function in various ways.  You will see that since we
did not provide any default for the *amt* argument, we must specify a value for
the it when calling the function.

::
    
    # Call the function providing a value for both amt and pct arguments
    >>> tip_calc(15.98, .18)
    2.8764
    # Call the function without providing a value for pct
    >>> tip_calc(15.98)     
    2.8764
    # Call the function without providing any values
    >>> tip_calc()     
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: tip_calc() takes at least 1 argument (0 given)

What if a function is called specifying only one or two arguments when there are
default values specified for each argument?  How does the function know to which argument
we wish to assign the passed values?  Python allows the names of arguments to be
specified when calling the function.  Such a design helps one move past issues such
as this.

::
    
    def tip_calc(amt=1.00, pct=.18):
        return amt * .18



With this being the general form of what it take::

  XXX what's a clear way to describe this? probably from the python tutorial or ref
  def f(param1[=default1], *args, **kwargs)

.. note:: 

  This is not exhaustive. You can also use tuple parameters, but in
  practice, they are not typically used, and were removed in Python
  3. We recommend you don't use them. For one thing, they cannot be
  properly introspected from Jython.

Calling a function is symmetric. 
You can call a function. The parentheses are mandatory. 

Calling functions is also done by with a familiar syntax. For example,
for the function x with parameters ``a,b,c`` that would be
x(a,b,c). Unlike some other dynamic languages like Ruby and Perl, the
use of parentheses is required syntax (due the function name being
just like any other name).

Objects are strongly typed, as we have seen. But function parameters,
like names in general in Python, are not typed.  This means that
any parameter can refer to any type of object.

We see this play out in the ``times2`` function. The ``*`` operator
not only means multiply for numbers, it also means repeat for
sequences (like strings and lists).  So you can use the ``times2``
function as follows::

  >>> times2(4)
  8
  >>> times2('abc')
  'abcabc'
  >>> times2([1,2,3])
  [1, 2, 3, 1, 2, 3]

All parameters in Python are passed by reference. This is identical to
how Java does it with object parameters. However, while Java does
support passing unboxed primitive types by value, there are no such
entities in Python. Everything is an object in Python.

Functions are objects too, and they can be passed as parameters::

  XXX passing a function as a parameter - We can simply pass its name, then in the function using it
  
XX JJ:  Try this example here:
::
    
    # Define a function that takes two values and a mathematical function
    >>> def perform_calc(value1, value2, func):
    ...     return func(value1, value2)
    ...
    # Define a mathematical function to pass
    >>> def mult_values(value1, value2):
    ...     return value1 * value2
    ... 
    >>> perform_calc(2, 4, mult_values)
    8
    
    # Define another mathematical function to pass
    >>> def add_values(value1, value2):
    ...     return value1 + value2
    ... 
    >>> perform_calc(2, 4, add_values) 
    6
    >>> 


If you have more than two or so arguments, it often makes more sense
to call a function by parameter, rather than by the defined
order. This tends to create more robust code. So if you have a
function ``draw_point(x,y)``, you might want to call it as
``draw_point(x=10,y=20)``.

Defaults further simplify calling a function. You use the form of
``param=default_value`` when defining the function. For instance, you
might take our ``times2`` function and generalize it::

  def times_by(n, by=2):
      return n * by

This function is equivalent to ``times2`` when called using that
default value.

There's one point to remember that oftens trips up developers. The
default value is initialized exactly once, when the function is
defined. That's certainly fine for immutable values like numbers,
strings, tuples, frozensets, and similar objects. But you need to
ensure that if the default value is mutable, that it's being used in
this fashion correctly. So a dictionary for a shared cache makes
sense. But this mechanism won't work for but a list where we expect it
is initialized to an empty list upon invocation. If you're doing that,
you need to write that explicitly in your code.

Lastly, a function can take an unspecified number of ordered
arguments, through ``*args``, and keyword args, through
``**kwargs``. These parameter names (``args`` and ``kwargs``) are conventional, so you can use whatever name makes sense for your function. The markers ``*`` and ``**`` are used to to determine that this functionality should be used.

  XXX by factors



Calling Functions - Recursion
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The code definition is separate from the name of the function.
This distinction proves to be useful for decorators, as we will see later.

  XXX Recursion. (I think it makes sense to not focus on recursion too
  much; it may be a fundamental aspect of computer science, but it's
  also rarely necessary for most end-user software development. So
  let's keep it in a sidebar.)  Demo Fibonacci, since this requires no
  explanation, and it's a non trivial use of recursion.

  Note that Jython, like CPython, is ultimately stack based [at least
  until we have some tail call optimization support in JVM]. Recursion
  can be useful for expressing an algorithm compactly, but deeply
  recursive solutions on Jython can exhaust the JVM stack.

   Memoization, as we will discuss with decorators, can make a
   recursive solution practical, however.


Function Body
~~~~~~~~~~~~~

Documenting Functions
^^^^^^^^^^^^^^^^^^^^^

First, you should specify a document string for the function. The
docstring, if it exists, is a string that occurs as the first value of
the function body::

   def times2(n):
       """Given n, returns n * 2"""
       return n * 2

By convention, use triple-quoted strings, even if your docstring is
not multiline. If it is multiline, this is how we recommend you format it::

   def fact(n):
       """Returns the factorial of n

       Computes the factorial of n recursively. Does not check its
       arguments if nonnegative integer or if would stack
       overflow. Use with care! 
       """

       if n in (0, 1):
           return 1
       else:
           return n * fact(n - 1)

Any such docstring, but with leading indendetation stripped, becomes
the ``__doc__`` attribute of that function object. Incidentally,
docstrings are also used for modules and classes, and they work
exactly the same way.

You can now use the ``help`` built-in function to get the docstring,
or see them from various IDEs like PyDev for Eclipse and nbPython for
NetBeans as part of the auto-complete::

  XXX help(fact)

Returning Values
^^^^^^^^^^^^^^^^

All functions return some value.

 In
``times2``, we use the ``return`` statement to exit the function with
that value.

Functions can easily return multiple values at once by returning a tuple or
other structure::

  XXX especially show the construct return x, y - this is an elegant way to do multiple values
  
XX JJ:  A simple example returning two values

::

    >>> def calc_tips(amount):
    ...     return (amount * .18), (amount * .20)
    ... 
    >>> calc_tips(25.25)
    (4.545, 5.050000000000001)


A function can return at any time::

  XXX

And it can also return any object as its value. So you can have a
function that looks like this::

  XXX think of an interesting, simple function that returns different values based on input
  
XX JJ: In this example, we rewrite the perform_calc function to accept only positive numbers
       and return an error message if a negative number is passed into it.

::

    >>> def check_pos_perform_calc(num1, num2, func):                          
    ...     if num1 > 0 and num2 > 0:                                          
    ...         return func(num1, num2)                                        
    ...     else:                                                              
    ...         return 'Only positive numbers can be used with this function!' 
    ... 
    >>> def mult_values(value1, value2):
    ...     return value1 * value2
    ... 
    >>> check_pos_perform_calc(3, 4, mult_values)
    12
    >>> check_pos_perform_calc(3, -44, mult_values)
    'Only positive numbers can be used with this function!'


 If a return statement is not used, the value ``None`` is returned. There is no
equivalent to a ``void`` method in Java, because every function in Python
returns a value. However, the Python console will not show the return
value when it's ``None``, so you need to explicitly print it to see
what is returned::

   >>> do_nothing()
   >>> print do_nothing()
   None

A delighter in Python is the ease by which it enables returning multiple values::

  XXX function - return a, b

XX JJ:  Already covered this, no?

We can then readily unpack the return value.


Introducing Variables
^^^^^^^^^^^^^^^^^^^^^

XXX local variables - extend this with discussion 
XXX global variables

A function introduces a scope for new names, such as variables. Any
names that are created in the function are only visible within that
scope::

  XXX scope

XX JJ:  In the following example, the sq variable is defined within the
        scope of the function definition itself.  If we try to use it outside
        of the function then we'll receive an error.
::

    >>> def square_num(num):
    ...     sq = num * num
    ...     return sq                  
    ... 
    >>> square_num(35)
    1225
    >>> sq
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    NameError: name 'sq' is not defined

(Example showing a syntax error...)


.. sidebar:: Global Variables

  global keyword - [Useful for certain circumstances, certainly not
  core/essential, much like nonlocal in Py3K, so let's not put too
  much focus on it.]

  The `global` keyword is used to declare that a variable name is from
  the module scope (or script) containing this function. Using
  `global` is rarely necessary in practice, since it is not necessary
  if the name is called as a function or an attribute is accessed
  (through dotted notation).

  This is a good example of where Python is providing a complex
  balancing between a complex idea - the lexical scoping of names, and
  the operations on them - and the fact that in practice it is doing
  the right thing.
  
  XXX rewrite above, confusing


Functions create scopes for their variables.
Assigning a variable, just like in a simple script, implicitly

XX JJ:  I'd say that detail can be left out...the example we provided
        above demonstrates function scope


Other Statements
^^^^^^^^^^^^^^^^

.. sidebar:: Acceptable Statements
 
  So what can go in a function body? Pretty much any statement,
  including material that we will cover later in this book. So you can
  define functions or classes or use even import, within the scope of
  that function.

  In particular, performing a potentially expensive operation like
  import as last as possible, can reduce the startup time of your
  app. It's even possible it will be never needed too.

  There are a couple of exceptions to this rule. In both cases, these
  statements must go at the beginning of a module, similar to what we
  see in a static language like Java:

    * Compiler directives. Python supports a limited set of compiler
      directives that have the provocative syntax of ``from __future__
      import X``; see :pep:`236`. These are features that will be
      eventually be made available, generally in the next minor
      revision (such as 2.5 to 2.6). In addition, it's a popular place
      to put Easter eggs, such as ``from __future__ import
      braces``. (Try it in the console, which also relaxes what it
      means to be performed at the beginning.)

    * Source encoding declaration. Although technically not a
      statement -- it's in a specially parsed comment -- this must go
      in the first or second line.


Empty Functions
^^^^^^^^^^^^^^^

An empty function still needs something in its body. You can use the
``pass`` statement::

  def do_nothing():
      pass # here's how to specify an empty body of code

Or you can just have a docstring for the function body::

  def empty_callback(*args, **kwargs):
      """Use this function where we need to supply a callback,
         but have nothing further to do.
      """

Why have a function that does nothing? As in math, it's useful to have
an operation that stands for doing nothing, like "add zero" or
"multiply by one". These identity functions eliminate special
cases. Likewise, as see with ``empty_callback``, we may need to
specify a callback function when calling an API, but nothing actually
needs to be done. By passing in an empty function -- or having this be
the default -- we can simplify the API.




Miscellaneous
^^^^^^^^^^^^^

XXX various limits
XXX currently limits of 64K java bytecode instructions when compiled. this will be relaxed in a future version

.. sidebar:: What do functions look like from Java?

  They are instances of PyObject, supporting the ``__call__`` method.

  Additional introspection is available. If a function object is just
  a standard function written in Python, it will be of class
  PyFunction. A builtin function will be of class
  PyBuiltinFunction. But don't assume that in your code, because many
  other objects support the function interface (``__call__``), and
  these potentially could be proxying, perhaps several layers deep, a
  given function. You can only assume it's a PyObject.
  
XX JJ:  This works in a sidebar, but is too much info for a beginner.


Builtin Functions
-----------------

Builtin functions are those functions that are always in the Python
namespace. In other words, these functions -- and builtin exceptions,
boolean values, and some other objects -- are the only truly globally
defined names. If you are familiar with Java, they are somewhat like
the classes from ``java.lang``.

Builtins are rarely sufficient, however; even a simple command line
script generally needs to parse its arguments or read in from its
standard input. So for this case you would need to ``import sys``. And
in the context of Jython, you will need to import the relevant Java
classes you are using, perhaps with ``import java``. But the
builtin functions are really the core function that almost all Python
code uses.

.. include:: builtins.rst

XX JJ:  Maybe this can be a separate chapter?

XXX let's just pull in the actual documentation, then modify/augment
as desired. I still prefer the grouping that we are doing here,
especially if we can create an index.

XXX Let's list these by functionality, that is 
Group by functionality; this is the standard docs, augmented by our
perspectives on how to use them.


Alternative Ways to Define Functions
------------------------------------

The :keyword:`def` keyword is not the only way to define a function. Here are
some alternatives:

   * :keyword:`lambda` functions. The :keyword:`lambda` keyword creates an unnamed
     function. Some people like this because it requires minimal
     space, especially when used in a callback::

     XXX lambda in a keyed sort, maybe combine last name, first name?
     
XX JJ:  Example of using a lambda function to combine two strings.  In this case, a first
        and last name

::

    >>> name_combo = lambda first,last: first + ' ' + last
    >>> name_combo('Jim','Baker')
    'Jim Baker'
    
     XXX gen exp ex

   * Classes. In addition, we can also create objects with classes
     whose instance objects look like ordinary functions.  Objects
     supporting the __call__ protocol. This should be covered in a
     later chapter.  For Java developers, this is familiar. Classes
     implement such single-method interfaces as Callable or Runnable.
     
   * Bound methods. Instead of calling x.a(), I can pass x.a as a
     parameter or bind to another name. Then I can invoke this
     name. The first parameter of the method will be passed the bound
     object, which in OO terms is the receiver of the method. This is
     a simple way of creating callbacks. (In Java you would have just
     passed the object of course, then having the callback invoke the
     appropriate method such as `call` or `run`.)

   * staticmethod, classmethod, descriptors functools, such as for
     partial construction.

   * Other function constructors, including yours?



Lambda Functions
~~~~~~~~~~~~~~~~

Limitations.

XX JJ:  I imagine that this will be filled out, should the lambda function from
        above be moved into this section?


Generator Functions
-------------------

Generators are functions that construct objects implementing Python's
iterator protocol.

iter() - obj.__iter__
Call obj.next


Advance to the next point by calling the special method
``next``. Usually that's done implicitly, typically through a loop or
a consuming function that accepts iterators, including generators.

XX JJ:  More explanation needed here.  It would be a good idea to mention
        that StopIteration is thrown when a generator has been "used up" if the
        generator is not being used in a loop.


Defining Generators
^^^^^^^^^^^^^^^^^^^

A generator function is written so that it consists of one or more
yield points, which are marked through the use of the keyword
``yield``::

  def g():
      print "before yield point 1"
      yield 1
      print "after 1, before 2"
      yield 2
      yield 3


XX JJ:  Perhaps anotherp more useful example:

::
    
    >>> def step_to(factor, stop):
    ...     step = factor
    ...     start = 0
    ...     while start <= stop:
    ...         yield start
    ...         start += step
    ...
    >>> for x in step_to(1, 10):
    ...     print x
    ... 
    0
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    >>> for x in step_to(2, 10):
    ...     print x
    ... 
    0
    2
    4
    6
    8
    10
    >>> 


If the ``yield`` keyword is seen in the scope of a function, it's
compiled as if it's a generator function.

Unlike other functions, you use the ``return`` statement only to say,
"I'm done", that is, to exit the generator::

  XXX code
  
XX JJ:  Let's change the step_to function just a bit to check and ensure
        that the factor is less than the stopping point.  We'll add a return
        statement to exit the generator if the factor is gt or equal to the stop.
::
    
        >>> def step_return(factor, stop):
        ...     step = factor             
        ...     start = 0                 
        ...     if factor >= stop:
        ...         return
        ...     while start <= stop:
        ...         yield start
        ...         start += step
        ... 
        >>> for x in step_return(1,10):
        ...     print x
        ... 
        0
        1
        2
        3
        4
        5
        6
        7
        8
        9
        10
        >>> for x in step_return(3,10):
        ...     print x
        ... 
        0
        3
        6
        9
        >>> for x in step_return(3,3):
        ...     print x
        ... 

  


You can't return an argument::

  def g():
      yield 1
      yield 2
      return None

  for i in g():
      print i
  
  SyntaxError: 'return' with argument inside generator

But it's not necessary to explicitly return. You can think of
``return`` as acting like a ``break`` in a for-loop or while-loop.

Many useful generators actually will have an infinite loop around
their yield expression, instead of ever exiting, explicitly or not::

  XXX while True:
     yield stuff

This works because a generator object can be garbage collected, just
like any other object implementing the iteration protocol. The fact
that it uses the machinery of function objects to implement itself
doesn't matter.



.. sidebar:: How it actually works

  Generators are actually compiled differently from other
  functions. Each yield point saves the state of unnamed local
  variables (Java temporaries) into the frame object, then returns the
  value to the function that had called ``next`` (or ``send`` in the
  case of a coroutine). The generator is then indefinitely suspended,
  just like any other iterator. Upon calling next again, the generator
  is resumed by restoring these local variables, then executing the
  next bytecode instruction following the yield point. This process
  continues until the generator is either garbage collected or it
  exits.

  You can determine if the underlying function is a generator if its
  code object has the ``CO_GENERATOR`` flag set in ``co_flags``.

  Generators can also be resumed from any thread, although some care
  is necessary to ensure that underlying system state is shared (or
  compatible). We will explore how to use effectively use this
  capability in the chapter on concurrency.


Generator Expressions
^^^^^^^^^^^^^^^^^^^^^

This is an alternative way to create the generator object. Please note
this is not a generator function! It's the equivalent to what a
generator function returns when called.

XX JJ:  Maybe we should say "yields" when called?  

. Creates an unnamed generator. But cover
this later with respect to generators. Note that generators are not callable objects::

  >>> x = (2 * x for x in [1,2,3,4])
  >>> x
  <generator object at 0x1>
  >>> x()
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
  TypeError: 'generator' object is not callable
  
XX JJ:  Show call to x.next()

::
    
    >>> for v in x:
    ...     print v
    ... 
    2
    4
    6
    8
    >>> 




Using Generators
~~~~~~~~~~~~~~~~

Python iteration protocol. iter, next.


Generator Example
~~~~~~~~~~~~~~~~~

contextlib

Jar scanner

How to use in interesting ways with Java. For example, we wrap
everything in Java that supports ``java.util.Iterator`` so it supports
the Python iteration protocol.

Maybe something simple like walking a directory tree?

XX JJ:  I like the walker idea.  Perhaps a different implementation of Frank's from chapter 7?

In conjunction with glob type functionality? And possibly other analysis.
Maybe process every single file, etc.
That could be sort of cool, and something I don't think is so easy from Java (no, it's not).
Also we will want to wrap it up with RAII semantics too, to ensure closing.

Lastly - what sort of Java client code would want such an iterator? That's the other part of the equation to be solved here.
Maybe some sort of plugin?
Don't want to make the example too contrived.
Some relevant discussion here in a Java tutorial: http://java.sun.com/docs/books/tutorial/essential/io/walk.html

What about a simple Jar scanner? That's sort of handy... and feeds into other functionality too.
Could be the subject of Ant integration too. (Or Maven or Ivy, but perhaps this is going beyond my knowledge here.)

One common usage of a generator is to watch a log file for changes (tail -f). We can create something similar with the NIO package, although this does require the use of a thread for the watcher (but this of course can be multiplexed across multiple directories).

Watching a directory for changes. In CPython, this requires fcntl on Unix/Linux systems, and the use of a completely different Win32 API on Windows systems. http://stackoverflow.com/questions/182197/how-do-i-watch-a-file-for-changes-using-python Java provides a simple approach:
http://java.sun.com/docs/books/tutorial/essential/io/notification.html  - how to do it in Java


Generator Expressions
~~~~~~~~~~~~~~~~~~~~~

XXX Maybe something simple with Java Mail? Could show how to attach files that meet a certain criteria?


Namespaces, Nested Scopes and Closures
--------------------------------------

Functions can be nested.

Most importantly this allows the construction of closures.


Namespaces
Note that you can introduce other namespaces into your function definition. So::

  def f():
      from NS import A, B


Function Decorators
-------------------

Function decorators are two things:

 * A convenient syntax that describes how to transform a function. You
   might want to *memoize* a given function, so it uses a cache, with
   a desired policy, to remember a result for a given set of
   parameters. Or you may want to create a static method in a class.

 * A powerful, yet simple design where the decorator is a function on
   function that results in the decorated, or transformed, function.

(Class decorators are similar, except they are functions on classes).

XXX example - XXX How about a decorator for Java integration? eg add support of a given interface to facilitate callbacks


Creating Decorators
~~~~~~~~~~~~~~~~~~~

Memoization decorator. For our same Fibonacci example.


Often a function definition is not the simplest way to write the
desired decorator function. Instead, you might want to create a class,
as we described in alternate ways to create function objects.

XXX In addition, ``functools``, specifically the ``wraps`` function.

XXX ref Eckel's article on decorators.


Using Decorators
~~~~~~~~~~~~~~~~







XXX Chopping block


Coroutines
----------

 One thing
to remember: coroutines do not mix with generators, despite being
related in both syntax and implementation. Coroutines use push;
generators use pull.

XXX The PyCon tutorial on coroutines has some useful coroutine
examples - certainly need similar coverage.

XXX Might be nice to show how to use this in
conjunction with parallelism. but that's a later chapter anyway

XX JJ: I think that Coroutines need to be in this book in some form...don't cut them out.  Even
       if it is just a short section with an example.


Advanced Function Usage
-----------------------

Frames
Tracebacks
Profiling and tracing
Introspection on functions - various attributes, etc, not to mention the use of inspect


.. sidebar:: Function Code Bodies

  Jython, like CPython, only has one unit of compilation, the function
  code body. When a module is compiled, every function in it is
  compiled even if it's not ultimately bound to a name. In addition, a
  script or module is itself treated as a function when
  compiled. These function definitions are compiled to Java
  bytecode. (There's experimental support for other formats, namely
  Python bytecode, which we may see be used in later versions of
  Jython.)


.. sidebar:: Partitioning this global namespace with shadowing.



  The inspect module. Determining parameters, etc.
  One thing that is not supported: introspecting on code objects themselves.






.. sidebar::

  Behind the scenes, this function application is compiled to
  x.__call__(*args, **kwargs), and that's how it's called from Java. A
  convenience method is also provided, invoke, that combines method
  lookup and dispatch together. So you can directly call Python
  functions from Java code in this way. We will look at this more in
  the chapter on Java integration.





.. sidebar:: Special syntax support for operators

  x.a
  del x
  x[i]
  etc.

  All operators are also available as functions from the :module:`operator`. 
  It should be noted that operators on built-in types (int, str, dict,
  etc.) will usually execute faster on the JVM because they do not
  require dynamic dispatch. Invokedynamic, part of JDK 7, is exciting
  because it makes that cost go away, but we will have to wait for
  that. 
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.