1. Doug Hellmann
  2. sphinx-graphviz-paragraphs


georg.brandl  committed dc0a21b

Integrate the HOWTOs into the tree.

  • Participants
  • Parent commits 17ca679
  • Branches default

Comments (0)

Files changed (25)

File Doc-26/ACKS

View file
 * Steve Alexander
 * Jim Ahlstrom
 * Fred Allen
-* A. Amoroso
+* \A. Amoroso
 * Pehr Anderson
 * Oliver Andrich
 * Jesús Cea Avión
 * Jeremy Craven
 * Andrew Dalke
 * Ben Darnell
-* L. Peter Deutsch
+* \L. Peter Deutsch
 * Robert Donohue
 * Fred L. Drake, Jr.
 * Jeff Epler

File Doc-26/TODO

View file
 * split very large files and add toctrees
-* integrate standalone HOWTOs
 * find out which files get "comments disabled" metadata
 * write "About these documents"
 * finish "Documenting Python"

File Doc-26/contents.rst

View file
+   howto/index.rst

File Doc-26/howto/advocacy.rst

View file
+  Python Advocacy HOWTO
+:Author: A.M. Kuchling
+:Release: 0.03
+.. topic:: Abstract
+   It's usually difficult to get your management to accept open source software,
+   and Python is no exception to this rule.  This document discusses reasons to use
+   Python, strategies for winning acceptance, facts and arguments you can use, and
+   cases where you *shouldn't* try to use Python.
+Reasons to Use Python
+There are several reasons to incorporate a scripting language into your
+development process, and this section will discuss them, and why Python has some
+properties that make it a particularly good choice.
+Programs are often organized in a modular fashion.  Lower-level operations are
+grouped together, and called by higher-level functions, which may in turn be
+used as basic operations by still further upper levels.
+For example, the lowest level might define a very low-level set of functions for
+accessing a hash table.  The next level might use hash tables to store the
+headers of a mail message, mapping a header name like ``Date`` to a value such
+as ``Tue, 13 May 1997 20:00:54 -0400``.  A yet higher level may operate on
+message objects, without knowing or caring that message headers are stored in a
+hash table, and so forth.
+Often, the lowest levels do very simple things; they implement a data structure
+such as a binary tree or hash table, or they perform some simple computation,
+such as converting a date string to a number.  The higher levels then contain
+logic connecting these primitive operations.  Using the approach, the primitives
+can be seen as basic building blocks which are then glued together to produce
+the complete product.
+Why is this design approach relevant to Python?  Because Python is well suited
+to functioning as such a glue language.  A common approach is to write a Python
+module that implements the lower level operations; for the sake of speed, the
+implementation might be in C, Java, or even Fortran.  Once the primitives are
+available to Python programs, the logic underlying higher level operations is
+written in the form of Python code.  The high-level logic is then more
+understandable, and easier to modify.
+John Ousterhout wrote a paper that explains this idea at greater length,
+entitled "Scripting: Higher Level Programming for the 21st Century".  I
+recommend that you read this paper; see the references for the URL.  Ousterhout
+is the inventor of the Tcl language, and therefore argues that Tcl should be
+used for this purpose; he only briefly refers to other languages such as Python,
+Perl, and Lisp/Scheme, but in reality, Ousterhout's argument applies to
+scripting languages in general, since you could equally write extensions for any
+of the languages mentioned above.
+In *The Mythical Man-Month*, Fredrick Brooks suggests the following rule when
+planning software projects: "Plan to throw one away; you will anyway."  Brooks
+is saying that the first attempt at a software design often turns out to be
+wrong; unless the problem is very simple or you're an extremely good designer,
+you'll find that new requirements and features become apparent once development
+has actually started.  If these new requirements can't be cleanly incorporated
+into the program's structure, you're presented with two unpleasant choices:
+hammer the new features into the program somehow, or scrap everything and write
+a new version of the program, taking the new features into account from the
+Python provides you with a good environment for quickly developing an initial
+prototype.  That lets you get the overall program structure and logic right, and
+you can fine-tune small details in the fast development cycle that Python
+provides.  Once you're satisfied with the GUI interface or program output, you
+can translate the Python code into C++, Fortran, Java, or some other compiled
+Prototyping means you have to be careful not to use too many Python features
+that are hard to implement in your other language.  Using ``eval()``, or regular
+expressions, or the :mod:`pickle` module, means that you're going to need C or
+Java libraries for formula evaluation, regular expressions, and serialization,
+for example.  But it's not hard to avoid such tricky code, and in the end the
+translation usually isn't very difficult.  The resulting code can be rapidly
+debugged, because any serious logical errors will have been removed from the
+prototype, leaving only more minor slip-ups in the translation to track down.
+This strategy builds on the earlier discussion of programmability. Using Python
+as glue to connect lower-level components has obvious relevance for constructing
+prototype systems.  In this way Python can help you with development, even if
+end users never come in contact with Python code at all.  If the performance of
+the Python version is adequate and corporate politics allow it, you may not need
+to do a translation into C or Java, but it can still be faster to develop a
+prototype and then translate it, instead of attempting to produce the final
+version immediately.
+One example of this development strategy is Microsoft Merchant Server. Version
+1.0 was written in pure Python, by a company that subsequently was purchased by
+Microsoft.  Version 2.0 began to translate the code into C++, shipping with some
+C++code and some Python code.  Version 3.0 didn't contain any Python at all; all
+the code had been translated into C++.  Even though the product doesn't contain
+a Python interpreter, the Python language has still served a useful purpose by
+speeding up development.
+This is a very common use for Python.  Past conference papers have also
+described this approach for developing high-level numerical algorithms; see
+David M. Beazley and Peter S. Lomdahl's paper "Feeding a Large-scale Physics
+Application to Python" in the references for a good example.  If an algorithm's
+basic operations are things like "Take the inverse of this 4000x4000 matrix",
+and are implemented in some lower-level language, then Python has almost no
+additional performance cost; the extra time required for Python to evaluate an
+expression like ``m.invert()`` is dwarfed by the cost of the actual computation.
+It's particularly good for applications where seemingly endless tweaking is
+required to get things right. GUI interfaces and Web sites are prime examples.
+The Python code is also shorter and faster to write (once you're familiar with
+Python), so it's easier to throw it away if you decide your approach was wrong;
+if you'd spent two weeks working on it instead of just two hours, you might
+waste time trying to patch up what you've got out of a natural reluctance to
+admit that those two weeks were wasted.  Truthfully, those two weeks haven't
+been wasted, since you've learnt something about the problem and the technology
+you're using to solve it, but it's human nature to view this as a failure of
+some sort.
+Simplicity and Ease of Understanding
+Python is definitely *not* a toy language that's only usable for small tasks.
+The language features are general and powerful enough to enable it to be used
+for many different purposes.  It's useful at the small end, for 10- or 20-line
+scripts, but it also scales up to larger systems that contain thousands of lines
+of code.
+However, this expressiveness doesn't come at the cost of an obscure or tricky
+syntax.  While Python has some dark corners that can lead to obscure code, there
+are relatively few such corners, and proper design can isolate their use to only
+a few classes or modules.  It's certainly possible to write confusing code by
+using too many features with too little concern for clarity, but most Python
+code can look a lot like a slightly-formalized version of human-understandable
+In *The New Hacker's Dictionary*, Eric S. Raymond gives the following definition
+for "compact":
+.. epigraph::
+   Compact *adj.*  Of a design, describes the valuable property that it can all be
+   apprehended at once in one's head. This generally means the thing created from
+   the design can be used with greater facility and fewer errors than an equivalent
+   tool that is not compact. Compactness does not imply triviality or lack of
+   power; for example, C is compact and FORTRAN is not, but C is more powerful than
+   FORTRAN. Designs become non-compact through accreting features and cruft that
+   don't merge cleanly into the overall design scheme (thus, some fans of Classic C
+   maintain that ANSI C is no longer compact).
+   (From http://www.catb.org/ esr/jargon/html/C/compact.html)
+In this sense of the word, Python is quite compact, because the language has
+just a few ideas, which are used in lots of places.  Take namespaces, for
+example.  Import a module with ``import math``, and you create a new namespace
+called ``math``.  Classes are also namespaces that share many of the properties
+of modules, and have a few of their own; for example, you can create instances
+of a class. Instances?  They're yet another namespace.  Namespaces are currently
+implemented as Python dictionaries, so they have the same methods as the
+standard dictionary data type: .keys() returns all the keys, and so forth.
+This simplicity arises from Python's development history.  The language syntax
+derives from different sources; ABC, a relatively obscure teaching language, is
+one primary influence, and Modula-3 is another.  (For more information about ABC
+and Modula-3, consult their respective Web sites at http://www.cwi.nl/
+steven/abc/ and http://www.m3.org.)  Other features have come from C, Icon,
+Algol-68, and even Perl.  Python hasn't really innovated very much, but instead
+has tried to keep the language small and easy to learn, building on ideas that
+have been tried in other languages and found useful.
+Simplicity is a virtue that should not be underestimated.  It lets you learn the
+language more quickly, and then rapidly write code, code that often works the
+first time you run it.
+Java Integration
+If you're working with Java, Jython (http://www.jython.org/) is definitely worth
+your attention.  Jython is a re-implementation of Python in Java that compiles
+Python code into Java bytecodes.  The resulting environment has very tight,
+almost seamless, integration with Java.  It's trivial to access Java classes
+from Python, and you can write Python classes that subclass Java classes.
+Jython can be used for prototyping Java applications in much the same way
+CPython is used, and it can also be used for test suites for Java code, or
+embedded in a Java application to add scripting capabilities.
+Arguments and Rebuttals
+Let's say that you've decided upon Python as the best choice for your
+application.  How can you convince your management, or your fellow developers,
+to use Python?  This section lists some common arguments against using Python,
+and provides some possible rebuttals.
+**Python is freely available software that doesn't cost anything. How good can
+it be?**
+Very good, indeed.  These days Linux and Apache, two other pieces of open source
+software, are becoming more respected as alternatives to commercial software,
+but Python hasn't had all the publicity.
+Python has been around for several years, with many users and developers.
+Accordingly, the interpreter has been used by many people, and has gotten most
+of the bugs shaken out of it.  While bugs are still discovered at intervals,
+they're usually either quite obscure (they'd have to be, for no one to have run
+into them before) or they involve interfaces to external libraries.  The
+internals of the language itself are quite stable.
+Having the source code should be viewed as making the software available for
+peer review; people can examine the code, suggest (and implement) improvements,
+and track down bugs.  To find out more about the idea of open source code, along
+with arguments and case studies supporting it, go to http://www.opensource.org.
+**Who's going to support it?**
+Python has a sizable community of developers, and the number is still growing.
+The Internet community surrounding the language is an active one, and is worth
+being considered another one of Python's advantages. Most questions posted to
+the comp.lang.python newsgroup are quickly answered by someone.
+Should you need to dig into the source code, you'll find it's clear and well-
+organized, so it's not very difficult to write extensions and track down bugs
+yourself.  If you'd prefer to pay for support, there are companies and
+individuals who offer commercial support for Python.
+**Who uses Python for serious work?**
+Lots of people; one interesting thing about Python is the surprising diversity
+of applications that it's been used for.  People are using Python to:
+* Run Web sites
+* Write GUI interfaces
+* Control number-crunching code on supercomputers
+* Make a commercial application scriptable by embedding the Python interpreter
+  inside it
+* Process large XML data sets
+* Build test suites for C or Java code
+Whatever your application domain is, there's probably someone who's used Python
+for something similar.  Yet, despite being useable for such high-end
+applications, Python's still simple enough to use for little jobs.
+See http://wiki.python.org/moin/OrganizationsUsingPython for a list of some of
+the  organizations that use Python.
+**What are the restrictions on Python's use?**
+They're practically nonexistent.  Consult the :file:`Misc/COPYRIGHT` file in the
+source distribution, or http://www.python.org/doc/Copyright.html for the full
+language, but it boils down to three conditions.
+* You have to leave the copyright notice on the software; if you don't include
+  the source code in a product, you have to put the copyright notice in the
+  supporting documentation.
+* Don't claim that the institutions that have developed Python endorse your
+  product in any way.
+* If something goes wrong, you can't sue for damages.  Practically all software
+  licences contain this condition.
+Notice that you don't have to provide source code for anything that contains
+Python or is built with it.  Also, the Python interpreter and accompanying
+documentation can be modified and redistributed in any way you like, and you
+don't have to pay anyone any licensing fees at all.
+**Why should we use an obscure language like Python instead of well-known
+language X?**
+I hope this HOWTO, and the documents listed in the final section, will help
+convince you that Python isn't obscure, and has a healthily growing user base.
+One word of advice: always present Python's positive advantages, instead of
+concentrating on language X's failings.  People want to know why a solution is
+good, rather than why all the other solutions are bad.  So instead of attacking
+a competing solution on various grounds, simply show how Python's virtues can
+Useful Resources
+   The Python Success Stories are a collection of stories from successful users of
+   Python, with the emphasis on business and corporate users.
+.. % \term{\url{http://www.fsbassociates.com/books/pythonchpt1.htm}}
+.. % The first chapter of \emph{Internet Programming with Python} also
+.. % examines some of the reasons for using Python.  The book is well worth
+.. % buying, but the publishers have made the first chapter available on
+.. % the Web.
+   John Ousterhout's white paper on scripting is a good argument for the utility of
+   scripting languages, though naturally enough, he emphasizes Tcl, the language he
+   developed.  Most of the arguments would apply to any scripting language.
+   The authors, David M. Beazley and Peter S. Lomdahl,  describe their use of
+   Python at Los Alamos National Laboratory. It's another good example of how
+   Python can help get real work done. This quotation from the paper has been
+   echoed by many people:
+   .. epigraph::
+      Originally developed as a large monolithic application for massively parallel
+      processing systems, we have used Python to transform our application into a
+      flexible, highly modular, and extremely powerful system for performing
+      simulation, data analysis, and visualization. In addition, we describe how
+      Python has solved a number of important problems related to the development,
+      debugging, deployment, and maintenance of scientific software.
+   This interview with Andy Feit, discussing Infoseek's use of Python, can be used
+   to show that choosing Python didn't introduce any difficulties into a company's
+   development process, and provided some substantial benefits.
+.. % \term{\url{http://www.python.org/psa/Commercial.html}}
+.. % Robin Friedrich wrote this document on how to support Python's use in
+.. % commercial projects.
+   For the 6th Python conference, Greg Stein presented a paper that traced Python's
+   adoption and usage at a startup called eShop, and later at Microsoft.
+   Management may be doubtful of the reliability and usefulness of software that
+   wasn't written commercially.  This site presents arguments that show how open
+   source software can have considerable advantages over closed-source software.
+   The Linux Advocacy mini-HOWTO was the inspiration for this document, and is also
+   well worth reading for general suggestions on winning acceptance for a new
+   technology, such as Linux or Python.  In general, you won't make much progress
+   by simply attacking existing systems and complaining about their inadequacies;
+   this often ends up looking like unfocused whining.  It's much better to point
+   out some of the many areas where Python is an improvement over other systems.

File Doc-26/howto/curses.rst

View file
+  Curses Programming with Python
+:Author: A.M. Kuchling, Eric S. Raymond
+:Release: 2.02
+.. topic:: Abstract
+   This document describes how to write text-mode programs with Python 2.x, using
+   the :mod:`curses` extension module to control the display.
+What is curses?
+The curses library supplies a terminal-independent screen-painting and keyboard-
+handling facility for text-based terminals; such terminals include VT100s, the
+Linux console, and the simulated terminal provided by X11 programs such as xterm
+and rxvt.  Display terminals support various control codes to perform common
+operations such as moving the cursor, scrolling the screen, and erasing areas.
+Different terminals use widely differing codes, and often have their own minor
+In a world of X displays, one might ask "why bother"?  It's true that character-
+cell display terminals are an obsolete technology, but there are niches in which
+being able to do fancy things with them are still valuable.  One is on small-
+footprint or embedded Unixes that  don't carry an X server.  Another is for
+tools like OS installers and kernel configurators that may have to run before X
+is available.
+The curses library hides all the details of different terminals, and provides
+the programmer with an abstraction of a display, containing multiple non-
+overlapping windows.  The contents of a window can be changed in various ways--
+adding text, erasing it, changing its appearance--and the curses library will
+automagically figure out what control codes need to be sent to the terminal to
+produce the right output.
+The curses library was originally written for BSD Unix; the later System V
+versions of Unix from AT&T added many enhancements and new functions. BSD curses
+is no longer maintained, having been replaced by ncurses, which is an open-
+source implementation of the AT&T interface.  If you're using an open-source
+Unix such as Linux or FreeBSD, your system almost certainly uses ncurses.  Since
+most current commercial Unix versions are based on System V code, all the
+functions described here will probably be available.  The older versions of
+curses carried by some proprietary Unixes may not support everything, though.
+No one has made a Windows port of the curses module.  On a Windows platform, try
+the Console module written by Fredrik Lundh.  The Console module provides
+cursor-addressable text output, plus full support for mouse and keyboard input,
+and is available from http://effbot.org/efflib/console.
+The Python curses module
+Thy Python module is a fairly simple wrapper over the C functions provided by
+curses; if you're already familiar with curses programming in C, it's really
+easy to transfer that knowledge to Python.  The biggest difference is that the
+Python interface makes things simpler, by merging different C functions such as
+:func:`addstr`, :func:`mvaddstr`, :func:`mvwaddstr`, into a single
+:meth:`addstr` method.  You'll see this covered in more detail later.
+This HOWTO is simply an introduction to writing text-mode programs with curses
+and Python. It doesn't attempt to be a complete guide to the curses API; for
+that, see the Python library guide's section on ncurses, and the C manual pages
+for ncurses.  It will, however, give you the basic ideas.
+Starting and ending a curses application
+Before doing anything, curses must be initialized.  This is done by calling the
+:func:`initscr` function, which will determine the terminal type, send any
+required setup codes to the terminal, and create various internal data
+structures.  If successful, :func:`initscr` returns a window object representing
+the entire screen; this is usually called ``stdscr``, after the name of the
+corresponding C variable. ::
+   import curses
+   stdscr = curses.initscr()
+Usually curses applications turn off automatic echoing of keys to the screen, in
+order to be able to read keys and only display them under certain circumstances.
+This requires calling the :func:`noecho` function. ::
+   curses.noecho()
+Applications will also commonly need to react to keys instantly, without
+requiring the Enter key to be pressed; this is called cbreak mode, as opposed to
+the usual buffered input mode. ::
+   curses.cbreak()
+Terminals usually return special keys, such as the cursor keys or navigation
+keys such as Page Up and Home, as a multibyte escape sequence.  While you could
+write your application to expect such sequences and process them accordingly,
+curses can do it for you, returning a special value such as
+:const:`curses.KEY_LEFT`.  To get curses to do the job, you'll have to enable
+keypad mode. ::
+   stdscr.keypad(1)
+Terminating a curses application is much easier than starting one. You'll need
+to call  ::
+   curses.nocbreak(); stdscr.keypad(0); curses.echo()
+to reverse the curses-friendly terminal settings. Then call the :func:`endwin`
+function to restore the terminal to its original operating mode. ::
+   curses.endwin()
+A common problem when debugging a curses application is to get your terminal
+messed up when the application dies without restoring the terminal to its
+previous state.  In Python this commonly happens when your code is buggy and
+raises an uncaught exception.  Keys are no longer be echoed to the screen when
+you type them, for example, which makes using the shell difficult.
+In Python you can avoid these complications and make debugging much easier by
+importing the module :mod:`curses.wrapper`.  It supplies a :func:`wrapper`
+function that takes a callable.  It does the initializations described above,
+and also initializes colors if color support is present.  It then runs your
+provided callable and finally deinitializes appropriately.  The callable is
+called inside a try-catch clause which catches exceptions, performs curses
+deinitialization, and then passes the exception upwards.  Thus, your terminal
+won't be left in a funny state on exception.
+Windows and Pads
+Windows are the basic abstraction in curses.  A window object represents a
+rectangular area of the screen, and supports various methods to display text,
+erase it, allow the user to input strings, and so forth.
+The ``stdscr`` object returned by the :func:`initscr` function is a window
+object that covers the entire screen.  Many programs may need only this single
+window, but you might wish to divide the screen into smaller windows, in order
+to redraw or clear them separately. The :func:`newwin` function creates a new
+window of a given size, returning the new window object. ::
+   begin_x = 20 ; begin_y = 7
+   height = 5 ; width = 40
+   win = curses.newwin(height, width, begin_y, begin_x)
+A word about the coordinate system used in curses: coordinates are always passed
+in the order *y,x*, and the top-left corner of a window is coordinate (0,0).
+This breaks a common convention for handling coordinates, where the *x*
+coordinate usually comes first.  This is an unfortunate difference from most
+other computer applications, but it's been part of curses since it was first
+written, and it's too late to change things now.
+When you call a method to display or erase text, the effect doesn't immediately
+show up on the display.  This is because curses was originally written with slow
+300-baud terminal connections in mind; with these terminals, minimizing the time
+required to redraw the screen is very important.  This lets curses accumulate
+changes to the screen, and display them in the most efficient manner.  For
+example, if your program displays some characters in a window, and then clears
+the window, there's no need to send the original characters because they'd never
+be visible.
+Accordingly, curses requires that you explicitly tell it to redraw windows,
+using the :func:`refresh` method of window objects.  In practice, this doesn't
+really complicate programming with curses much. Most programs go into a flurry
+of activity, and then pause waiting for a keypress or some other action on the
+part of the user.  All you have to do is to be sure that the screen has been
+redrawn before pausing to wait for user input, by simply calling
+``stdscr.refresh()`` or the :func:`refresh` method of some other relevant
+A pad is a special case of a window; it can be larger than the actual display
+screen, and only a portion of it displayed at a time. Creating a pad simply
+requires the pad's height and width, while refreshing a pad requires giving the
+coordinates of the on-screen area where a subsection of the pad will be
+displayed.   ::
+   pad = curses.newpad(100, 100)
+   #  These loops fill the pad with letters; this is
+   # explained in the next section
+   for y in range(0, 100):
+       for x in range(0, 100):
+           try: pad.addch(y,x, ord('a') + (x*x+y*y) % 26 )
+           except curses.error: pass
+   #  Displays a section of the pad in the middle of the screen
+   pad.refresh( 0,0, 5,5, 20,75)
+The :func:`refresh` call displays a section of the pad in the rectangle
+extending from coordinate (5,5) to coordinate (20,75) on the screen; the upper
+left corner of the displayed section is coordinate (0,0) on the pad.  Beyond
+that difference, pads are exactly like ordinary windows and support the same
+If you have multiple windows and pads on screen there is a more efficient way to
+go, which will prevent annoying screen flicker at refresh time.  Use the
+:meth:`noutrefresh` method of each window to update the data structure
+representing the desired state of the screen; then change the physical screen to
+match the desired state in one go with the function :func:`doupdate`.  The
+normal :meth:`refresh` method calls :func:`doupdate` as its last act.
+Displaying Text
+From a C programmer's point of view, curses may sometimes look like a twisty
+maze of functions, all subtly different.  For example, :func:`addstr` displays a
+string at the current cursor location in the ``stdscr`` window, while
+:func:`mvaddstr` moves to a given y,x coordinate first before displaying the
+string. :func:`waddstr` is just like :func:`addstr`, but allows specifying a
+window to use, instead of using ``stdscr`` by default. :func:`mvwaddstr` follows
+Fortunately the Python interface hides all these details; ``stdscr`` is a window
+object like any other, and methods like :func:`addstr` accept multiple argument
+forms.  Usually there are four different forms.
+| Form                            | Description                                   |
+| *str* or *ch*                   | Display the string *str* or character *ch* at |
+|                                 | the current position                          |
+| *str* or *ch*, *attr*           | Display the string *str* or character *ch*,   |
+|                                 | using attribute *attr* at the current         |
+|                                 | position                                      |
+| *y*, *x*, *str* or *ch*         | Move to position *y,x* within the window, and |
+|                                 | display *str* or *ch*                         |
+| *y*, *x*, *str* or *ch*, *attr* | Move to position *y,x* within the window, and |
+|                                 | display *str* or *ch*, using attribute *attr* |
+Attributes allow displaying text in highlighted forms, such as in boldface,
+underline, reverse code, or in color.  They'll be explained in more detail in
+the next subsection.
+The :func:`addstr` function takes a Python string as the value to be displayed,
+while the :func:`addch` functions take a character, which can be either a Python
+string of length 1 or an integer.  If it's a string, you're limited to
+displaying characters between 0 and 255.  SVr4 curses provides constants for
+extension characters; these constants are integers greater than 255.  For
+example, :const:`ACS_PLMINUS` is a +/- symbol, and :const:`ACS_ULCORNER` is the
+upper left corner of a box (handy for drawing borders).
+Windows remember where the cursor was left after the last operation, so if you
+leave out the *y,x* coordinates, the string or character will be displayed
+wherever the last operation left off.  You can also move the cursor with the
+:func:`move(y,x)` method.  Because some terminals always display a flashing
+cursor, you may want to ensure that the cursor is positioned in some location
+where it won't be distracting; it can be confusing to have the cursor blinking
+at some apparently random location.
+If your application doesn't need a blinking cursor at all, you can call
+:func:`curs_set(0)` to make it invisible.  Equivalently, and for compatibility
+with older curses versions, there's a :func:`leaveok(bool)` function.  When
+*bool* is true, the curses library will attempt to suppress the flashing cursor,
+and you won't need to worry about leaving it in odd locations.
+Attributes and Color
+Characters can be displayed in different ways.  Status lines in a text-based
+application are commonly shown in reverse video; a text viewer may need to
+highlight certain words.  curses supports this by allowing you to specify an
+attribute for each cell on the screen.
+An attribute is a integer, each bit representing a different attribute.  You can
+try to display text with multiple attribute bits set, but curses doesn't
+guarantee that all the possible combinations are available, or that they're all
+visually distinct.  That depends on the ability of the terminal being used, so
+it's safest to stick to the most commonly available attributes, listed here.
+| Attribute            | Description                          |
+| :const:`A_BLINK`     | Blinking text                        |
+| :const:`A_BOLD`      | Extra bright or bold text            |
+| :const:`A_DIM`       | Half bright text                     |
+| :const:`A_REVERSE`   | Reverse-video text                   |
+| :const:`A_STANDOUT`  | The best highlighting mode available |
+| :const:`A_UNDERLINE` | Underlined text                      |
+So, to display a reverse-video status line on the top line of the screen, you
+could code::
+   stdscr.addstr(0, 0, "Current mode: Typing mode",
+   	      curses.A_REVERSE)
+   stdscr.refresh()
+The curses library also supports color on those terminals that provide it, The
+most common such terminal is probably the Linux console, followed by color
+To use color, you must call the :func:`start_color` function soon after calling
+:func:`initscr`, to initialize the default color set (the
+:func:`curses.wrapper.wrapper` function does this automatically).  Once that's
+done, the :func:`has_colors` function returns TRUE if the terminal in use can
+actually display color.  (Note: curses uses the American spelling 'color',
+instead of the Canadian/British spelling 'colour'.  If you're used to the
+British spelling, you'll have to resign yourself to misspelling it for the sake
+of these functions.)
+The curses library maintains a finite number of color pairs, containing a
+foreground (or text) color and a background color.  You can get the attribute
+value corresponding to a color pair with the :func:`color_pair` function; this
+can be bitwise-OR'ed with other attributes such as :const:`A_REVERSE`, but
+again, such combinations are not guaranteed to work on all terminals.
+An example, which displays a line of text using color pair 1::
+   stdscr.addstr( "Pretty text", curses.color_pair(1) )
+   stdscr.refresh()
+As I said before, a color pair consists of a foreground and background color.
+:func:`start_color` initializes 8 basic colors when it activates color mode.
+They are: 0:black, 1:red, 2:green, 3:yellow, 4:blue, 5:magenta, 6:cyan, and
+7:white.  The curses module defines named constants for each of these colors:
+:const:`curses.COLOR_BLACK`, :const:`curses.COLOR_RED`, and so forth.
+The :func:`init_pair(n, f, b)` function changes the definition of color pair
+*n*, to foreground color f and background color b.  Color pair 0 is hard-wired
+to white on black, and cannot be changed.
+Let's put all this together. To change color 1 to red text on a white
+background, you would call::
+   curses.init_pair(1, curses.COLOR_RED, curses.COLOR_WHITE)
+When you change a color pair, any text already displayed using that color pair
+will change to the new colors.  You can also display new text in this color
+   stdscr.addstr(0,0, "RED ALERT!", curses.color_pair(1) )
+Very fancy terminals can change the definitions of the actual colors to a given
+RGB value.  This lets you change color 1, which is usually red, to purple or
+blue or any other color you like.  Unfortunately, the Linux console doesn't
+support this, so I'm unable to try it out, and can't provide any examples.  You
+can check if your terminal can do this by calling :func:`can_change_color`,
+which returns TRUE if the capability is there.  If you're lucky enough to have
+such a talented terminal, consult your system's man pages for more information.
+User Input
+The curses library itself offers only very simple input mechanisms. Python's
+support adds a text-input widget that makes up some of the lack.
+The most common way to get input to a window is to use its :meth:`getch` method.
+:meth:`getch` pauses and waits for the user to hit a key, displaying it if
+:func:`echo` has been called earlier.  You can optionally specify a coordinate
+to which the cursor should be moved before pausing.
+It's possible to change this behavior with the method :meth:`nodelay`. After
+:meth:`nodelay(1)`, :meth:`getch` for the window becomes non-blocking and
+returns ``curses.ERR`` (a value of -1) when no input is ready.  There's also a
+:func:`halfdelay` function, which can be used to (in effect) set a timer on each
+:meth:`getch`; if no input becomes available within the number of milliseconds
+specified as the argument to :func:`halfdelay`, curses raises an exception.
+The :meth:`getch` method returns an integer; if it's between 0 and 255, it
+represents the ASCII code of the key pressed.  Values greater than 255 are
+special keys such as Page Up, Home, or the cursor keys. You can compare the
+value returned to constants such as :const:`curses.KEY_PPAGE`,
+:const:`curses.KEY_HOME`, or :const:`curses.KEY_LEFT`.  Usually the main loop of
+your program will look something like this::
+   while 1:
+       c = stdscr.getch()
+       if c == ord('p'): PrintDocument()
+       elif c == ord('q'): break  # Exit the while()
+       elif c == curses.KEY_HOME: x = y = 0
+The :mod:`curses.ascii` module supplies ASCII class membership functions that
+take either integer or 1-character-string arguments; these may be useful in
+writing more readable tests for your command interpreters.  It also supplies
+conversion functions  that take either integer or 1-character-string arguments
+and return the same type.  For example, :func:`curses.ascii.ctrl` returns the
+control character corresponding to its argument.
+There's also a method to retrieve an entire string, :const:`getstr()`.  It isn't
+used very often, because its functionality is quite limited; the only editing
+keys available are the backspace key and the Enter key, which terminates the
+string.  It can optionally be limited to a fixed number of characters. ::
+   curses.echo()            # Enable echoing of characters
+   # Get a 15-character string, with the cursor on the top line 
+   s = stdscr.getstr(0,0, 15)  
+The Python :mod:`curses.textpad` module supplies something better. With it, you
+can turn a window into a text box that supports an Emacs-like set of
+keybindings.  Various methods of :class:`Textbox` class support editing with
+input validation and gathering the edit results either with or without trailing
+spaces.   See the library documentation on :mod:`curses.textpad` for the
+For More Information
+This HOWTO didn't cover some advanced topics, such as screen-scraping or
+capturing mouse events from an xterm instance.  But the Python library page for
+the curses modules is now pretty complete.  You should browse it next.
+If you're in doubt about the detailed behavior of any of the ncurses entry
+points, consult the manual pages for your curses implementation, whether it's
+ncurses or a proprietary Unix vendor's.  The manual pages will document any
+quirks, and provide complete lists of all the functions, attributes, and
+:const:`ACS_\*` characters available to you.
+Because the curses API is so large, some functions aren't supported in the
+Python interface, not because they're difficult to implement, but because no one
+has needed them yet.  Feel free to add them and then submit a patch.  Also, we
+don't yet have support for the menus or panels libraries associated with
+ncurses; feel free to add that.
+If you write an interesting little program, feel free to contribute it as
+another demo.  We can always use more of them!
+The ncurses FAQ: http://dickey.his.com/ncurses/ncurses.faq.html

File Doc-26/howto/doanddont.rst

View file
+  Idioms and Anti-Idioms in Python  
+:Author: Moshe Zadka
+This document is placed in the public doman.
+.. topic:: Abstract
+   This document can be considered a companion to the tutorial. It shows how to use
+   Python, and even more importantly, how *not* to use Python.
+Language Constructs You Should Not Use
+While Python has relatively few gotchas compared to other languages, it still
+has some constructs which are only useful in corner cases, or are plain
+from module import \*
+Inside Function Definitions
+``from module import *`` is *invalid* inside function definitions. While many
+versions of Python do not check for the invalidity, it does not make it more
+valid, no more then having a smart lawyer makes a man innocent. Do not use it
+like that ever. Even in versions where it was accepted, it made the function
+execution slower, because the compiler could not be certain which names are
+local and which are global. In Python 2.1 this construct causes warnings, and
+sometimes even errors.
+At Module Level
+While it is valid to use ``from module import *`` at module level it is usually
+a bad idea. For one, this loses an important property Python otherwise has ---
+you can know where each toplevel name is defined by a simple "search" function
+in your favourite editor. You also open yourself to trouble in the future, if
+some module grows additional functions or classes.
+One of the most awful question asked on the newsgroup is why this code::
+   f = open("www")
+   f.read()
+does not work. Of course, it works just fine (assuming you have a file called
+"www".) But it does not work if somewhere in the module, the statement ``from os
+import *`` is present. The :mod:`os` module has a function called :func:`open`
+which returns an integer. While it is very useful, shadowing builtins is one of
+its least useful properties.
+Remember, you can never know for sure what names a module exports, so either
+take what you need --- ``from module import name1, name2``, or keep them in the
+module and access on a per-need basis ---  ``import module;print module.name``.
+When It Is Just Fine
+There are situations in which ``from module import *`` is just fine:
+* The interactive prompt. For example, ``from math import *`` makes Python an
+  amazing scientific calculator.
+* When extending a module in C with a module in Python.
+* When the module advertises itself as ``from import *`` safe.
+Unadorned :keyword:`exec`, :func:`execfile` and friends
+The word "unadorned" refers to the use without an explicit dictionary, in which
+case those constructs evaluate code in the *current* environment. This is
+dangerous for the same reasons ``from import *`` is dangerous --- it might step
+over variables you are counting on and mess up things for the rest of your code.
+Simply do not do that.
+Bad examples::
+   >>> for name in sys.argv[1:]:
+   >>>     exec "%s=1" % name
+   >>> def func(s, **kw):
+   >>>     for var, val in kw.items():
+   >>>         exec "s.%s=val" % var  # invalid!
+   >>> execfile("handler.py")
+   >>> handle()
+Good examples::
+   >>> d = {}
+   >>> for name in sys.argv[1:]:
+   >>>     d[name] = 1
+   >>> def func(s, **kw):
+   >>>     for var, val in kw.items():
+   >>>         setattr(s, var, val)
+   >>> d={}
+   >>> execfile("handle.py", d, d)
+   >>> handle = d['handle']
+   >>> handle()
+from module import name1, name2
+This is a "don't" which is much weaker then the previous "don't"s but is still
+something you should not do if you don't have good reasons to do that. The
+reason it is usually bad idea is because you suddenly have an object which lives
+in two seperate namespaces. When the binding in one namespace changes, the
+binding in the other will not, so there will be a discrepancy between them. This
+happens when, for example, one module is reloaded, or changes the definition of
+a function at runtime.
+Bad example::
+   # foo.py
+   a = 1
+   # bar.py
+   from foo import a
+   if something():
+       a = 2 # danger: foo.a != a 
+Good example::
+   # foo.py
+   a = 1
+   # bar.py
+   import foo
+   if something():
+       foo.a = 2
+Python has the ``except:`` clause, which catches all exceptions. Since *every*
+error in Python raises an exception, this makes many programming errors look
+like runtime problems, and hinders the debugging process.
+The following code shows a great example::
+   try:
+       foo = opne("file") # misspelled "open"
+   except:
+       sys.exit("could not open file!")
+The second line triggers a :exc:`NameError` which is caught by the except
+clause. The program will exit, and you will have no idea that this has nothing
+to do with the readability of ``"file"``.
+The example above is better written ::
+   try:
+       foo = opne("file") # will be changed to "open" as soon as we run it
+   except IOError:
+       sys.exit("could not open file")
+There are some situations in which the ``except:`` clause is useful: for
+example, in a framework when running callbacks, it is good not to let any
+callback disturb the framework.
+Exceptions are a useful feature of Python. You should learn to raise them
+whenever something unexpected occurs, and catch them only where you can do
+something about them.
+The following is a very popular anti-idiom ::
+   def get_status(file):
+       if not os.path.exists(file):
+           print "file not found"
+           sys.exit(1)
+       return open(file).readline()
+Consider the case the file gets deleted between the time the call to
+:func:`os.path.exists` is made and the time :func:`open` is called. That means
+the last line will throw an :exc:`IOError`. The same would happen if *file*
+exists but has no read permission. Since testing this on a normal machine on
+existing and non-existing files make it seem bugless, that means in testing the
+results will seem fine, and the code will get shipped. Then an unhandled
+:exc:`IOError` escapes to the user, who has to watch the ugly traceback.
+Here is a better way to do it. ::
+   def get_status(file):
+       try:
+           return open(file).readline()
+       except (IOError, OSError):
+           print "file not found"
+           sys.exit(1)
+In this version, \*either\* the file gets opened and the line is read (so it
+works even on flaky NFS or SMB connections), or the message is printed and the
+application aborted.
+Still, :func:`get_status` makes too many assumptions --- that it will only be
+used in a short running script, and not, say, in a long running server. Sure,
+the caller could do something like ::
+   try:
+       status = get_status(log)
+   except SystemExit:
+       status = None
+So, try to make as few ``except`` clauses in your code --- those will usually be
+a catch-all in the :func:`main`, or inside calls which should always succeed.
+So, the best version is probably ::
+   def get_status(file):
+       return open(file).readline()
+The caller can deal with the exception if it wants (for example, if it  tries
+several files in a loop), or just let the exception filter upwards to *its*
+The last version is not very good either --- due to implementation details, the
+file would not be closed when an exception is raised until the handler finishes,
+and perhaps not at all in non-C implementations (e.g., Jython). ::
+   def get_status(file):
+       fp = open(file)
+       try:
+           return fp.readline()
+       finally:
+           fp.close()
+Using the Batteries
+Every so often, people seem to be writing stuff in the Python library again,
+usually poorly. While the occasional module has a poor interface, it is usually
+much better to use the rich standard library and data types that come with
+Python then inventing your own.
+A useful module very few people know about is :mod:`os.path`. It  always has the
+correct path arithmetic for your operating system, and will usually be much
+better then whatever you come up with yourself.
+   # ugh!
+   return dir+"/"+file
+   # better
+   return os.path.join(dir, file)
+More useful functions in :mod:`os.path`: :func:`basename`,  :func:`dirname` and
+There are also many useful builtin functions people seem not to be aware of for
+some reason: :func:`min` and :func:`max` can find the minimum/maximum of any
+sequence with comparable semantics, for example, yet many people write their own
+:func:`max`/:func:`min`. Another highly useful function is :func:`reduce`. A
+classical use of :func:`reduce` is something like ::
+   import sys, operator
+   nums = map(float, sys.argv[1:])
+   print reduce(operator.add, nums)/len(nums)
+This cute little script prints the average of all numbers given on the command
+line. The :func:`reduce` adds up all the numbers, and the rest is just some pre-
+and postprocessing.
+On the same note, note that :func:`float`, :func:`int` and :func:`long` all
+accept arguments of type string, and so are suited to parsing --- assuming you
+are ready to deal with the :exc:`ValueError` they raise.
+Using Backslash to Continue Statements
+Since Python treats a newline as a statement terminator, and since statements
+are often more then is comfortable to put in one line, many people do::
+   if foo.bar()['first'][0] == baz.quux(1, 2)[5:9] and \
+      calculate_number(10, 20) != forbulate(500, 360):
+         pass
+You should realize that this is dangerous: a stray space after the ``XXX`` would
+make this line wrong, and stray spaces are notoriously hard to see in editors.
+In this case, at least it would be a syntax error, but if the code was::
+   value = foo.bar()['first'][0]*baz.quux(1, 2)[5:9] \
+           + calculate_number(10, 20)*forbulate(500, 360)
+then it would just be subtly wrong.
+It is usually much better to use the implicit continuation inside parenthesis:
+This version is bulletproof::
+   value = (foo.bar()['first'][0]*baz.quux(1, 2)[5:9] 
+           + calculate_number(10, 20)*forbulate(500, 360))

File Doc-26/howto/functional.rst

View file
+  Functional Programming HOWTO
+:Author: \A. M. Kuchling
+:Release: 0.30
+(This is a first draft.  Please send comments/error reports/suggestions to
+amk@amk.ca.  This URL is probably not going to be the final location of the
+document, so be careful about linking to it -- you may want to add a
+In this document, we'll take a tour of Python's features suitable for
+implementing programs in a functional style.  After an introduction to the
+concepts of functional programming, we'll look at language features such as
+iterators and generators and relevant library modules such as :mod:`itertools`
+and :mod:`functools`.
+This section explains the basic concept of functional programming; if you're
+just interested in learning about Python language features, skip to the next
+Programming languages support decomposing problems in several different ways:
+* Most programming languages are **procedural**: programs are lists of
+  instructions that tell the computer what to do with the program's input.  C,
+  Pascal, and even Unix shells are procedural languages.
+* In **declarative** languages, you write a specification that describes the
+  problem to be solved, and the language implementation figures out how to
+  perform the computation efficiently.  SQL is the declarative language you're
+  most likely to be familiar with; a SQL query describes the data set you want
+  to retrieve, and the SQL engine decides whether to scan tables or use indexes,
+  which subclauses should be performed first, etc.
+* **Object-oriented** programs manipulate collections of objects.  Objects have
+  internal state and support methods that query or modify this internal state in
+  some way. Smalltalk and Java are object-oriented languages.  C++ and Python
+  are languages that support object-oriented programming, but don't force the
+  use of object-oriented features.
+* **Functional** programming decomposes a problem into a set of functions.
+  Ideally, functions only take inputs and produce outputs, and don't have any
+  internal state that affects the output produced for a given input.  Well-known
+  functional languages include the ML family (Standard ML, OCaml, and other
+  variants) and Haskell.
+The designers of some computer languages have chosen one approach to programming
+that's emphasized.  This often makes it difficult to write programs that use a
+different approach.  Other languages are multi-paradigm languages that support
+several different approaches.  Lisp, C++, and Python are multi-paradigm; you can
+write programs or libraries that are largely procedural, object-oriented, or
+functional in all of these languages.  In a large program, different sections
+might be written using different approaches; the GUI might be object-oriented
+while the processing logic is procedural or functional, for example.
+In a functional program, input flows through a set of functions. Each function
+operates on its input and produces some output.  Functional style frowns upon
+functions with side effects that modify internal state or make other changes
+that aren't visible in the function's return value.  Functions that have no side
+effects at all are called **purely functional**.  Avoiding side effects means
+not using data structures that get updated as a program runs; every function's
+output must only depend on its input.
+Some languages are very strict about purity and don't even have assignment
+statements such as ``a=3`` or ``c = a + b``, but it's difficult to avoid all
+side effects.  Printing to the screen or writing to a disk file are side
+effects, for example.  For example, in Python a ``print`` statement or a
+``time.sleep(1)`` both return no useful value; they're only called for their
+side effects of sending some text to the screen or pausing execution for a
+Python programs written in functional style usually won't go to the extreme of
+avoiding all I/O or all assignments; instead, they'll provide a
+functional-appearing interface but will use non-functional features internally.
+For example, the implementation of a function will still use assignments to
+local variables, but won't modify global variables or have other side effects.
+Functional programming can be considered the opposite of object-oriented
+programming.  Objects are little capsules containing some internal state along
+with a collection of method calls that let you modify this state, and programs
+consist of making the right set of state changes.  Functional programming wants
+to avoid state changes as much as possible and works with data flowing between
+functions.  In Python you might combine the two approaches by writing functions
+that take and return instances representing objects in your application (e-mail
+messages, transactions, etc.).
+Functional design may seem like an odd constraint to work under.  Why should you
+avoid objects and side effects?  There are theoretical and practical advantages
+to the functional style:
+* Formal provability.
+* Modularity.
+* Composability.
+* Ease of debugging and testing.
+Formal provability
+A theoretical benefit is that it's easier to construct a mathematical proof that
+a functional program is correct.
+For a long time researchers have been interested in finding ways to
+mathematically prove programs correct.  This is different from testing a program
+on numerous inputs and concluding that its output is usually correct, or reading
+a program's source code and concluding that the code looks right; the goal is
+instead a rigorous proof that a program produces the right result for all
+possible inputs.
+The technique used to prove programs correct is to write down **invariants**,
+properties of the input data and of the program's variables that are always
+true.  For each line of code, you then show that if invariants X and Y are true
+**before** the line is executed, the slightly different invariants X' and Y' are
+true **after** the line is executed.  This continues until you reach the end of
+the program, at which point the invariants should match the desired conditions
+on the program's output.
+Functional programming's avoidance of assignments arose because assignments are
+difficult to handle with this technique; assignments can break invariants that
+were true before the assignment without producing any new invariants that can be
+propagated onward.
+Unfortunately, proving programs correct is largely impractical and not relevant
+to Python software. Even trivial programs require proofs that are several pages
+long; the proof of correctness for a moderately complicated program would be
+enormous, and few or none of the programs you use daily (the Python interpreter,
+your XML parser, your web browser) could be proven correct.  Even if you wrote
+down or generated a proof, there would then be the question of verifying the
+proof; maybe there's an error in it, and you wrongly believe you've proved the
+program correct.
+A more practical benefit of functional programming is that it forces you to
+break apart your problem into small pieces.  Programs are more modular as a
+result.  It's easier to specify and write a small function that does one thing
+than a large function that performs a complicated transformation.  Small
+functions are also easier to read and to check for errors.
+Ease of debugging and testing 
+Testing and debugging a functional-style program is easier.
+Debugging is simplified because functions are generally small and clearly
+specified.  When a program doesn't work, each function is an interface point
+where you can check that the data are correct.  You can look at the intermediate
+inputs and outputs to quickly isolate the function that's responsible for a bug.
+Testing is easier because each function is a potential subject for a unit test.
+Functions don't depend on system state that needs to be replicated before
+running a test; instead you only have to synthesize the right input and then
+check that the output matches expectations.
+As you work on a functional-style program, you'll write a number of functions
+with varying inputs and outputs.  Some of these functions will be unavoidably
+specialized to a particular application, but others will be useful in a wide
+variety of programs.  For example, a function that takes a directory path and
+returns all the XML files in the directory, or a function that takes a filename
+and returns its contents, can be applied to many different situations.
+Over time you'll form a personal library of utilities.  Often you'll assemble
+new programs by arranging existing functions in a new configuration and writing
+a few functions specialized for the current task.
+I'll start by looking at a Python language feature that's an important
+foundation for writing functional-style programs: iterators.
+An iterator is an object representing a stream of data; this object returns the
+data one element at a time.  A Python iterator must support a method called
+``next()`` that takes no arguments and always returns the next element of the
+stream.  If there are no more elements in the stream, ``next()`` must raise the
+``StopIteration`` exception.  Iterators don't have to be finite, though; it's
+perfectly reasonable to write an iterator that produces an infinite stream of
+The built-in :func:`iter` function takes an arbitrary object and tries to return
+an iterator that will return the object's contents or elements, raising
+:exc:`TypeError` if the object doesn't support iteration.  Several of Python's
+built-in data types support iteration, the most common being lists and
+dictionaries.  An object is called an **iterable** object if you can get an
+iterator for it.
+You can experiment with the iteration interface manually::
+    >>> L = [1,2,3]
+    >>> it = iter(L)
+    >>> print it
+    <iterator object at 0x8116870>
+    >>> it.next()
+    1
+    >>> it.next()
+    2
+    >>> it.next()
+    3
+    >>> it.next()
+    Traceback (most recent call last):
+      File "<stdin>", line 1, in ?
+    StopIteration
+    >>>      
+Python expects iterable objects in several different contexts, the most
+important being the ``for`` statement.  In the statement ``for X in Y``, Y must
+be an iterator or some object for which ``iter()`` can create an iterator.
+These two statements are equivalent::
+        for i in iter(obj):
+            print i
+        for i in obj:
+            print i
+Iterators can be materialized as lists or tuples by using the :func:`list` or
+:func:`tuple` constructor functions::
+    >>> L = [1,2,3]
+    >>> iterator = iter(L)
+    >>> t = tuple(iterator)
+    >>> t
+    (1, 2, 3)
+Sequence unpacking also supports iterators: if you know an iterator will return
+N elements, you can unpack them into an N-tuple::
+    >>> L = [1,2,3]
+    >>> iterator = iter(L)
+    >>> a,b,c = iterator
+    >>> a,b,c
+    (1, 2, 3)
+Built-in functions such as :func:`max` and :func:`min` can take a single
+iterator argument and will return the largest or smallest element.  The ``"in"``
+and ``"not in"`` operators also support iterators: ``X in iterator`` is true if
+X is found in the stream returned by the iterator.  You'll run into obvious
+problems if the iterator is infinite; ``max()``, ``min()``, and ``"not in"``
+will never return, and if the element X never appears in the stream, the
+``"in"`` operator won't return either.
+Note that you can only go forward in an iterator; there's no way to get the
+previous element, reset the iterator, or make a copy of it.  Iterator objects
+can optionally provide these additional capabilities, but the iterator protocol
+only specifies the ``next()`` method.  Functions may therefore consume all of
+the iterator's output, and if you need to do something different with the same
+stream, you'll have to create a new iterator.
+Data Types That Support Iterators
+We've already seen how lists and tuples support iterators.  In fact, any Python
+sequence type, such as strings, will automatically support creation of an
+Calling :func:`iter` on a dictionary returns an iterator that will loop over the
+dictionary's keys::
+    >>> m = {'Jan': 1, 'Feb': 2, 'Mar': 3, 'Apr': 4, 'May': 5, 'Jun': 6,
+    ...      'Jul': 7, 'Aug': 8, 'Sep': 9, 'Oct': 10, 'Nov': 11, 'Dec': 12}
+    >>> for key in m:
+    ...     print key, m[key]
+    Mar 3
+    Feb 2
+    Aug 8
+    Sep 9
+    May 5
+    Jun 6
+    Jul 7
+    Jan 1
+    Apr 4
+    Nov 11
+    Dec 12
+    Oct 10
+Note that the order is essentially random, because it's based on the hash
+ordering of the objects in the dictionary.
+Applying ``iter()`` to a dictionary always loops over the keys, but dictionaries
+have methods that return other iterators.  If you want to iterate over keys,
+values, or key/value pairs, you can explicitly call the ``iterkeys()``,
+``itervalues()``, or ``iteritems()`` methods to get an appropriate iterator.
+The :func:`dict` constructor can accept an iterator that returns a finite stream
+of ``(key, value)`` tuples::
+    >>> L = [('Italy', 'Rome'), ('France', 'Paris'), ('US', 'Washington DC')]
+    >>> dict(iter(L))
+    {'Italy': 'Rome', 'US': 'Washington DC', 'France': 'Paris'}
+Files also support iteration by calling the ``readline()`` method until there
+are no more lines in the file.  This means you can read each line of a file like
+    for line in file:
+        # do something for each line
+        ...
+Sets can take their contents from an iterable and let you iterate over the set's
+    S = set((2, 3, 5, 7, 11, 13))
+    for i in S:
+        print i
+Generator expressions and list comprehensions
+Two common operations on an iterator's output are 1) performing some operation
+for every element, 2) selecting a subset of elements that meet some condition.
+For example, given a list of strings, you might want to strip off trailing
+whitespace from each line or extract all the strings containing a given
+List comprehensions and generator expressions (short form: "listcomps" and
+"genexps") are a concise notation for such operations, borrowed from the
+functional programming language Haskell (http://www.haskell.org).  You can strip
+all the whitespace from a stream of strings with the following code::
+        line_list = ['  line 1\n', 'line 2  \n', ...]
+        # Generator expression -- returns iterator
+        stripped_iter = (line.strip() for line in line_list)
+        # List comprehension -- returns list
+        stripped_list = [line.strip() for line in line_list]
+You can select only certain elements by adding an ``"if"`` condition::
+        stripped_list = [line.strip() for line in line_list
+                         if line != ""]
+With a list comprehension, you get back a Python list; ``stripped_list`` is a
+list containing the resulting lines, not an iterator.  Generator expressions
+return an iterator that computes the values as necessary, not needing to
+materialize all the values at once.  This means that list comprehensions aren't
+useful if you're working with iterators that return an infinite stream or a very
+large amount of data.  Generator expressions are preferable in these situations.
+Generator expressions are surrounded by parentheses ("()") and list
+comprehensions are surrounded by square brackets ("[]").  Generator expressions
+have the form::
+    ( expression for expr in sequence1 
+                 if condition1
+                 for expr2 in sequence2
+                 if condition2
+                 for expr3 in sequence3 ...
+                 if condition3
+                 for exprN in sequenceN
+                 if conditionN )
+Again, for a list comprehension only the outside brackets are different (square
+brackets instead of parentheses).
+The elements of the generated output will be the successive values of
+``expression``.  The ``if`` clauses are all optional; if present, ``expression``
+is only evaluated and added to the result when ``condition`` is true.
+Generator expressions always have to be written inside parentheses, but the
+parentheses signalling a function call also count.  If you want to create an
+iterator that will be immediately passed to a function you can write::
+        obj_total = sum(obj.count for obj in list_all_objects())
+The ``for...in`` clauses contain the sequences to be iterated over.  The
+sequences do not have to be the same length, because they are iterated over from
+left to right, **not** in parallel.  For each element in ``sequence1``,
+``sequence2`` is looped over from the beginning.  ``sequence3`` is then looped
+over for each resulting pair of elements from ``sequence1`` and ``sequence2``.
+To put it another way, a list comprehension or generator expression is
+equivalent to the following Python code::
+    for expr1 in sequence1:
+        if not (condition1):
+            continue   # Skip this element
+        for expr2 in sequence2:
+            if not (condition2):
+                continue    # Skip this element
+            ...
+            for exprN in sequenceN:
+                 if not (conditionN):
+                     continue   # Skip this element
+                 # Output the value of 
+                 # the expression.
+This means that when there are multiple ``for...in`` clauses but no ``if``
+clauses, the length of the resulting output will be equal to the product of the
+lengths of all the sequences.  If you have two lists of length 3, the output
+list is 9 elements long::
+    seq1 = 'abc'
+    seq2 = (1,2,3)
+    >>> [ (x,y) for x in seq1 for y in seq2]
+    [('a', 1), ('a', 2), ('a', 3), 
+     ('b', 1), ('b', 2), ('b', 3), 
+     ('c', 1), ('c', 2), ('c', 3)]
+To avoid introducing an ambiguity into Python's grammar, if ``expression`` is
+creating a tuple, it must be surrounded with parentheses.  The first list
+comprehension below is a syntax error, while the second one is correct::
+    # Syntax error
+    [ x,y for x in seq1 for y in seq2]
+    # Correct
+    [ (x,y) for x in seq1 for y in seq2]
+Generators are a special class of functions that simplify the task of writing
+iterators.  Regular functions compute a value and return it, but generators
+return an iterator that returns a stream of values.
+You're doubtless familiar with how regular function calls work in Python or C.
+When you call a function, it gets a private namespace where its local variables
+are created.  When the function reaches a ``return`` statement, the local
+variables are destroyed and the value is returned to the caller.  A later call
+to the same function creates a new private namespace and a fresh set of local
+variables. But, what if the local variables weren't thrown away on exiting a
+function?  What if you could later resume the function where it left off?  This
+is what generators provide; they can be thought of as resumable functions.
+Here's the simplest example of a generator function::
+    def generate_ints(N):
+        for i in range(N):
+            yield i
+Any function containing a ``yield`` keyword is a generator function; this is
+detected by Python's bytecode compiler which compiles the function specially as
+a result.
+When you call a generator function, it doesn't return a single value; instead it
+returns a generator object that supports the iterator protocol.  On executing
+the ``yield`` expression, the generator outputs the value of ``i``, similar to a
+``return`` statement.  The big difference between ``yield`` and a ``return``
+statement is that on reaching a ``yield`` the generator's state of execution is
+suspended and local variables are preserved.  On the next call to the
+generator's ``.next()`` method, the function will resume executing.
+Here's a sample usage of the ``generate_ints()`` generator::
+    >>> gen = generate_ints(3)
+    >>> gen
+    <generator object at 0x8117f90>
+    >>> gen.next()
+    0
+    >>> gen.next()
+    1
+    >>> gen.next()
+    2
+    >>> gen.next()
+    Traceback (most recent call last):
+      File "stdin", line 1, in ?
+      File "stdin", line 2, in generate_ints
+    StopIteration
+You could equally write ``for i in generate_ints(5)``, or ``a,b,c =
+Inside a generator function, the ``return`` statement can only be used without a
+value, and signals the end of the procession of values; after executing a
+``return`` the generator cannot return any further values.  ``return`` with a
+value, such as ``return 5``, is a syntax error inside a generator function.  The
+end of the generator's results can also be indicated by raising
+``StopIteration`` manually, or by just letting the flow of execution fall off
+the bottom of the function.
+You could achieve the effect of generators manually by writing your own class
+and storing all the local variables of the generator as instance variables.  For
+example, returning a list of integers could be done by setting ``self.count`` to
+0, and having the ``next()`` method increment ``self.count`` and return it.
+However, for a moderately complicated generator, writing a corresponding class
+can be much messier.
+The test suite included with Python's library, ``test_generators.py``, contains
+a number of more interesting examples.  Here's one generator that implements an
+in-order traversal of a tree using generators recursively.
+    # A recursive generator that generates Tree leaves in in-order.
+    def inorder(t):
+        if t:
+            for x in inorder(t.left):
+                yield x
+            yield t.label
+            for x in inorder(t.right):
+                yield x
+Two other examples in ``test_generators.py`` produce solutions for the N-Queens
+problem (placing N queens on an NxN chess board so that no queen threatens
+another) and the Knight's Tour (finding a route that takes a knight to every
+square of an NxN chessboard without visiting any square twice).
+Passing values into a generator
+In Python 2.4 and earlier, generators only produced output.  Once a generator's
+code was invoked to create an iterator, there was no way to pass any new
+information into the function when its execution is resumed.  You could hack
+together this ability by making the generator look at a global variable or by
+passing in some mutable object that callers then modify, but these approaches
+are messy.
+In Python 2.5 there's a simple way to pass values into a generator.
+:keyword:`yield` became an expression, returning a value that can be assigned to
+a variable or otherwise operated on::
+    val = (yield i)
+I recommend that you **always** put parentheses around a ``yield`` expression
+when you're doing something with the returned value, as in the above example.
+The parentheses aren't always necessary, but it's easier to always add them
+instead of having to remember when they're needed.
+(PEP 342 explains the exact rules, which are that a ``yield``-expression must
+always be parenthesized except when it occurs at the top-level expression on the
+right-hand side of an assignment.  This means you can write ``val = yield i``
+but have to use parentheses when there's an operation, as in ``val = (yield i)
++ 12``.)
+Values are sent into a generator by calling its ``send(value)`` method.  This
+method resumes the generator's code and the ``yield`` expression returns the
+specified value.  If the regular ``next()`` method is called, the ``yield``
+returns ``None``.
+Here's a simple counter that increments by 1 and allows changing the value of
+the internal counter.
+    def counter (maximum):
+        i = 0
+        while i < maximum:
+            val = (yield i)
+            # If value provided, change counter
+            if val is not None:
+                i = val
+            else:
+                i += 1
+And here's an example of changing the counter:
+    >>> it = counter(10)
+    >>> print it.next()
+    0
+    >>> print it.next()
+    1
+    >>> print it.send(8)
+    8
+    >>> print it.next()
+    9
+    >>> print it.next()
+    Traceback (most recent call last):
+      File ``t.py'', line 15, in ?
+        print it.next()
+    StopIteration
+Because ``yield`` will often be returning ``None``, you should always check for
+this case.  Don't just use its value in expressions unless you're sure that the
+``send()`` method will be the only method used resume your generator function.
+In addition to ``send()``, there are two other new methods on generators:
+* ``throw(type, value=None, traceback=None)`` is used to raise an exception
+  inside the generator; the exception is raised by the ``yield`` expression
+  where the generator's execution is paused.
+* ``close()`` raises a :exc:`GeneratorExit` exception inside the generator to
+  terminate the iteration.  On receiving this exception, the generator's code
+  must either raise :exc:`GeneratorExit` or :exc:`StopIteration`; catching the
+  exception and doing anything else is illegal and will trigger a
+  :exc:`RuntimeError`.  ``close()`` will also be called by Python's garbage
+  collector when the generator is garbage-collected.
+  If you need to run cleanup code when a :exc:`GeneratorExit` occurs, I suggest
+  using a ``try: ... finally:`` suite instead of catching :exc:`GeneratorExit`.
+The cumulative effect of these changes is to turn generators from one-way
+producers of information into both producers and consumers.
+Generators also become **coroutines**, a more generalized form of subroutines.
+Subroutines are entered at one point and exited at another point (the top of the
+function, and a ``return`` statement), but coroutines can be entered, exited,
+and resumed at many different points (the ``yield`` statements).
+Built-in functions
+Let's look in more detail at built-in functions often used with iterators.
+Two Python's built-in functions, :func:`map` and :func:`filter`, are somewhat
+obsolete; they duplicate the features of list comprehensions but return actual
+lists instead of iterators.
+``map(f, iterA, iterB, ...)`` returns a list containing ``f(iterA[0], iterB[0]),
+f(iterA[1], iterB[1]), f(iterA[2], iterB[2]), ...``.
+    def upper(s):
+        return s.upper()
+    map(upper, ['sentence', 'fragment']) =>
+    [upper(s) for s in ['sentence', 'fragment']] =>
+As shown above, you can achieve the same effect with a list comprehension.  The
+:func:`itertools.imap` function does the same thing but can handle infinite
+iterators; it'll be discussed later, in the section on the :mod:`itertools` module.
+``filter(predicate, iter)`` returns a list that contains all the sequence
+elements that meet a certain condition, and is similarly duplicated by list
+comprehensions.  A **predicate** is a function that returns the truth value of
+some condition; for use with :func:`filter`, the predicate must take a single
+    def is_even(x):
+        return (x % 2) == 0
+    filter(is_even, range(10)) =>
+      [0, 2, 4, 6, 8]
+This can also be written as a list comprehension::
+    >>> [x for x in range(10) if is_even(x)]
+    [0, 2, 4, 6, 8]
+:func:`filter` also has a counterpart in the :mod:`itertools` module,
+:func:`itertools.ifilter`, that returns an iterator and can therefore handle
+infinite sequences just as :func:`itertools.imap` can.
+``reduce(func, iter, [initial_value])`` doesn't have a counterpart in the
+:mod:`itertools` module because it cumulatively performs an operation on all the
+iterable's elements and therefore can't be applied to infinite iterables.
+``func`` must be a function that takes two elements and returns a single value.
+:func:`reduce` takes the first two elements A and B returned by the iterator and
+calculates ``func(A, B)``.  It then requests the third element, C, calculates
+``func(func(A, B), C)``, combines this result with the fourth element returned,
+and continues until the iterable is exhausted.  If the iterable returns no
+values at all, a :exc:`TypeError` exception is raised.  If the initial value is
+supplied, it's used as a starting point and ``func(initial_value, A)`` is the
+first calculation.
+    import operator
+    reduce(operator.concat, ['A', 'BB', 'C']) =>
+      'ABBC'
+    reduce(operator.concat, []) =>
+      TypeError: reduce() of empty sequence with no initial value
+    reduce(operator.mul, [1,2,3], 1) =>
+      6
+    reduce(operator.mul, [], 1) =>
+      1
+If you use :func:`operator.add` with :func:`reduce`, you'll add up all the
+elements of the iterable.  This case is so common that there's a special
+built-in called :func:`sum` to compute it::
+    reduce(operator.add, [1,2,3,4], 0) =>
+      10
+    sum([1,2,3,4]) =>
+      10
+    sum([]) =>
+      0
+For many uses of :func:`reduce`, though, it can be clearer to just write the
+obvious :keyword:`for` loop::
+    # Instead of:
+    product = reduce(operator.mul, [1,2,3], 1)
+    # You can write:
+    product = 1
+    for i in [1,2,3]:
+        product *= i
+``enumerate(iter)`` counts off the elements in the iterable, returning 2-tuples
+containing the count and each element.
+    enumerate(['subject', 'verb', 'object']) =>
+      (0, 'subject'), (1, 'verb'), (2, 'object')
+:func:`enumerate` is often used when looping through a list and recording the
+indexes at which certain conditions are met::
+    f = open('data.txt', 'r')
+    for i, line in enumerate(f):
+        if line.strip() == '':
+            print 'Blank line at line #%i' % i
+``sorted(iterable, [cmp=None], [key=None], [reverse=False)`` collects all the
+elements of the iterable into a list, sorts the list, and returns the sorted
+result.  The ``cmp``, ``key``, and ``reverse`` arguments are passed through to
+the constructed list's ``.sort()`` method.
+    import random
+    # Generate 8 random numbers between [0, 10000)
+    rand_list = random.sample(range(10000), 8)
+    rand_list =>
+      [769, 7953, 9828, 6431, 8442, 9878, 6213, 2207]
+    sorted(rand_list) =>
+      [769, 2207, 6213, 6431, 7953, 8442, 9828, 9878]
+    sorted(rand_list, reverse=True) =>
+      [9878, 9828, 8442, 7953, 6431, 6213, 2207, 769]
+(For a more detailed discussion of sorting, see the Sorting mini-HOWTO in the
+Python wiki at http://wiki.python.org/moin/HowTo/Sorting.)
+The ``any(iter)`` and ``all(iter)`` built-ins look at the truth values of an
+iterable's contents.  :func:`any` returns True if any element in the iterable is
+a true value, and :func:`all` returns True if all of the elements are true
+    any([0,1,0]) =>
+      True
+    any([0,0,0]) =>
+      False
+    any([1,1,1]) =>
+      True
+    all([0,1,0]) =>
+      False
+    all([0,0,0]) => 
+      False
+    all([1,1,1]) =>
+      True
+Small functions and the lambda expression
+When writing functional-style programs, you'll often need little functions that
+act as predicates or that combine elements in some way.
+If there's a Python built-in or a module function that's suitable, you don't
+need to define a new function at all::
+        stripped_lines = [line.strip() for line in lines]
+        existing_files = filter(os.path.exists, file_list)
+If the function you need doesn't exist, you need to write it.  One way to write
+small functions is to use the ``lambda`` statement.  ``lambda`` takes a number
+of parameters and an expression combining these parameters, and creates a small
+function that returns the value of the expression::
+        lowercase = lambda x: x.lower()
+        print_assign = lambda name, value: name + '=' + str(value)
+        adder = lambda x, y: x+y
+An alternative is to just use the ``def`` statement and define a function in the
+usual way::
+        def lowercase(x):
+            return x.lower()
+        def print_assign(name, value):
+            return name + '=' + str(value)
+        def adder(x,y):
+            return x + y
+Which alternative is preferable?  That's a style question; my usual course is to
+avoid using ``lambda``.
+One reason for my preference is that ``lambda`` is quite limited in the
+functions it can define.  The result has to be computable as a single
+expression, which means you can't have multiway ``if... elif... else``
+comparisons or ``try... except`` statements.  If you try to do too much in a
+``lambda`` statement, you'll end up with an overly complicated expression that's
+hard to read.  Quick, what's the following code doing?
+    total = reduce(lambda a, b: (0, a[1] + b[1]), items)[1]
+You can figure it out, but it takes time to disentangle the expression to figure
+out what's going on.  Using a short nested ``def`` statements makes things a
+little bit better::
+    def combine (a, b):
+        return 0, a[1] + b[1]
+    total = reduce(combine, items)[1]
+But it would be best of all if I had simply used a ``for`` loop::
+     total = 0
+     for a, b in items:
+         total += b
+Or the :func:`sum` built-in and a generator expression::
+     total = sum(b for a,b in items)
+Many uses of :func:`reduce` are clearer when written as ``for`` loops.
+Fredrik Lundh once suggested the following set of rules for refactoring uses of
+1) Write a lambda function.
+2) Write a comment explaining what the heck that lambda does.
+3) Study the comment for a while, and think of a name that captures the essence
+   of the comment.
+4) Convert the lambda to a def statement, using that name.
+5) Remove the comment.
+I really like these rules, but you're free to disagree that this lambda-free
+style is better.
+The itertools module
+The :mod:`itertools` module contains a number of commonly-used iterators as well
+as functions for combining several iterators.  This section will introduce the
+module's contents by showing small examples.
+The module's functions fall into a few broad classes:
+* Functions that create a new iterator based on an existing iterator.
+* Functions for treating an iterator's elements as function arguments.
+* Functions for selecting portions of an iterator's output.
+* A function for grouping an iterator's output.
+Creating new iterators
+``itertools.count(n)`` returns an infinite stream of integers, increasing by 1
+each time.  You can optionally supply the starting number, which defaults to 0::
+        itertools.count() =>
+          0, 1, 2, 3, 4, 5, 6, 7, 8, 9, ...
+        itertools.count(10) =>
+          10, 11, 12, 13, 14, 15, 16, 17, 18, 19, ...
+``itertools.cycle(iter)`` saves a copy of the contents of a provided iterable
+and returns a new iterator that returns its elements from first to last.  The
+new iterator will repeat these elements infinitely.
+        itertools.cycle([1,2,3,4,5]) =>
+          1, 2, 3, 4, 5, 1, 2, 3, 4, 5, ...
+``itertools.repeat(elem, [n])`` returns the provided element ``n`` times, or
+returns the element endlessly if ``n`` is not provided.
+    itertools.repeat('abc') =>
+      abc, abc, abc, abc, abc, abc, abc, abc, abc, abc, ...
+    itertools.repeat('abc', 5) =>
+      abc, abc, abc, abc, abc
+``itertools.chain(iterA, iterB, ...)`` takes an arbitrary number of iterables as
+input, and returns all the elements of the first iterator, then all the elements
+of the second, and so on, until all of the iterables have been exhausted.
+    itertools.chain(['a', 'b', 'c'], (1, 2, 3)) =>
+      a, b, c, 1, 2, 3
+``itertools.izip(iterA, iterB, ...)`` takes one element from each iterable and
+returns them in a tuple::
+    itertools.izip(['a', 'b', 'c'], (1, 2, 3)) =>
+      ('a', 1), ('b', 2), ('c', 3)
+It's similiar to the built-in :func:`zip` function, but doesn't construct an
+in-memory list and exhaust all the input iterators before returning; instead
+tuples are constructed and returned only if they're requested.  (The technical
+term for this behaviour is `lazy evaluation
+This iterator is intended to be used with iterables that are all of the same