Anonymous avatar Anonymous committed 28b0486

Updating with final versions of all chapters and appendicies

Comments (0)

Files changed (54)

Binary file modified.

+Chapter 19: Concurrency
++++++++++++++++++++++++
+
+Supporting concurrency is increasingly important.
+In the past, mainstream concurrent programming generally meant ensuring that the code interacting with relatively slow network, disk, database, and other I/O resources did not unduly slow things down.
+Exploiting parallelism was typically only seen in such domains as scientific computing with the apps running on supercomputers.
+
+But there are new factors at work now.
+The semiconductor industry continues to work feverishly to uphold Moore’s Law of exponential increase in chip density.
+Chip designers used to apply this bounty to speeding up an individual CPU.
+For a variety of reasons this old approach no longer works as well, so now chip designers are cramming chips with more CPUs and hardware threads.
+Speeding up execution means harnessing the parallelism of the hardware, and it is now our job as software developers to do that work.
+
+The Java platform can help out here.
+The Java platform is arguably the most robust environment for running concurrent code today, and this functionality can be readily be used from Jython.
+The problem remains that writing concurrent code is still not easy.
+This is especially true with respect to a concurrency model based on threads, which is what today’s hardware natively exposes.
+
+This means we have to be concerned with thread safety, which arises as an issue because of the existence of mutable objects that are shared between threads.
+(Mutable state might be avoidable in functional programming, but it would be hard to avoid in any but the most trivial Python code.) If you attempt to solve concurrency issues through synchronization, you run into other problems: besides the potential performance hit, there are opportunities for deadlock and livelock.
+
+.. note::
+    
+    Implementations of the JVM (like HotSpot) can often avoid the overhead of synchronization.
+    We will discuss what’s necessary for this scenario to happen later in this chapter.
+
+Given all of these issues, it has been argued that threading is just too difficult to get right.
+We’re not convinced by that argument.
+Useful concurrent systems have been written on the JVM, and that includes apps written in Jython.
+Key success factors in writing such code include:
+
+- Keep the concurrency simple.
+
+
+- Use tasks, which can be mapped to a thread pool.
+
+
+- Use immutable objects where possible.
+
+
+- Avoid unnecessary sharing of mutable objects.
+
+
+- Minimize sharing of mutable objects.
+  Queues and related objects -- like synchronization barriers -- provide a structured mechanism to hand over objects between threads.
+  This can enable a design where an object is visible to only one thread when its state changes.
+
+
+- Code defensively.
+  Make it possible to cancel or interrupt tasks.
+  Use timeouts.
+
+
+Java or Python APIs?
+====================
+
+One issue that you will have to consider in writing concurrent code is how much to make your implementation dependent on the Java platform.
+Here are our recommendations:
+
+- If you are porting an existing Python code base that uses concurrency, you can just use the standard Python threading module.
+  Such code can still interoperate with Java, because Jython threads are always mapped to Java threads.
+  (If you are coming from Java, you will recognize this API, since it is substantially based on Java’s.)
+
+
+- Jython implements dict and set by using Java’s ConcurrentHashMap.
+  This means you can just use these standard Python types, and still get high performance concurrency.
+  (They are also atomic like in CPython, as we will describe.)
+
+
+- You can also any use of the collections from java.util.concurrent.
+  So if it fits your app’s needs, you may want to consider using such collections as CopyOnWriteArrayList and ConcurrentSkipListMap (new in Java 6).
+  The Google Collections Library is another good choice that works well with Jython.
+
+
+- Use higher-level primitives from Java instead of creating your own.
+  This is particular true of the executor services for running and managing tasks against thread pools.
+  So for example, avoid using threading.Timer, because you can use timed execution services in its place.
+  But still use threading.Condition and threading.Lock.
+  In particular, these constructs have been optimized to work in the context of a with-statement, as we will discuss.
+
+In practice, using Java’s support for higher level primitives should not impact the portability of your code so much.
+Using tasks in particular tends to keep all of this well isolated, and such thread safety considerations as thread confinement and safe publication remain the same.
+
+Lastly, remember you can always mix and match.
+
+Working With Threads
+====================
+
+Creating threads is easy, perhaps too easy.
+This example downloads a web page concurrently.
+
+Listing 19-1. test_thread_creation.py
+::
+    
+    from threading import Thread
+    import urllib2
+    
+    downloaded_page = None # global
+    
+    def download(url):
+        """Download ``url`` as a single string"""
+        global downloaded_page
+    
+        downloaded_page = urllib2.urlopen(url).read()
+        print "Downloaded", downloaded_page[:200]
+    
+    
+    def main_work():
+        # do some busy work in parallel
+        print "Started main task"
+        x = 0
+        for i in xrange(100000000):
+            x += 1
+        print "Completed main task"
+    
+    if __name__ == '__main__':
+        # perform the download in the background
+        Thread(target=lambda: download("http://www.jython.org")).start()
+        main_work()
+
+Be careful not to inadvertently invoke the function; target takes a reference to the function object (typically a name if a normal function).
+Calling the function instead creates an amusing bug where your target function runs now, so everything looks fine at first.
+But no concurrency is happening, because the function call is actually being run by the invoking thread, not this new thread.
+
+The target function can be a regular function, or an object that is callable (implements __call__).
+This latter case can make it harder to see that the target is a function object!
+
+To wait for a thread to complete, call join on it.
+This enables working with the concurrent result.
+The only problem is getting the result.
+As we will see, publishing results into variables is safe in Jython, but it’s not the nicest way.
+
+.. note::
+    
+    Daemon threads present an alluring alternative to managing the lifecycle of threads.
+    A thread is set to be a daemon thread before it is started:
+    
+    # create a thread t
+    ::
+        
+        t.setDaemon(True)t.start()
+    
+    Daemon status is inherited by any child threads.
+    Upon JVM shutdown, any daemon threads are simply terminated, without an opportunity—or need—to perform cleanup or orderly shutdown.
+    
+    This lack of cleanup means it’s important that daemon threads never hold any external resources, such as database connections or file handles.
+    Any such resource will not be properly closed upon a JVM exit.
+    For similar reasons, a daemon thread should never make an import attempt, as this can interfere with Jython’s orderly shutdown.
+    
+    In production, the only use case for daemon threads is when they are strictly used to work with in-memory objects, typically for some sort of housekeeping.
+    For example, you might use them to maintain a cache or compute an index.
+    
+    Having said that, daemon threads are certainly convenient when playing around with some ideas.
+    Maybe your lifecycle management of a program is to use "Control-C" to terminate.
+    Unlike regular threads, running daemon threads won’t get in the way and prevent JVM shutdown.
+    Likewise, a later example demonstrating deadlock uses daemon threads to enable shutdown without waiting on these deadlocked threads.
+    
+    With that in mind, it’s generally best not use daemon threads.
+    At the very least, serious thought should be given to their usage.
+
+Thread Locals
+=============
+
+The threading.local class enables each thread to have its own instances of some objects in an otherwise shared environment.
+Its usage is deceptively simple:simply create an instance of threading.local, or a subclass, and assign it to a variable or other name.
+This variable could be global, or part of some other namespace.
+So far, this is just like working with any other object in Python.
+
+Threads can then share the variable, but with a twist: each thread will see a different, thread-specific version of the object.
+This object can have arbitrary attributes added to it, each of which will not be visible to other threads.
+
+Other options include subclassing threading.local.
+As usual, this allows one to define defaults and specify a more nuanced properties model.
+But one unique, and potentially useful, aspect is that any attributes specified in __slots__ will be *shared* across threads.
+
+However, there’s a big problem when working with thread locals.
+Usually they don’t make sense because threads are not the right scope, but an object or a function is, especially through a closure.
+If you are using thread locals, you are implicitly adopting a model where threads are partitioning the work.
+But then you are binding the given piece of work to a thread.
+This makes using a thread pool problematic, because you have to clean up after the thread.
+
+.. note::
+    
+    In fact, we see this very problem in the Jython runtime.
+    A certain amount of context needs to be made available to execute Python code.
+    In the past, we would look this “ThreadState” up from the thread.
+    Historically, this may have been in fact faster, but it now slows things down, and unnecessarily limits what a given thread can do.
+    A future refactoring of Jython will likely remove the use of “ThreadState” completely, simultaneously speeding and cleaning things up.
+    Having said that, thread locals might be useful in certain cases.
+    One common scenario is where your code is being called by a component that you didn’t write.
+    Or you may need to access a thread-local singleton.
+    And of course, if you are using code whose architecture mandates thread locals, it’s just something you will have to work with.
+
+But often this is unnecessary.
+Your code may be different, but Python gives you good tools to avoid action at a distance.
+You can use closures, decorators, even sometimes selectively monkey patching modules.
+Take advantage of the fact that Python is a dynamic language, with strong support for metaprogramming, and remember that the Jython implementation makes these techniques accessible when working with even recalcitrant Java code.
+
+In the end, thread locals are an interesting aside.
+They do not work well in a task-oriented model, because you don’t want to associate context with a worker thread that will be assigned to arbitrary tasks.
+Without a lot of care, this can make for a confused mess.
+
+No Global Interpreter Lock
+==========================
+
+Jython lacks the global interpreter lock (GIL), which is an implementation detail of CPython.
+For CPython, the GIL means that only one thread *at a time* can run Python code.
+This restriction also applies to much of the supporting runtime as well as extension modules that do not release the GIL.
+(Unfortunately development efforts to remove the GIL in CPython have so far only had the effect of slowing down Python execution significantly.)
+
+The impact of the GIL on CPython programming is that threads are not as useful as they are in Jython.
+Concurrency will only be seen in interacting with I/O as well as scenarios where computation is performed by an extension module on data structures managed outside of CPython’s runtime.
+Instead, developers typically will use a process-oriented model to evade the restrictiveness of the GIL.
+
+Again, Jython does not have the straightjacket of the GIL.
+This is because all Python threads are mapped to Java threads and use standard Java garbage collection support (the main reason for the GIL in CPython is because of the reference counting GC system).
+The important ramification here is that you can use threads for compute-intensive tasks that are written in Python.
+
+Module Import Lock
+==================
+
+Python does, however, define a *module import lock*, which is implemented by Jython.
+This lock is acquired whenever an import of any name is made.
+This is true whether the import goes through the import statement, the equivalent __import__ builtin, or related code.
+It’s important to note that even if the corresponding module has already been imported, the module import lock will still be acquired, if only briefly.
+
+So don’t write code like this in a hot loop, especially in threaded code:
+
+Listing 19-2.
+::
+    
+    def slow_things_way_down():
+        from foo import bar, baz
+        ...
+
+It may still make sense to defer your imports.
+Such deferral can decrease the start time of your app.
+Just keep in mind that thread(s) performing such imports will be forced to run single threaded because of this lock.
+So it might make sense for your code to perform deferred imports in a background thread.
+
+Listing 19-3. background_import.py
+::
+    
+    from threading import Thread
+    import time
+    
+    def make_imports():
+        import unicodedata
+    
+    background_import = Thread(target=make_imports)
+    background_import.start()
+    
+    print "Do something else while we wait for the import"
+    for i in xrange(10):
+        print i
+        time.sleep(0.1)
+    print "Now join..."
+    background_import.join()
+    
+    print "And actually use unicodedata"
+    import unicodedata
+
+So as you can see, you need to do at least two imports of a given module; one in the background thread; the other in the actual place(s) where the module’s namespace is being used.
+
+Here’s why we need the module import lock: upon the first import, the import procedure runs the (implicit) top-level function of the module.
+Even though many modules are often declarative in nature, in Python all definitions are done at runtime.
+Such definitions potentially include further imports (recursive imports), and the top-level function can certainly perform much more complex tasks.
+The module import lock simplifies this setup so that it’s safely published.
+We will discuss this concept further later in this chapter.
+
+Note that in the current implementation, the module import lock is global for the entire Jython runtime.
+This may change in the future.
+
+Working With Tasks
+==================
+
+It’s usually best to avoid managing the lifecycle of threads directly.
+Instead, the task model often provides a better abstraction.
+
+*Tasks* describe the asynchronous computation to be performed.
+Although there are other options, the object you submit to be executed should implement Java’s Callable interface (a call method without arguments), as this best maps into working with a Python method or function.
+Tasks move through the states of being created, submitted (to an executor), started, and completed.
+Tasks can also be canceled or interrupted.
+
+*Executors* run tasks using a set of threads.
+This might be one thread, a thread pool, or as many threads as necessary to run all currently submitted tasks concurrently.
+The specific choice comprises the executor policy, but generally you want to use a thread pool so as to control the degree of concurrency.
+
+*Futures* allow code to access the result of a computation -- or an exception, if thrown -- in a task only at the point when it’s needed.
+Up until that point, the using code can run concurrently with that task.
+If it’s not ready, a wait-on dependency is introduced.
+
+We are going to look at how we can use this functionality by using the example of downloading web pages.
+We will wrap this up so it’s easy to work with, tracking the state of the download, as well as any timing information.
+
+Listing 19-4. downloader.py
+::
+    
+    import threading
+    import time
+    import urllib2
+    from java.util.concurrent import Callable
+    
+    class Downloader(Callable):
+        def __init__(self, url):
+            self.url = url
+            self.started = None
+            self.completed = None
+            self.result = None
+            self.thread_used = None
+            self.exception = None
+    
+        def __str__(self):
+            if self.exception:
+                 return "[%s] %s download error %s in %.2fs" % \
+                    (self.thread_used, self.url, self.exception, 
+                     self.completed - self.started, ) #, self.result)
+            elif self.completed:
+                return "[%s] %s downloaded %dK in %.2fs" % \
+                    (self.thread_used, self.url, len(self.result)/1024, 
+                     self.completed - self.started, ) #, self.result)
+            elif self.started:
+                return "[%s] %s started at %s" % \
+                    (self.thread_used, self.url, self.started)
+            else:
+                return "[%s] %s not yet scheduled" % \
+                    (self.thread_used, self.url)
+    
+        # needed to implement the Callable interface;
+        # any exceptions will be wrapped as either ExecutionException
+        # or InterruptedException
+        def call(self):
+            self.thread_used = threading.currentThread().getName()
+            self.started = time.time()
+            try:
+                self.result = urllib2.urlopen(self.url).read()
+            except Exception, ex:
+                self.exception = ex
+            self.completed = time.time()
+            return self
+
+In Jython any other task could be done in this fashion, whether it is a database query or a computationally intensive task written in Python.
+It just needs to support the Callable interface.
+
+Next, we need to create the futures.
+Upon completion of a future, either the result is returned, or an exception is thrown into the caller.
+This exception will be one of:
+
+- InterruptedException
+
+
+- ExecutionException.
+  Your code can retrieve the underlying exception with the cause attribute.
+
+(This pushing of the exception into the asynchronous caller is thus similar to how a coroutine works when send is called on it.)
+
+Now we have what we need to multiplex the downloads of several web pages over a thread pool.
+
+Listing 19-5. test_futures.py
+::
+    
+    from downloader import Downloader
+    from shutdown import shutdown_and_await_termination
+    from java.util.concurrent import Executors, TimeUnit
+    
+    MAX_CONCURRENT = 3
+    SITES = [
+        "http://www.cnn.com/",
+        "http://www.nytimes.com/",
+        "http://www.washingtonpost.com/",
+        "http://www.dailycamera.com/",
+        "http://www.timescall.com/",
+        ]
+    
+    pool = Executors.newFixedThreadPool(MAX_CONCURRENT)
+    downloaders = [Downloader(url) for url in SITES]
+    futures = pool.invokeAll(downloaders)
+    
+    for future in futures:
+        print future.get(5, TimeUnit.SECONDS)
+    
+    shutdown_and_await_termination(pool, 5)
+
+Up until the get method on the returned future, the caller runs concurrently with this task.
+The get call then introduces a wait-on dependency on the task’s completion.
+(So this is like calling join on the supporting thread.)
+
+Shutting down a thread pool should be as simple as calling the shutdown method on the pool.
+However, you may need to take into account that this shutdown can happen during extraordinary times in your code.
+Here’s the Jython version of a robust shutdown function, shutdown_and_await_termination, as provided in the standard Java docs.
+
+Listing 19-6. shutdown.py
+::
+    
+    from java.util.concurrent import TimeUnit
+    
+    def shutdown_and_await_termination(pool, timeout):
+        pool.shutdown()
+        try:
+            if not pool.awaitTermination(timeout, TimeUnit.SECONDS):
+                pool.shutdownNow()
+                if (not pool.awaitTermination(timeout, TimeUnit.SECONDS)):
+                    print >> sys.stderr, "Pool did not terminate"
+        except InterruptedException, ex:
+            # (Re-)Cancel if current thread also interrupted
+            pool.shutdownNow()
+            # Preserve interrupt status
+            Thread.currentThread().interrupt()
+
+The CompletionService interface provides a nice abstraction to working with futures.
+The scenario is that instead of waiting for all the futures to complete, as our code did with invokeAll, or otherwise polling them, the completion service will push futures as they are completed onto a synchronized queue.
+This queue can then be consumed, by consumers running in one or more threads.
+
+Listing 19-7. test_completion.py
+::
+    
+    from downloader import Downloader
+    from shutdown import shutdown_and_await_termination
+    from java.util.concurrent import Executors, ExecutorCompletionService
+    import os
+    import hashlib
+    
+    MAX_CONCURRENT = 3
+    SITES = [
+        "http://www.cnn.com/",
+        "http://www.nytimes.com/",
+        "http://www.washingtonpost.com/",
+        "http://www.dailycamera.com/",
+        "http://www.timescall.com/",
+        # generate a random web site name that is very, very unlikely to exist
+        "http://" + hashlib.md5(
+            "unlikely-web-site-" + os.urandom(4)).hexdigest() + ".com",
+        ]
+    
+    pool = Executors.newFixedThreadPool(MAX_CONCURRENT)
+    ecs = ExecutorCompletionService(pool)
+    
+    # this function could spider the links from these roots;
+    # for now just schedule these roots directly
+    def scheduler(roots):
+        for site in roots:
+            yield site
+    
+    # submit tasks indefinitely
+    for site in scheduler(SITES):
+        ecs.submit(Downloader(site))
+    
+    # work with results as soon as they become available
+    submitted = len(SITES)
+    while submitted > 0:
+        result = ecs.take().get()
+        # here we just do something unimaginative with the result;
+        # consider parsing it with tools like beautiful soup
+        print result
+        submitted -= 1
+    
+    print "shutting pool down..."
+    shutdown_and_await_termination(pool, 5)
+    print "done"
+
+This setup enables a natural flow.
+Although it may be tempting to then schedule everything through the completion service’s queue, there are limits.
+For example, if you’re writing a scalable web spider, you would want to externalize this work queue, but for simple management, it would certainly suffice.
+
+.. note::
+    
+    Why use tasks instead of threads? A common practice too often seen in production code is the addition of threading in a haphazard fashion:
+
+    Heterogeneous threads.
+    Perhaps you have one thread that queries the database.
+    And another that rebuilds an associated index.
+    What happens when you need to add another query?
+    
+    Dependencies are managed through a variety of channels, instead of being formally structured.
+    This can result in a rats’ nest of threads synchronizing on a variety of objects, often with timers and other event sources thrown in the mix.
+    
+    It’s certainly possible to make this sort of setup work—just debug away—but using tasks, with explicit wait-on dependencies and time scheduling, makes it far simpler to build a simple, scalable system.
+
+Thread Safety
+=============
+
+Thread safety addresses such questions as:
+
+- Can the (unintended) interaction of two or more threads corrupt a mutable object? This is especially dangerous for a collection like a list or a dictionary, because such corruption could potentially render the underlying data structure unusable or even produce infinite loops when traversing it.
+
+- Can an update get lost? Perhaps the canonical example is incrementing a counter.
+  In this case, there can be a data race with another thread in the time between retrieving the current value, and then updating with the incremented value.
+
+Jython ensures that its underlying mutable collection types -- dict, list, and set -- cannot be corrupted.
+But updates still might get lost in a data race.
+
+However, other Java collection objects that your code might use would typically not have such no-corruption guarantees.
+If you need to use LinkedHashMap, so as to support an ordered dictionary, you will need to consider thread safety if it will be both shared and mutated.
+
+Here’s a simple test harness we will use in our examples.
+ThreadSafetyTestCase subclasses unittest.TestCase, adding a new method assertContended.
+
+Listing 19-8. threadsafety.py
+::
+    
+    import threading
+    import unittest
+    
+    class ThreadSafetyTestCase(unittest.TestCase):
+    
+        def assertContended(self, f, num_threads=20, timeout=2., args=()):
+            threads = []
+            for i in xrange(num_threads):
+                t = threading.Thread(target=f, args=args)
+                t.start()
+                threads.append(t)
+            for t in threads:
+                t.join(timeout)
+                timeout = 0.
+            for t in threads:
+                self.assertFalse(t.isAlive())
+
+This new method runs a target function and asserts that all threads properly terminate.
+Then the testing code needs to check for any other invariants.
+
+For example, we use this idea in Jython to test that certain operations on the list type are atomic.
+The idea is to apply a sequence of operations that perform an operation, then reverse it.
+One step forward, one step back.
+The net result should be right where you started, an empty list, which is what the test code asserts.
+
+Listing 19-9. test_list.py
+::
+    
+    from threadsafety import ThreadSafetyTestCase
+    import threading
+    import time
+    import unittest
+    
+    class ListThreadSafety(ThreadSafetyTestCase):
+    
+        def test_append_remove(self):
+            lst = []
+            def tester():
+                # preserve invariant by adding, then removing a unique
+                # value (in this case, a reference to the worker thread
+                # executing this function)
+                ct = threading.currentThread()
+                for i in range(1000):
+                    lst.append(ct)
+                    time.sleep(0.0001)
+                    lst.remove(ct)
+            self.assertContended(tester)
+            self.assertEqual(lst, [])
+    
+    
+    if __name__ == '__main__':
+        unittest.main()
+
+Of course these concerns do not apply at all to immutable objects.
+Commonly used objects like strings, numbers, datetimes, tuples, and frozen sets are immutable, and you can create your own immutable objects too.
+
+There are a number of other strategies in solving thread safety issues.
+We will look at them as follows:
+
+- Synchronization
+
+- Atomicity
+
+- Thread Confinement
+
+- Safe Publication
+
+Synchronization
+---------------
+
+We use synchronization to control the entry of threads into code blocks corresponding to synchronized resources.
+Through this control we can prevent data races, assuming a correct synchronization protocol.
+(This can be a big assumption!)
+
+A threading.Lock ensures entry by only one thread.
+(In Jython, but unlike CPython, such locks are always reentrant; there’s no distinction between threading.Lock and threading.RLock.) Other threads have to wait until that thread exits the lock.
+Such explicit locks are the simplest and perhaps most portable synchronization to perform.
+
+You should generally manage the entry and exit of such locks through a with-statement; failing that, you must use a try-finally to ensure that the lock is always released when exiting a block of code.
+
+Here’s some example code using the with-statement.
+The code allocates a lock, then shares it amongst some tasks.
+
+Listing 19-10. test_lock.py—LockTestCase.test_with_lock
+::
+    
+    def test_with_lock(self):
+            counter = [0]
+            lock = Lock()
+            def loop100(counter):
+                for i in xrange(100):
+                    with lock:
+                        counter[0] += 1
+                    # sleeping helps ensures that all threads run in our test
+                    time.sleep(0.0001)
+    
+            self.assertContended(loop100, args=(counter,), num_threads=20)
+            self.assertEqual(counter[0], 2000) # 20 threads * 100 loops/thread
+
+Alternatively, you can do this with try-finally.
+
+Listing 19-11. test_lock.py—LockTestCase.test_try_finally_lock
+::
+    
+    def test_try_finally_lock(self):
+            counter = [0]
+            lock = Lock()
+            def loop100(counter):
+                for i in xrange(100):
+                    lock.acquire()
+                    try:
+                        counter[0] += 1
+                    finally:
+                        lock.release()
+                    time.sleep(0.0001)
+    
+            self.assertContended(loop100, args=(counter,), num_threads=20)
+            self.assertEqual(counter[0], 2000)
+
+But don’t do this.
+It’s actually slower than the with-statement, and using the with-statement version also results in more idiomatic Python code.
+
+Another possibility is to use the synchronize module, which is specific to Jython.
+This module provides a ``make_synchronized`` decorator function, which wraps any callable in Jython in a synchronized block.
+
+Listing 19-12. test_synchronized.py
+::
+    
+    from synchronize import make_synchronized
+    from threadsafety import ThreadSafetyTestCase
+    import time
+    import unittest
+    
+    @make_synchronized
+    def increment_counter(counter):
+        counter[0] += 1
+        # sleeping helps ensures that all threads run in our test
+        time.sleep(0.0001)
+    
+    class SynchronizedTestCase(ThreadSafetyTestCase):
+        
+        def test_counter(self):
+            def loop100(counter):
+                for i in xrange(100):
+                    increment_counter(counter)
+    
+            counter = [0]
+            self.assertContended(loop100, args=(counter,), num_threads=20)
+            self.assertEqual(counter[0], 2000) # 20 threads * 100 loops/thread
+    
+    if __name__ == '__main__':
+        unittest.main()
+
+In this case, you don’t need to explicitly release anything.
+Even in the case of an exception, the synchronization lock is always released upon exit from the function.
+Again, this version is also slower than the with-statement form, and it doesn’t use explicit locks.
+
+.. note:: Synchronization and the with-statement
+    
+    Jython’s current runtime (as of 2.5.1) can execute the with-statement form more efficiently through both runtime support and how this statement is compiled.
+    The reason is that most JVMs can perform analysis on a chunk of code (the *compilation unit*, including any inlining) to avoid synchronization overhead, so long as two conditions are met.
+    First, the chunk contains both the lock and unlock.
+    And second, the chunk is not too long for the JVM to perform its analysis.
+    The with-statement’s semantics make it relatively easy for us to do that when working with built-in types like threading.Lock, while avoiding the overhead of Java runtime reflection.
+    
+    In the future, support of the new invokedynamic bytecode should collapse these performance differences.
+
+The threading module offers portability, but it’s also minimalist.
+You may want to use the synchronizers in Java.util.concurrent instead of their wrapped versions in threading.
+In particular, this approach is necessary if you want to wait on a lock with a timeout.
+Also, you may want to use factories like Collections.synchronizedMap, when applicable, to ensure the underlying Java object has the desired synchronization.
+
+Deadlocks
+---------
+
+But use synchronization carefully: this code will always eventually deadlock.
+
+Listing 19-13. deadlock.py
+::
+    
+    from __future__ import with_statement
+    from threading import Thread, Lock
+    from java.lang.management import ManagementFactory
+    import time, threading
+    
+    def cause_deadlock():
+        counter = [0]
+        lock_one = Lock()
+        lock_two = Lock()
+        threads = [
+            Thread(
+                name="thread #1", target=acquire_locks,
+                args=(counter, lock_one, lock_two)), 
+            Thread(
+                name="thread #2 (reversed)", target=acquire_locks,
+                args=(counter, lock_two, lock_one))]
+        for thread in threads:
+            thread.setDaemon(True) # make shutdown possible after deadlock
+            thread.start()
+    
+        thread_mxbean = ManagementFactory.getThreadMXBean()
+        while True:
+            time.sleep(1)
+            print "monitoring thread", counter[0]
+            thread_ids = thread_mxbean.findDeadlockedThreads()
+            if thread_ids:
+                print "monitoring thread: deadlock detected, shutting down", list(thread_ids)
+                break
+    
+    def acquire_locks(counter, lock1, lock2):
+        # Should eventually deadlock if locks are acquired in different order
+        name = threading.currentThread().getName()
+        while True:
+            with lock1:
+                with lock2:
+                    counter[0] += 1
+                    print name, counter[0]
+    
+    
+    if __name__ == '__main__':
+        cause_deadlock()
+
+Deadlock results from a cycle of any length of wait-on dependencies.
+For example, Alice is waiting on Bob, but Bob is waiting on Alice.
+Without a timeout or other change in strategy—Alice just gets tired of waiting on Bob!—this deadlock will not be broken.
+
+Avoiding deadlocks can be done by never acquiring locks such that a cycle like that can be created.
+If we rewrote the example so that locks are acquired in the same order (Bob always allows Alice to go first), there would be no deadlocks.
+However, this ordering is not always so easy to do.
+Often, a more robust strategy is to allow for timeouts.
+
+Other Synchronization Objects
+-----------------------------
+
+The Queue module implements a first-in, first-out synchronized queue.
+(Synchronized queues are also called blocking queues, and that’s how they are described in java.util.concurrent.) Such queues represent a thread-safe way to send objects from one or more producing threads to one or more consuming threads.
+
+Often, you will define a poison object to shut down the queue.
+This will allow any consuming, but waiting, threads to immediately shut down.
+Or just use Java’s support for executors to get an off-the-shelf solution.
+
+If you need to implement another policy, such as last-in, first-out or based on a priority, you can use the comparable synchronized queues in java.util.concurrent as appropriate.
+(Note these have since been implemented in Python 2.6, so they will be made available when Jython 2.6 is eventually released.)
+
+Condition objects allow for one thread to notify another thread that’s waiting on a condition to wake up; notifyAll is used to wake up all such threads.
+Along with Queue, this is probably the most versatile of the synchronizing objects for real usage.
+
+Condition objects are always associated with a Lock.
+Your code needs to bracket waiting and notifying the condition by acquiring the corresponding lock, then finally (as always!) releasing it.
+As usual, this is easiest done in the context of the with-statement.
+
+For example, here’s how we actually implement a Queue in the standard library of Jython (just modified here to use the with-statement).
+We can’t use a standard Java blocking queue, because the requirement of being able to join on the queue when there’s no more work to be performed requires a third condition variable.
+
+Listing 19-15. Queue.py
+::
+    
+    """A multi-producer, multi-consumer queue."""
+    
+    from __future__ import with_statement
+    from time import time as _time
+    from collections import deque
+    
+    __all__ = ['Empty', 'Full', 'Queue']
+    
+    class Empty(Exception):
+        "Exception raised by Queue.get(block=0)/get_nowait()."
+        pass
+    
+    class Full(Exception):
+        "Exception raised by Queue.put(block=0)/put_nowait()."
+        pass
+    
+    class Queue:
+        """Create a queue object with a given maximum size.
+    
+        If maxsize is <= 0, the queue size is infinite.
+        """
+        def __init__(self, maxsize=0):
+            try:
+                import threading
+            except ImportError:
+                import dummy_threading as threading
+            self._init(maxsize)
+            # mutex must be held whenever the queue is mutating.  All methods
+            # that acquire mutex must release it before returning.  mutex
+            # is shared between the three conditions, so acquiring and
+            # releasing the conditions also acquires and releases mutex.
+            self.mutex = threading.Lock()
+            # Notify not_empty whenever an item is added to the queue; a
+            # thread waiting to get is notified then.
+            self.not_empty = threading.Condition(self.mutex)
+            # Notify not_full whenever an item is removed from the queue;
+            # a thread waiting to put is notified then.
+            self.not_full = threading.Condition(self.mutex)
+            # Notify all_tasks_done whenever the number of unfinished tasks
+            # drops to zero; thread waiting to join() is notified to resume
+            self.all_tasks_done = threading.Condition(self.mutex)
+            self.unfinished_tasks = 0
+    
+        def task_done(self):
+            """Indicate that a formerly enqueued task is complete.
+    
+            Used by Queue consumer threads.  For each get() used to fetch a task,
+            a subsequent call to task_done() tells the queue that the processing
+            on the task is complete.
+    
+            If a join() is currently blocking, it will resume when all items
+            have been processed (meaning that a task_done() call was received
+            for every item that had been put() into the queue).
+    
+            Raises a ValueError if called more times than there were items
+            placed in the queue.
+            """
+            with self.all_tasks_done:
+                unfinished = self.unfinished_tasks - 1
+                if unfinished <= 0:
+                    if unfinished < 0:
+                        raise ValueError('task_done() called too many times')
+                    self.all_tasks_done.notifyAll()
+                self.unfinished_tasks = unfinished
+    
+        def join(self):
+            """Blocks until all items in the Queue have been gotten and processed.
+    
+            The count of unfinished tasks goes up whenever an item is added to the
+            queue. The count goes down whenever a consumer thread calls task_done()
+            to indicate the item was retrieved and all work on it is complete.
+    
+            When the count of unfinished tasks drops to zero, join() unblocks.
+            """
+            with self.all_tasks_done:
+                while self.unfinished_tasks:
+                    self.all_tasks_done.wait()
+    
+        def qsize(self):
+            """Return the approximate size of the queue (not reliable!)."""
+            self.mutex.acquire()
+            n = self._qsize()
+            self.mutex.release()
+            return n
+    
+        def empty(self):
+            """Return True if the queue is empty, False otherwise (not reliable!)."""
+            self.mutex.acquire()
+            n = self._empty()
+            self.mutex.release()
+            return n
+    
+        def full(self):
+            """Return True if the queue is full, False otherwise (not reliable!)."""
+            self.mutex.acquire()
+            n = self._full()
+            self.mutex.release()
+            return n
+    
+        def put(self, item, block=True, timeout=None):
+            """Put an item into the queue.
+    
+            If optional args 'block' is true and 'timeout' is None (the default),
+            block if necessary until a free slot is available. If 'timeout' is
+            a positive number, it blocks at most 'timeout' seconds and raises
+            the Full exception if no free slot was available within that time.
+            Otherwise ('block' is false), put an item on the queue if a free slot
+            is immediately available, else raise the Full exception ('timeout'
+            is ignored in that case).
+            """
+            with self.not_full:
+                if not block:
+                    if self._full():
+                        raise Full
+                elif timeout is None:
+                    while self._full():
+                        self.not_full.wait()
+                else:
+                    if timeout < 0:
+                        raise ValueError("'timeout' must be a positive number")
+                    endtime = _time() + timeout
+                    while self._full():
+                        remaining = endtime - _time()
+                        if remaining <= 0.0:
+                            raise Full
+                        self.not_full.wait(remaining)
+                self._put(item)
+                self.unfinished_tasks += 1
+                self.not_empty.notify()
+    
+        def put_nowait(self, item):
+            """Put an item into the queue without blocking.
+    
+            Only enqueue the item if a free slot is immediately available.
+            Otherwise raise the Full exception.
+            """
+            return self.put(item, False)
+    
+        def get(self, block=True, timeout=None):
+            """Remove and return an item from the queue.
+    
+            If optional args 'block' is true and 'timeout' is None (the default),
+            block if necessary until an item is available. If 'timeout' is
+            a positive number, it blocks at most 'timeout' seconds and raises
+            the Empty exception if no item was available within that time.
+            Otherwise ('block' is false), return an item if one is immediately
+            available, else raise the Empty exception ('timeout' is ignored
+            in that case).
+            """
+            with self.not_empty:
+                if not block:
+                    if self._empty():
+                        raise Empty
+                elif timeout is None:
+                    while self._empty():
+                        self.not_empty.wait()
+                else:
+                    if timeout < 0:
+                        raise ValueError("'timeout' must be a positive number")
+                    endtime = _time() + timeout
+                    while self._empty():
+                        remaining = endtime - _time()
+                        if remaining <= 0.0:
+                            raise Empty
+                        self.not_empty.wait(remaining)
+                item = self._get()
+                self.not_full.notify()
+                return item
+    
+        def get_nowait(self):
+            """Remove and return an item from the queue without blocking.
+    
+            Only get an item if one is immediately available. Otherwise
+            raise the Empty exception.
+            """
+            return self.get(False)
+    
+        # Override these methods to implement other queue organizations
+        # (e.g. stack or priority queue).
+        # These will only be called with appropriate locks held
+    
+        # Initialize the queue representation
+        def _init(self, maxsize):
+            self.maxsize = maxsize
+            self.queue = deque()
+    
+        def _qsize(self):
+            return len(self.queue)
+    
+        # Check whether the queue is empty
+        def _empty(self):
+            return not self.queue
+    
+        # Check whether the queue is full
+        def _full(self):
+            return self.maxsize > 0 and len(self.queue) == self.maxsize
+    
+        # Put a new item in the queue
+        def _put(self, item):
+            self.queue.append(item)
+    
+        # Get an item from the queue
+        def _get(self):
+            return self.queue.popleft()
+
+There are other mechanisms to synchronize, including exchangers, barriers, latches, etc.
+You can use semaphores to describe scenarios where it’s possible for multiple threads to enter, or use locks that are set up to distinguish reads from writes.
+There are many possibilities for the Java platform.
+In our experience, Jython should be able to work with any of them.
+
+Atomic Operations
+-----------------
+
+An atomic operation is inherently thread safe.
+Data races and object corruption do not occur, and it’s not possible for other threads to see an inconsistent view.
+
+Atomic operations are therefore simpler to use than synchronization.
+In addition, atomic operations will often use underlying support in the CPU, such as a compare-and-swap instruction.
+They may use locking too.
+The important thing to know is that the lock is not directly visible.
+Also, if synchronization is used, it’s not possible to expand the scope of the synchronization.
+In particular, callbacks and iteration are not feasible.
+
+Python guarantees the atomicity of certain operations, although at best it’s only informally documented.
+Fredrik Lundh’s article on “Thread Synchronization Methods in Python” summarizes the mailing list discussions and the state of the CPython implementation.
+Quoting his article, the following are atomic operations for Python code:
+
+- Reading or replacing a single instance attribute
+
+- Reading or replacing a single global variable
+
+- Fetching an item from a list
+
+- Modifying a list in place (e.g. adding an item using append)
+
+- Fetching an item from a dictionary
+
+- Modifying a dictionary in place (e.g. adding an item, or calling the clear method)
+
+Although unstated, this also applies to equivalent ops on the builtin set type.
+
+For CPython, this atomicity emerges from combining its Global Interpreter Lock (GIL), the Python bytecode virtual machine execution loop, and the fact that types like dict and list are implemented natively in C and do not release the GIL.
+
+Despite the fact that this is in some sense accidentally emergent, it is a useful simplification for the developer.
+It’s what existing Python code expects, so this is what we have implemented in Jython.
+
+In particular, because dict is a ConcurrentHashMap, we also expose the following methods to atomically update dictionaries:
+
+* ``setifabsent``
+
+* ``update``
+
+It’s important to note that iterations are not atomic, even on a ConcurrentHashMap.
+
+Atomic operations are useful, but they are pretty limited too.
+Often, you still need to use synchronization to prevent data races, and this has to be done with care to avoid deadlocks and starvation.
+
+Thread Confinement
+------------------
+
+Thread confinement is often the best way to resolve problems seen in working with mutable objects.
+In practice, you probably don’t need to share a large percentage of the mutable objects used in your code.
+Very simply put, if you don’t share, then thread safety issues go away.
+
+Not all problems can be reduced to using thread confinement.
+There are likely some shared objects in your system, but in practice most can be eliminated.
+And often the shared state can be someone else’s problem:
+
+- Intermediate objects don’t require sharing.
+  For example, if you are building up a buffer that is only pointed to by a local variable, you don’t need to synchronize.
+  It’s an easy prescription to follow, so long as you are not trying to keep around these intermediate objects to avoid allocation overhead: don’t do that.
+
+- Producer-consumer.
+  Construct an object in one thread, then hand it off to another thread.
+  You just need to use an appropriate synchronizer object, such as a Queue.
+
+- Application containers.
+  The typical database-driven web application makes for the classic case.
+  For example, if you are using modjy, then the database connection pools and thread pools are the responsibility of the servlet container.
+  And they are not directly observable.
+  (But don’t do things like share database connections across threads.) Caches and databases then are where you will see shared state.
+
+- Actors.
+  The actor model is another good example.
+  Send and receive messages to an actor (effectively an independent thread) and let it manipulate any objects it owns on your behalf.
+  Effectively this reduces the problem to sharing one mutable object, the message queue.
+  The message queue can then ensure any accesses are appropriately serialized, so there are no thread safety issues.
+
+Unfortunately thread confinement is not without issues in Jython.
+For example, if you use StringIO, you have to pay the cost that this class uses list, which is synchronized.
+Although it’s possible to further optimize the Jython implementation of the Python standard library, if a section of code is hot enough, you may want to consider rewriting that in Java to ensure no additional synchronization overhead.
+
+Lastly, thread confinement is not perfect in Python, because of the possibility of introspecting on frame objects.
+This means your code can see local variables in other threads, and the objects they point to.
+But this is really more of an issue for how optimizable Jython is when run on the JVM.
+It won’t cause thread safety issues if you don’t exploit this loophole.
+We will discuss this more in the next section, on the Python Memory Model.
+
+Python Memory Model
+===================
+
+Reasoning about concurrency in Python is easier than in Java.
+This is because the memory model is not as surprising to our conventional reasoning about how programs operate.
+However, this also means that Python code sacrifices significant performance to keep it simpler.
+
+Here’s why.
+In order to maximize Java performance, it’s allowed for a CPU to arbitrarily re-order the operations performed by Java code, subject to the constraints imposed by *happens-before* and *synchronizes-with* relationships.
+(The published Java memory model goes into more details on these constraints.) Although such reordering is not visible within a given thread, the problem is that it’s visible to other threads.
+Of course, this visibility only applies to changes made to non-local objects; thread confinement still applies.
+In particular, this means you cannot rely on the apparent sequential ordering of Java code when looking at two or more threads.
+
+Python is different.
+The fundamental thing to know about Python, and what we have implemented in Jython, is that setting any attribute in Python is a volatile write; and getting any attribute is a volatile read.
+This is because Python attributes are stored in dictionaries, and in Jython, this follows the semantics of the backing ConcurrentHashMap.
+So get and set are volatile.
+So this means that Python code has sequential consistency.
+Execution follows the ordering of statements in the code.
+There are no surprises here.
+
+This means that *safe publication* is pretty much trivial in Python, when compared to Java.
+Safe publication means the thread safe association of an object with a name.
+Because this is always a memory-fenced operation in Python, your code simply needs to ensure that the object itself is built in a thread-safe fashion; then publish it all at once by setting the appropriate variable to this object.
+
+If you need to create module-level objects—singletons—then you should do this in the top-level script of the module so that the module import lock is in effect.
+
+Interruption
+============
+
+Long-threading threads should provide some opportunity for cancellation.
+The typical pattern is something like the following example.
+
+Listing 19-16.
+::
+    
+    class DoSomething(Runnable):
+        def __init__(self):
+            cancelled = False
+    
+        def run(self):
+            while not self.cancelled:
+                do_stuff()
+
+Remember, Python variables are always volatile, unlike Java.
+There are no problems with using a cancelled flag like this.
+
+Thread interruption allows for even more responsive cancellation.
+In particular, if a thread is waiting on most any synchronizers, such as a condition variable or on file I/O, this action will cause the waited-on method to exit with an InterruptedException.
+(Unfortunately lock acquisition, except under certain cases such as using lockInterruptibly on the underlying Java lock, is not interruptible.)
+
+Although Python’s threading module does not itself support interruption, it is available through the standard Java thread API.
+First, let’s import this class (we will rename it to JThread so it doesn’t conflict with Python’s version).
+
+Listing 19-17.
+::
+    
+    from java.lang import Thread as JThread
+
+As you have seen, you can use Java threads as if they are Python threads.
+So logically you should be able to do the converse: use Python threads as if they are Java threads.
+It would be nice to make calls like JThread.interrupt(obj).
+
+.. note::
+    
+    Incidentally, this formulation, instead of obj.interrupt(), looks like a static method on a class, as long as we pass in the object as the first argument.
+    This adaptation is a good use of Python’s explicit self.
+
+But there’s a problem here.
+As of the latest released version (Jython 2.5.1), we forgot to include an appropriate __tojava__ method on the Thread class! So this looks like you can’t do this trick after all.
+
+Or can you? What if you didn’t have to wait until we fix this bug? You could explore the source code -- or look at the class with dir.
+One possibility would be to use the nominally private _thread attribute on the Thread object.
+After all _thread is the attribute for the underlying Java thread.
+Yes, this is an implementation detail, but it’s probably fine to use.
+It’s not so likely to change.
+
+But we can do even better.
+We can *monkey patch* the Thread class such that it has an appropriate __tojava__ method, but only if it doesn’t exist.
+So this patching is likely to work with a future version of Jython because we are going to fix this missing method before we even consider changing its implementation and removing _thread.
+
+So here’s how we can monkey patch, following a recipe of Guido van Rossum.
+
+Listing 19-18. monkeypatch.py
+::
+    
+    # http://mail.python.org/pipermail/python-dev/2008-January/076194.html 
+    # - a recipe of Guido van Rossum
+    
+    def monkeypatch_method(cls):
+        def decorator(func):
+            setattr(cls, func.__name__, func)
+            return func
+        return decorator
+    
+    # and a useful variant, with a good ugly name
+    
+    def monkeypatch_method_if_not_set(cls):
+        def decorator(func):
+            if not hasattr(cls, func.__name__):
+                setattr(cls, func.__name__, func)
+            return func
+        return decorator
+
+This monkeypatch_method decorator allows us to add a method to a class after the fact.
+(This is what Ruby developers call *opening* a class.) Use this power with care.
+But again, you shouldn’t worry too much when you keep such fixes to a minimum, especially when it’s essentially a bug fix like this one.
+In our case, we will use a variant, the monkeypatch_method_if_not_set decorator, to ensure we only patch if it has not been fixed by a later version.
+
+Putting it all together, we have this code:
+
+Listing 19-19. interrupt.py
+::
+    
+    from __future__ import with_statement
+    from threading import Condition, Lock, Thread
+    from java.lang import Thread as JThread, InterruptedException
+    from monkeypatch import monkeypatch_method_if_not_set
+    import time, threading
+    
+    @monkeypatch_method_if_not_set(Thread)
+    def __tojava__(self, java_type):
+        return self._thread
+    
+    def be_unfair():
+        unfair_condition = Condition()
+        threads = [
+            Thread(
+                name="thread #%d" % i,
+                target=wait_until_interrupted,
+                args=(unfair_condition,)) 
+            for i in xrange(5)]
+        for thread in threads:
+            thread.start()
+        time.sleep(5)
+    
+        # threads should not be doing anything now, can verify by looking at some shared state
+    
+        # instead of notifying, we will interrupt the threads
+        for thread in threads:
+            JThread.interrupt(thread)
+            # or you can use this equivalent op
+            # thread.__tojava__(JThread).interrupt()
+        for thread in threads:
+            thread.join()
+    
+    def wait_until_interrupted(cv):
+        name = threading.currentThread().getName()
+        with cv:
+            while not JThread.currentThread().isInterrupted():
+                try:
+                    print "Waiting pointlessly %s" % name
+                    cv.wait()
+                except InterruptedException, e:
+                    break
+        print "Finished %s" % name
+    
+    
+    if __name__ == '__main__':
+        be_unfair()
+
+(It does rely on the use of threading.Condition to have something to wait on.)
+
+Lastly, you could simply access interruption through the cancel method provided by a Future.
+No need to monkey patch!
+
+Summary
+=======
+
+Jython can fully take advantage of the underlying Java platform’s support for concurrency.
+You can also use the standard Python threading constructs, which in most cases just wrap the corresponding Java functionality.
+The standard mutable Python collection types have been implemented in Jython with concurrency in mind; and Python’s sequential consistency removes some potential bugs.
+
+But concurrent programming is still not easy to get right, either in Python or in Java.
+You should consider higher-level concurrency primitives, such as tasks, and you should be disciplined in how your code shares mutable state.
-Chapter 2:  Data Types and Referencing - *Final v1.0*
-+++++++++++++++++++++++++++++++++++++++++++++++++++++
+Chapter 2:  Data Types and Referencing
+++++++++++++++++++++++++++++++++++++++
 
 Programming languages and applications need data. We define applications to work with
 data, and we need to have containers that can be used to hold it. This chapter is all

DatabasesAndJython.rst

-Chapter 12: Databases and Jython: Object Relational Mapping and Using JDBC - *Final v1.0*
-+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+Chapter 12: Databases and Jython: Object Relational Mapping and Using JDBC
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 
 
 In this chapter, we will look at zxJDBC package, which is a

DefiningFunctionsandUsingBuilt-Ins.rst

-Chapter 4:  Defining Functions and Using Built-ins - *Final v1.0*
-+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+Chapter 4:  Defining Functions and Using Built-ins
+++++++++++++++++++++++++++++++++++++++++++++++++++
 
 Functions are the fundamental unit of work in Python. A function in Python
 performs a task and returns a result. In this chapter, we will start with the

DeploymentTargets.rst

+Chapter 17: Deployment Targets
+++++++++++++++++++++++++++++++
+
+Deployment of Jython applications varies from container to container.
+However, they are all very similar and usually allow deployment of WAR file or exploded directory web applications.
+Deploying to “the cloud” is a different scenario all together.
+Some cloud environments have typical Java application servers available for hosting, while others such as the Google App Engine run a bit differently.
+In this chapter, we’ll discuss how to deploy web-based Jython applications to a few of the more widely used Java application servers.
+We will also cover deployment of Jython web applications to the Google App Engine and mobile devices.
+Although many of the deployment scenarios are quite similar, this chapter will walk through some of the differences from container to container.
+
+In the end, one of the most important things to remember is that we need to make Jython available to our application.
+There are different ways to do this: either by ensuring that the *jython.jar* file is included with the application server, or by packaging the JAR directly into each web application.
+This chapter assumes that you are using the latter technique.
+Placing the *jython.jar* directly into each web application is a good idea because it allows the web application to follow the Java paradigm of “deploy anywhere.” You do not need to worry whether you are deploying to Tomcat or Glassfish because the Jython runtime is embedded in your application.
+
+Lastly, this section will briefly cover some of the reasons why mobile deployment is not yet a viable option for Jython.
+While a couple of targets exist in the mobile world, namely Android and JavaFX, both environments are still very new and Jython has not yet been optimized to run on either.
+
+Application Servers
+===================
+
+As with any Java web application, the standard web archive (WAR) files are universal throughout the Java application servers available today.
+This is good because it makes things a bit easier when it comes to the “write once run everywhere” philosophy that has been brought forth with the Java name.
+The great part of using Jython for deployment to application servers is just that, we can harness the technologies of the JVM to make our lives easier and deploy a Jython web application to any application server in the WAR format with very little tweaking.
+
+If you have not yet used Django or Pylons on Jython, then you may not be aware that the resulting application to be deployed is in the WAR format.
+This is great because it leaves no assumption as to how the application should be deployed.
+All WAR files are deployed in the same manner according to each application server.
+This section will discuss how to deploy a WAR file on each of the three most widely used Java application servers.
+Now, all application servers are not covered in this section mainly due to the number of servers available today.
+Such a document would take more than one section of a book, no doubt.
+However, you should be able to follow similar deployment instructions as those discussed here for any of the application servers available today for deploying Jython web applications in the WAR file format.
+
+Tomcat
+------
+
+Arguably the most widely used of all Java application servers, Tomcat offers easy management and a small footprint compared to some of the other options available.
+Tomcat will plug into most IDEs that are in use today, so you can manage the web container from within your development environment.
+This makes it handy to deploy and undeploy applications on-the-fly.
+For the purposes of this section, we’ve used Netbeans 6.7, so there may be some references to it.
+
+To get started, download the Apache Tomcat server from the site at http://tomcat.apache.org/. Tomcat is constantly evolving, so we’ll note that when writing this book the deployment procedures were targeted for the 6.0.20 release.
+Once you have downloaded the server and placed it into a location on your hard drive, you may have to change permissions.
+We had to use the *chmod +x* command on the entire apache-tomcat-6.0.20 directory before we were able to run the server.
+You will also need to configure an administrative account by going into the */conf/tomcat-users.xml* file and adding one.
+Be sure to grant the administrative account the “manager” role.
+This should look something like the following once completed.
+
+Listing 17-1. tomcat-users.xml
+::
+    
+    <tomcat-users>
+       <user username="admin" password="myadminpassword" roles="manager"/>
+    </tomcat-users>
+
+After this has been done, you can add the installation to an IDE environment of your choice if you’d like.
+For instance, if you wish to add to Netbeans 6.7 you will need to go to the “Services” tab in the navigator, right-click on servers, choose “Tomcat 6.x” option, and then fill in the appropriate information pertaining to your environment.
+Once complete, you will be able to start, stop, and manage the Tomcat installation from the IDE.
+
+Deploying Web Start
+~~~~~~~~~~~~~~~~~~~
+
+Deploying a web start application is as easy as copying the necessary files to a location on the web server that is accessible via the web.
+In the case of Tomcat, you will need to copy the contents of your web start application to a single directory contained within the “<tomcat-root>/webapps/ROOT” directory.
+For instance, if you have a web-start application entitled , then you would package the JAR file along with the JNLP and HTML file for the application into a directory entitled  and then place that directory into the “<tomcat-root>/webapps/ROOT” directory.
+
+Once the application has been copied to the appropriate locations, you should be able to access it via the web if Tomcat is started.
+The URL should look something like the following: *http://your-server:8080/JythonWebStart/launch.jnlp*.
+Of course, you will need to use the server name and the port that you are using along with the appropriate JNLP name for your application.
+
+Deploying a WAR or Exploded Directory Application
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To deploy a web application to Tomcat, you have two options.
+You can either use a WAR file including all content for your entire web application, or you can deploy an exploded directory application which is basically copy-and-paste for your entire web application directory structure into the “<tomcat-root>/webapps/ROOT” directory.
+Either way will work the same, and we will discuss each technique in this section.
+
+For manual deployment of a web application, you can copy either your exploded directory web application or your WAR file into the “<tomcat-root>/webapps” directory.
+By default, Tomcat is setup to “autodeploy” applications.
+This means that you can have Tomcat started when you copy your WAR or exploded directory into the “webapps” location.
+Once you’ve done this, you should see some feedback from the Tomcat server if you have a terminal open (or from within the IDE).
+After a few seconds the application should be deployed successfully and available via the URL.
+The bonus to deploying exploded directory applications is that you can take any file within the application and change it at will.
+Once you are done with the changes, that file will be redeployed when you save it.
+. .this really saves on development time!
+
+If you do not wish to have autodeploy enabled (perhaps in a production environment), then you can deploy applications on startup of the server.
+This process is basically the same as “autodeploy,” except any new applications that are copied into the “webapps” directory are not deployed until the server is restarted.
+Lastly, you can always make use of the Tomcat manager to deploy web applications as well.
+To do this, open your web browser to the index of Tomcat, usually http://localhost:8080/index.html, and then click on the “Manager” link in the left-hand menu.
+You will need to authenticate at that point using your administrator password, but once you are in the console deployment is quite easy.
+In an effort to avoid redundancy, we will once again redirect you to the Tomcat documentation for more information on deploying a web application via the Tomcat manager console.
+
+Glassfish
+---------
+
+At the time of writing, the Glassfish V2 application server was mainstream and widely used.
+The Glassfish V3 server was still in preview mode, but showed a lot of potential for Jython application deployment.
+In this section, we will cover WAR and web start deployment to Glassfish V2, because it is the most widely used version.
+We will also discuss deployment for Django on Glassfish V3, because this version has added support for Django (and more Python web frameworks soon).
+Glassfish is very similar to Tomcat in terms of deployment, but there are a couple of minor differences which will be covered in this section.
+
+To start out, you will need to download a glassfish distribution from the site at https://glassfish.dev.java.net/. Again, we recommend downloading V2, because it is the most widely used at the time of this writing.
+Installation is quite easy, but a little more involved than that of Tomcat.
+The installation of Glassfish will not be covered in this text, because it varies depending upon which version you are using.
+There are detailed instructions for each version located on the Glassfish website, so we will redirect you there for more information.
+
+Once you have Glassfish installed, you can utilize the server via the command-line or terminal, or you can use an IDE just like Tomcat.
+To register a Glassfish V2 or V3 installation with Netbeans 6.7, just go to the “Services” tab in the Netbeans navigator and right-click on “Servers” and then add the version you are planning to register.
+Once the “Add Server Instance” window appears, simply fill in the information depending upon your environment.
+
+There is an administrative user named “admin” that is set up by default with a Glassfish installation.
+In order to change the default password, it is best to startup Glassfish and log into the administrative console.
+The default administrative console port is 4848.
+
+Deploying Web Start
+~~~~~~~~~~~~~~~~~~~
+
+Deploying a web start application is basically the same as any other web server, you simply make the web start JAR, JNLP, and HTML file accessible via the web.
+On Glassfish, you need to traverse into your “domain” directory and you will find a “docroot” inside.
+The path should be similar to “<glassfish-install-loc>/domains/domain1/docroot”. Anything placed within the docroot area is visible to the web, so of course this is where you will place any web-start application directories.
+Again, a typical web start application will consist of your application JAR file, a JNLP file, and an HTML page used to open the JNLP.
+All of these files should typically be placed inside a directory appropriately named per your application, and then you can copy this directory into docroot.
+
+WAR File and Exploded Directory Deployment
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Again, there are a variety of ways to deploy an application using Glassfish.
+Let’s assume that you are using V2, you have the option to “hot deploy” or use the Glassfish Admin Console to deploy your application.
+Glassfish will work with either an exploded directory or WAR file deployment scenario.
+By default, the Glassfish “autodeploy” option is turned on, so it is quite easy to either copy your WAR or exploded directory application into the autodeploy location to deploy.
+If the application server is started, it will automatically start your application (if it runs without issues).
+The autodeploy directory for Glassfish V2 resides in the location “<glassfish-install-loc>/domains/domain1/autodeploy.”
+
+Glassfish v3 Django Deployment
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The Glassfish V3 server has some capabilities built into it to help facilitate the process of deploying a Django application.
+In the future, there will also be support for other Jython web frameworks such as Pylons.
+
+Other Java Application Servers
+------------------------------
+
+If you have read through the information contained in the previous sections, then you have a fairly good idea of what it is like to deploy a Jython web application to a Java application server.
+There is no difference between deploying Jython web applications and Java web applications for the most part.
+You must be sure that you include  as mentioned in the introduction, but for the most part deployment is the same.
+However, we have run into cases with some application servers such as JBoss where it wasn’t so cut-and-dry to run a Jython application.
+For instance, we have tried to deploy a Jython servlet application on JBoss application server 5.1.0 GA and had lots of issues.
+For one, we had to manually add  to the application because we were unable to compile the application in Netbeans without doing so...this was not the case with Tomcat or Glassfish.
+Similarly, we had issues trying to deploy a Jython web application to JBoss as there were several errors that had incurred when the container was scanning  for some reason.
+
+All in all, with a bit of tweaking and perhaps an additional XML configuration file in the application, Jython web applications will deploy to *most* Java application servers.
+The bonus to deploying your application on a Java application server is that you are in complete control of the environment.
+For instance, you could embed the *jython.jar* file into the application server lib directory so that it was loaded at startup and available for all applications running in the environment.
+Likewise, you are in control of other necessary components such as database connection pools and so forth.
+If you deploy to another service that lives in “the cloud,” you have very little control over the environment.
+In the next section, we’ll study one such environment by Google which is known as the Google App Engine.
+While this “cloud” service is an entirely different environment than your basic Java web application server, it contains some nice features that allow one to test applications prior to deployment in the cloud.
+
+Google App Engine
+=================
+
+The new kid on the block, at least for the time of this writing, is the Google App Engine.
+Fresh to the likes of the Java platform, the Google App Engine can be used for deploying applications written in just about any language that runs on the JVM, Jython included.
+The App Engine went live in April of 2008, allowing Python developers to begin using its services to host Python applications and libraries.
+In the spring of 2009, the App Engine added support for the Java platform.
+Along with support of the Java language, most other languages that run on the JVM will also deploy and run on the Google App Engine, including Jython.
+It has been mentioned that more programming languages will be supported at some point in the future, but at the time of this writing Python and Java were the only supported languages.
+
+The App Engine actually runs a slightly slimmed-down version of the standard Java library.
+You must download and develop using the Google App Engine SDK for Java in order to ensure that your application will run in the environment.
+You can download the SDK by visiting this link: http://code.google.com/appengine/downloads.html along with viewing the extensive documentation available on the Google App Engine site.
+The SDK comes complete with a development web server that can be used for testing your code before deploying, and several demo applications ranging from easy JSP programs to sophisticated demos that use Google authentication.
+No doubt about it, Google has done a good job at creating an easy learning environment for the App Engine so that developers can get up and running quickly.
+
+In this section you will learn how to get started using the Google App Engine SDK, and how to deploy some Jython web applications.
+You will learn how to deploy a Jython servlet application as well as a WSGI application utilizing modjy.
+Once you’ve learned how to develop and use a Jython Google App Engine program using the development environment, you will learn a few specifics about deploying to the cloud.
+If you have not done so already, be sure to visit the link mentioned in the previous paragraph and download the SDK so that you can follow along in the sections to come.
+
+n **Note** The Google App Engine is a very large topic.
+Entire books could be written on the subject of developing Jython applications to run on the App Engine.
+With that said, we will cover the basics to get you up and running with developing Jython applications for the App Engine.
+Once you’ve read through this section, we suggest going to the Google App Engine documentation for further details.
+
+Starting With an SDK Demo
+-------------------------
+
+We will start by running the demo application known as “guestbook” that comes with the Google App Engine SDK.
+This is a very simple Java application that allows one to sign in using an email address and post messages to the screen.
+In order to start the SDK web server and run the “guestbook” application, open up a terminal and traverse into the directory where you expanded the Google App Engine .zip file and run the following command:
+::
+    
+    <app-engine-base-directory>/bin/dev_appserver.sh demos/guestbook/war
+
+Of course, if you are running on windows there is a corresponding .bat script for you to run that will start the web server.
+Once you’ve issued the preceding command it will only take a second or two before the web server starts.
+You can then open a browser and traverse to *http://localhost:8080* to invoke the “guestbook” application.
+This is a basic JSP-based Java web application, but we can deploy a Jython application and use it in the same manner as we will see in a few moments.
+You can stop the web server by pressing “CTRL+C”.
+
+Deploying to the Cloud
+----------------------
+
+Prior to deploying your application to the cloud, you must of course set up an account with the Google App Engine.
+If you have another account with Google such as GMail, then you can easily activate your App Engine account using that same username.
+To do so, go to the Google App Engine link: http://code.google.com/appengine/ and click “Sign Up.” Enter your existing account information or create a new account to get started.
+
+After your account has been activated you will need to create an application by clicking on the “Create Application” button.
+You have a total of 10 available application slots to use if you are making use of the free App Engine account.
+Once you’ve created an application then you are ready to begin deploying to the cloud.
+In this section of the book, we create an application known as *jythongae*.
+This is the name of the application that you must create on the App Engine.
+You must also ensure that this name is supplied within the *appengine-web.xml* file.
+
+Working With a Project
+----------------------
+
+The Google App Engine provides project templates to get you started developing using the correct directory structure.
+Eclipse has a plug-in that makes it easy to generate Google App Engine projects and deploy them to the App Engine.
+If interested in making use of the plug-in, please visit http://code.google.com/appengine/docs/java/tools/eclipse.html to read more information and download the plug-in.
+Similarly, Netbeans has an App Engine plug-in that is available on the Kenai site appropriately named  (http://kenai.com/projects/nbappengine).
+In this text we will cover the use of Netbeans 6.7 to develop a simple Jython servlet application to deploy on the App Engine.
+You can either download and use the template available with one of these IDE plug-ins, or simply create a new Netbeans project and make use of the template provided with the App Engine SDK (<app-engine-base-directory/demos/new_project_template>) to create your project directory structure.
+
+
+For the purposes of this tutorial, we will make use of the *nbappengine* plug-in.
+If you are using Eclipse you will find a section following this tutorial that provides some Eclipse plug-in specifics.
+
+In order to install the nbappengine plug-in, you add the ‘App Engine’ update center to the Netbeans plug-in center by choosing the Settings tab and adding the update center using http://deadlock.netbeans.org/hudson/job/nbappengine/lastSuccessfulBuild/artifact/build/updates/updates.xml.gz as the URL.
+Once you’ve added the new update center you can select the Available Plugins tab and add all of the plug-ins in the “Google App Engine” category, then choose Install.
+After doing so, you can add the “App Engine” as a server in your Netbeans environment using the “Services” tab.
+To add the server, point to the base directory of your Google App Engine SDK.
+Once you have added the App Engine server to Netbeans, it will become an available deployment option for your web applications.
+
+Create a new Java web project and name it JythonGAE.
+For the deployment server, choose “Google App Engine,” and you will notice that when your web application is created an additional file will be created within the WEB-INF directory named appengine-web.xml.
+This is the Google App Engine configuration file for the JythonGAE application.
+Any of the .py files that we wish to use in our application must be mapped in this file so that they will not be treated as static files by the Google App Engine.
+By default, Google App Engine treats all files outside of the WEB-INF directory as static unless they are JSP files.
+Our application is going to make use of three Jython servlets, namely NewJythonServlet.py, AddNumbers.py and AddToPage.py.
+In our appengine-web.xml file we can exclude all .py files from being treated as static by adding the suffix to the exclusion list as follows.
+
+Listing 17-2. appengine-web.xml
+::
+    
+    <?xml version="1.0" encoding="UTF-8"?>
+    <appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
+        <application>jythongae</application>
+        <version>1</version>
+        <static-files>
+            <exclude path="/**.py"/>
+        </static-files>
+        <resource-files/>
+        <ssl-enabled>false</ssl-enabled>
+        <sessions-enabled>true</sessions-enabled>
+    </appengine-web-app>
+
+At this point we will need to create a couple of additional directories within our WEB-INF project directory.
+We should create a *lib* directory and place *jython.jar* and *appengine-api-1.0-sdk-1.2.2.jar* into the directory.
+Note that the App Engine JAR may be named differently according to the version that you are using.
+We should now have a directory structure that resembles the following:
+
+Listing 17-3. 
+::
+    
+    JythonGAE
+        WEB-INF
+            lib
+                jython.jar
+                appengine-api-1.0-sdk-1.2.2.jar
+            appengine-web.xml
+            web.xml
+        src
+        web
+
+Now that we have the application structure set up, it is time to begin building the actual logic.
+In a traditional Jython servlet application we need to ensure that the *PyServlet* class is initialized at startup and that all files ending in *.py* are passed to it.
+As we’ve seen in Chapter 13, this is done in the *web.xml* deployment descriptor.
+However, we have found that this alone does not work when deploying to the cloud.
+We found some inconsistencies while deploying against the Google App Engine development server and deploying to the cloud.
+For this reason, we will show you the way that we were able to get the application to function as expected in both the production and development Google App Engine environments.
+In Chapter 12, the object factory pattern for coercing Jython classes into Java was discussed.
+If this same pattern is applied to Jython servlet applications, then we can use the factories to coerce our Jython servlet into Java byte code at runtime.
+We then map the resulting coerced class to a servlet mapping in the application’s web.xml deployment descriptor.
+We can also deploy our Jython applets and make use of *PyServlet* mapping to the *.py* extension in the *web.xml*.
+We will comment in the source where the code for the two implementations differs.
+
+Object Factories with App Engine
+--------------------------------
+
+In order to use object factories to coerce our code, we must use an object factory along with a Java interface, and once again we will use the PlyJy project to make this happen.
+Please note that if you choose to not use the object factory pattern and instead use PyServlet you can safely skip forward to the next subsection.
+The first step is to add  to the  directory that we created previously to ensure it is bundled with our application.
+There is a Java servlet contained within the PlyJy project named , and what this Java servlet does is essentially use the  class to coerce a named Jython servlet and then invoke its resulting  and  methods.
+There is also a simple Java interface named  in the project, and it must be implemented by our Jython servlet in order for the coercion to work as expected.
+
+
+Using PyServlet Mapping
+-----------------------
+
+When we use the PyServlet mapping implementation, there is no need to coerce objects using factories.
+You simply set up a servlet mapping within  and use your Jython servlets directly with the .py extension in the URL.
+However, we’ve seen issues while using PyServlet on the App Engine in that this implementation will deploy to the development App Engine server environment, but when deployed to the cloud you will receive an error when trying to invoke the servlet.
+It is because of these inconsistencies that we chose to implement the object factory solution for Jython servlet to App Engine deployment.
+
+Example Jython Servlet Application for App Engine
+-------------------------------------------------
+
+The next piece of the puzzle is the code for our application.
+In this example, we’ll make use of a simple servlet that displays some text as well as the same example that was used in Chapter 13 with JSP and Jython.
+The following code sets up three Jython servlets.
+The first servlet simply displays some output, the next two perform some mathematical logic, and then there is a JSP to display the results for the mathematical servlets.
+
+Listing 17-4. NewJythonServlet.py
+::
+    
+    from javax.servlet.http import HttpServlet
+    from org.plyjy.interfaces import JythonServletInterface
+    
+    class NewJythonServlet (JythonServletInterface, HttpServlet):
+            def doGet(self,request,response):
+                    self.doPost (request,response)
+    
+            def doPost(self,request,response):
+                    toClient = response.getWriter()
+                    response.setContentType ("text/html")
+                    toClient.println ("<html><head><title>Jython Servlet Test Using Object Factory</title>" +
+                                                      "<body><h1>Jython Servlet Test for GAE</h1></body></html>")
+    
+            def getServletInfo(self):
+                return "Short Description"
+
+Listing 17-5. AddNumbers.py
+::
+    
+    import javax
+    class add_numbers(javax.servlet.http.HttpServlet):
+        def doGet(self, request, response):
+            self.doPost(request, response)
+        def doPost(self, request, response):
+            x = request.getParameter("x")
+            y = request.getParameter("y")
+            if not x or not y:
+                sum = "<font color='red'>You must place numbers in each value box</font>"
+            else:
+                try:
+                    sum = int(x) + int(y)
+                except ValueError, e:
+                    sum = "<font color='red'>You must place numbers only in each value box</font>"
+            request.setAttribute("sum", sum)
+            dispatcher = request.getRequestDispatcher("testJython.jsp")
+            dispatcher.forward(request, response)
+
+Listing 17-6. AddToPage.py
+::
+    
+    import java, javax, sys
+    
+    class add_to_page(javax.servlet.http.HttpServlet):
+        def doGet(self, request, response):
+            self.doPost(request, response)
+    
+        def doPost(self, request, response):
+            addtext = request.getParameter("p")
+            if not addtext:
+                addtext = ""
+    
+            request.setAttribute("page_text", addtext)
+            dispatcher = request.getRequestDispatcher("testJython.jsp")
+            dispatcher.forward(request, response)
+
+Listing 17-7. testjython.jsp
+
+Note that this implementation differs if you plan to make use of the object factory technique.
+Instead of using  and  as your actions, you would utilize the servlet instead, namely  and 
+::
+    
+    <html>
+        <head>
+            <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
+            <title>Jython JSP Test</title>
+        </head>
+        <body>
+            <form method="GET" action="add_to_page.py">
+                <input type="text" name="p">
+                <input type="submit">
+            </form>
+            <% Object page_text = request.getAttribute("page_text");
+               Object sum = request.getAttribute("sum");
+               if(page_text == null){
+                   page_text = "";
+               }
+               if(sum == null){
+                   sum = "";
+               }
+            %>
+            <br/>
+                <p><%= page_text %></p>
+            <br/>
+            <form method="GET" action="add_numbers.py">
+                <input type="text" name="x">
+                +
+                <input type="text" name="y">
+                =
+                <%= sum %>
+                <br/>
+                <input type="submit" title="Add Numbers">
+            </form>
+    
+        </body>
+    </html>
+
+As mentioned previously, it is important that all of the Jython servlets reside within your classpath somewhere.
+If using Netbeans, you can either place the servlets into the source root of your project (not inside a package), or you can place them in the web folder that contains your JSP files.
+If doing the latter, we have found that you may have to tweak your CLASSPATH a bit by adding the web folder to your list of libraries from within the project properties.
+Next, we need to ensure that the deployment descriptor includes the necessary servlet definitions and mappings for the application.
+Now, if you are using the object factory implementation and the *JythonServletFacade* servlet, you would have noticed that there is a variable named *PyServletName* which the JythonObjectFactory is using as the name of our Jython servlet.
+Well, within the *web.xml* we must pass an *<init-param>* using *PyServletName* as the *<param-name>* and the name of our Jython servlet as the *<param-value>*.
+This will basically pass the name of the Jython servlet to the *JythonServletFacade* servlet so that it can be used by the object factory.
+
+Listing 17-8. web.xml
+::
+    
+    <web-app>
+        <display-name>Jython Google App Engine</display-name>
+    
+        <!-- Used for the PyServlet Implementation -->
+        <servlet>
+            <servlet-name>PyServlet</servlet-name>
+            <servlet-class>org.python.util.PyServlet</servlet-class>
+        </servlet>
+    
+        <!-- The next three servlets are used for the object factory implementation only.
+            They can be excluded in the PyServlet implementation -->
+        <servlet>
+            <servlet-name>NewJythonServlet</servlet-name>
+            <servlet-class>org.plyjy.servlets.JythonServletFacade</servlet-class>
+            <init-param>
+                <param-name>PyServletName</param-name>
+                <param-value>NewJythonServlet</param-value>
+            </init-param>
+        </servlet>
+        <servlet>
+            <servlet-name>AddNumbers</servlet-name>
+            <servlet-class>org.plyjy.servlets.JythonServletFacade</servlet-class>
+            <init-param>
+                <param-name>PyServletName</param-name>
+                <param-value>AddNumbers</param-value>
+            </init-param>
+        </servlet>
+        <servlet>
+            <servlet-name>AddToPage</servlet-name>
+            <servlet-class>org.plyjy.servlets.JythonServletFacade</servlet-class>
+            <init-param>
+                <param-name>PyServletName</param-name>
+                <param-value>AddToPage</param-value>
+            </init-param>
+        </servlet>
+    
+        <!-- The following mapping should be used for the PyServlet implementation -->
+        <servlet-mapping>
+            <servlet-name>PyServlet</servlet-name>
+            <url-pattern>*.py</url-pattern>
+        </servlet-mapping>
+    
+        <!-- The following three mappings are used in the object factory implementation -->
+    
+        <servlet-mapping>
+            <servlet-name>NewJythonServlet</servlet-name>
+            <url-pattern>/NewJythonServlet</url-pattern>
+        </servlet-mapping>
+        <servlet-mapping>
+            <servlet-name>AddNumbers</servlet-name>
+            <url-pattern>/AddNumbers</url-pattern>
+        </servlet-mapping>
+        <servlet-mapping>
+            <servlet-name>AddToPage</servlet-name>
+            <url-pattern>/AddToPage</url-pattern>
+        </servlet-mapping>
+    </web-app>
+
+Note that when using the PyServlet implementation you should exclude those portions in the *web.xml* above that are used for the object factory implementation.
+The PyServlet mapping can be contained within the *web.xml* in both implementations without issue.
+That’s it, now you can deploy the application to your Google App Engine development environment, and it should run without any issues.
+You can also choose to deploy to another web server to test for compatibility if you wish.
+You can deploy directly to the cloud by right-clicking the application and choosing the “Deploy to App Engine” option.
+
+Using Eclipse
+-------------
+
+If you wish to use the Eclipse IDE for development, you should definitely download the Google App Engine plug-in using the link provided earlier in the chapter.
+You should also use the PyDev plug-in which is available at http://pydev.sourceforge.net/. For the purposes of this section, we used Eclipse Galileo and started a new project named “JythonGAE” as a Google Web Application.
+When creating the project, make sure you check the box for using Google App Engine and uncheck the Google Web Toolkit option.
+You will find that Eclipse creates a directory structure for your application that is much the same as the project template that is included with the Google App Engine SDK.
+
+If you follow through the code example from the previous section, you can create the same code and set up the *web.xml* and *appengine-web.xml* the same way.
+The key is to ensure that you create a *lib* directory within the *WEB-INF* and you place the files in the appropriate location.
+You will need to ensure that your Jython servlets are contained in your CLASSPATH by either adding them to the source root for your project, or by going into the project properties and adding the *war* directory to your *Java Build Path*.
+When doing so, make sure you do *not* include the *WEB-INF* directory or you will receive errors.
+
+When you are ready to deploy the application, you can choose to use the Google App Engine development environment or deploy to the cloud.
+You can run the application by right-clicking on the project and choosing *Run As* option and then choose the Google Web Application option.
+The first time you run the application you may need to set up the runtime.
+If you are ready to deploy to the cloud, you can right-click on the project and choose the *Google* -> *Deploy to App Engine* option.
+After entering your Google username and password then your application will be deployed.
+
+Deploy Modjy to GAE
+-------------------
+
+We can easily deploy WSGI applications using Jython’s modjy API as well.
+To do so, you need to add an archive of the Jython  directory to your WEB-INF project directory.
+According to the modjy web site, you need to obtain the source for Jython, then zip the  directory and place it into another directory along with a file that will act as a pointer to the zip archive.
+The modjy site names the directory  and names the pointer file . This pointer file can be named anything as long as the suffix is . Inside the pointer file you need to explicitly name the zip archive that you had created for the  directory contents.
+Let’s assume you named it lib.zip, in this case we will put the text “lib.zip” without the quotes into the  file.
+Now if we add the modjy  demonstration application to the project then our directory structure should look as follows:
+
+Listing 17-9. 
+::
+    
+    modjy_app
+        demo_app.py
+        WEB-INF
+            lib
+                jython.jar
+                appengine-api-1.0-sdk-1.2.2.jar
+            python-lib
+                lib.zip
+                all.pth
+
+Now if we run the application using Tomcat it should run as expected.
+Likewise, we can run it using the Google App Engine SDK web server and it should provide the expected results.
+
+The Google App Engine is certainly an important deployment target for Jython.
+Google offers free hosting for smaller applications, and they also base account pricing on bandwidth.
+No doubt that it is a good way to put up a small site, and possibly build on it later.
+Most importantly, you can deploy Django, Pylons, and other applications via Jython to the App Engine by setting up your App Engine applications like the examples we had shown in this chapter.
+
+Java Store
+==========
+
+Another deployment target that is hot off the presses at the time of this book is the Java Store or Java Warehouse.
+This is a new concept brought to market by Sun Microsystems in order to help Java software developers market their applications via a single shop that is available online via a web start application.
+Similar to other application venues, The Java Store is a storefront application where people can go to search for applications that have been submitted by developers.
+The Java Warehouse is the repository of applications that are contained within the Java Store.
+This looks to be a very promising target for Java and Jython developers alike.
+It  be as easy as generating a JAR file that contains a Jython application and deploying it to the Java Store.
+Unfortunately, because the program is still in alpha mode at this time, we are unable to provide any specific details on distributing Jython applications via the Java Store.
+However, there are future plans to make alternative VM language applications easily deployable to the Java Warehouse.
+At this time, it is certainly possible to deploy a Jython application to the warehouse, but it can only deploy as a Java application.
+As of the time of this writing, only Java and JavaFX applications are directly deployable to the Java Warehouse.
+Please note that because this product is still in alpha mode, this book will not discuss such aspects of the program as memberships or fees that may be incurred for hosing your applications on the Java Store.
+
+The requirements for publishing an application to the warehouse are as follows:
+
+- Your application packed in a single jar file
+
+- Descriptive text to document your application
+
+- Graphic image files used for icons and to give the consumer an idea of your application’s look.
+
+In Chapter 13, we took a look at packaging and distributing Jython GUI applications in a JAR file.
+When a Jython application is packaged in a JAR file then it is certainly possible to use Java Web Start to host the application via the web.
+On the other hand, if one wishes to make a Jython GUI application available for purchase or for free, the Java Store would be another way of doing so.
+One likely way to deploy applications in a single JAR is to use the method discussed in Chapter 13, but there are other solutions as well.
+For instance, one could use the *One-Jar* product to create a single JAR file containing all of the necessary Jython code as well as other JAR files essential to the application.
+In the following section, we will discuss deployment of a Jython application using One-JAR so that you can see some similarities and differences to using the Jython standalone JAR technique.
+
+Deploying a Single JAR
+----------------------
+
+In order to deploy an application to the Java Warehouse, it must be packaged as a single JAR file.
+We’ve already discussed packaging Jython applications into a JAR file using the Jython standalone method in Chapter 13. In this section, you will learn how to make use of the One-JAR (http://one-jar.sourceforge.net/) product to distribute client-based Jython applications.
+In order to get started, you will need to grab a copy of One-JAR.
+There are a few options available on the download site, but for our purposes we will package an application using the source files for One-JAR.
+Once downloaded, the source for the project should look as follows.
+
+Listing 17-10.
+::
+    
+    src
+        com
+            simontuffs
+                onejar
+                    Boot.java
+                    Handler.java
+                    IProperties.java
+                    JarClassLoader.java
+
+This source code for the One-Jar project must reside within the JAR file that we will build.
+Next, we need to create separate source directories for both our Jython source and our Java source.
+Likewise, we will create a separate source directory for the One-Jar source.
+Lastly, we’ll create a *lib* directory into which we will place all of the required JAR files for the application.
+In order to run a Jython application, we’ll need to package the Jython project source into a JAR file for our application.
+We will not need to use the entire *jython.jar*, but rather only a standalone version of it.
+The easiest way to obtain a standalone Jython JAR is to run the installer and choose the standalone option.
+After this is done, simply add the resulting jython.jar to the lib directory of application.
+In the end, the directory structure should resemble the following.
+
+Listing 17-11. 
+::
+    
+    one-jar-jython-example
+        java
+        lib
+            jython.jar
+        LICENSE.txt
+        onejar
+            src
+                com
+                
+                    simontuffs
+                        onejar
+                            Boot.java
+                            Handler.java
+                            IProperties.java
+                            JarClassLoader.java
+                one-jar-license.txt
+        src
+
+As you can see from the depiction of the file structure in this example, the *src* directory will contain our Jython source files.
+The LICENSE.txt included in this example was written by Ryan McGuire (http://www.enigmacurry.com).
+He has a detailed explanation of using One-Jar on his blog, and we’ve replicated some of his work in this example.
+. .including a version of the build.xml that we will put together in order to build the application.
+Let’s take a look at the build file that we will use to build the application JAR.
+In this example we are using Apache Ant for the build system, but you could choose something different if you’d like.
+
+Listing 17-12. build.xml
+::
+    
+    <project name="JythonSwingApp" default="dist" basedir=".">
+    
+      <!-- #################################################
+           These two properties are the only ones you are
+           likely to want to change for your own projects:     -->
+      <property name="jar.name" value="JythonSwingApp.jar" />
+      <property name="java-main-class" value="Main" />
+      <!-- ##################################################  -->
+    
+      <!-- Below here you don't need to change for simple projects -->
+      <property name="src.dir" location="src"/>
+      <property name="java.dir" location="java"/>
+      <property name="onejar.dir" location="onejar"/>
+      <property name="java-build.dir" location="java-build"/>
+      <property name="build.dir" location="build"/>
+      <property name="lib.dir" location="lib"/>
+    
+      <path id="classpath">
+        <fileset dir="${lib.dir}" includes="**/*.jar"/>
+      </path>
+    
+      <target name="clean">
+        <delete dir="${java-build.dir}"/>
+        <delete dir="${build.dir}"/>
+        <delete file="${jar.name}"/>
+      </target>
+    
+      <target name="dist" depends="clean">
+        <!-- prepare the build directory -->
+        <mkdir dir="${build.dir}/lib"/>
+        <mkdir dir="${build.dir}/main"/>
+        <!-- Build java code -->
+        <mkdir dir="${java-build.dir}"/>
+        <javac srcdir="${java.dir}" destdir="${java-build.dir}" classpathref="classpath"/>
+        <!-- Build main.jar -->
+        <jar destfile="${build.dir}/main/main.jar" basedir="${java-build.dir}">
+          <manifest>
+                <attribute name="Main-Class" value="Main" />
+          </manifest>
+        </jar>
+        <delete file="${java-build.dir}"/>
+        <!-- Add the python source -->
+        <copy todir="${build.dir}">
+          <fileset dir="${src.dir}"/>
+        </copy>
+        <!-- Add the libs -->
+        <copy todir="${build.dir}/lib">
+          <fileset dir="${lib.dir}"/>
+        </copy>
+        <!-- Compile OneJar -->
+        <javac srcdir="${onejar.dir}" destdir="${build.dir}" classpathref="classpath"/>
+        <!-- Copy the OneJar license file -->
+        <copy file="${onejar.dir}/one-jar-license.txt" tofile="${build.dir}/one-jar-license.txt" />
+        <!-- Build the jar -->
+        <jar destfile="${jar.name}" basedir="${build.dir}">
+          <manifest>
+                <attribute name="Main-Class" value="com.simontuffs.onejar.Boot" />
+                <attribute name="Class-Path" value="lib/jython-full.jar" />
+          </manifest>
+        </jar>
+        <!-- clean up -->
+        <delete dir="${java-build.dir}" />
+        <delete dir="${build.dir}" />
+      </target>
+    
+    </project>
+
+Because this is a Jython application, we can use as much Java source as we’d like.
+In this example, we will only use one Java source file *Main.java* to “drive” our application.
+In this case, we’ll use the *PythonInterpreter* inside of our *Main.java* to invoke our simple Jython Swing application.
+Now let’s take a look at the *Main.java* source.
+
+Listing 17-13. Main.java
+::
+    
+    import org.python.core.PyException;
+    import org.python.util.PythonInterpreter;
+    
+    public class Main {
+        public static void main(String[] args) throws PyException{
+            PythonInterpreter intrp = new PythonInterpreter();
+            intrp.exec("import JythonSimpleSwing as jy");
+            intrp.exec("jy.JythonSimpleSwing().start()");
+        }
+    }
+
+Now that we’ve written the driver class, we’ll place it into our *java* source directory.
+As stated previously, we’ll place our Jython code into the *src* directory.
+In this example we are using the same simple Jython Swing application that we wrote for Chapter 13.
+
+Listing 17-14. JythonSimpleSwing.py
+::
+    
+    import sys
+    
+    import javax.swing as swing
+    import java.awt as awt
+    
+    class JythonSimpleSwing(object):
+        def __init__(self):
+            self.frame=swing.JFrame(title="My Frame", size=(300,300))
+            self.frame.defaultCloseOperation=swing.JFrame.EXIT_ON_CLOSE;
+            self.frame.layout=awt.BorderLayout()
+            self.panel1=swing.JPanel(awt.BorderLayout())
+            self.panel2=swing.JPanel(awt.GridLayout(4,1))
+            self.panel2.preferredSize = awt.Dimension(10,100)
+            self.panel3=swing.JPanel(awt.BorderLayout())
+    
+            self.title=swing.JLabel("Text Rendering")
+            self.button1=swing.JButton("Print Text", actionPerformed=self.printMessage)
+            self.button2=swing.JButton("Clear Text", actionPerformed=self.clearMessage)
+            self.textField=swing.JTextField(30)
+            self.outputText=swing.JTextArea(4,15)
+    
+    
+            self.panel1.add(self.title)
+            self.panel2.add(self.textField)
+            self.panel2.add(self.button1)
+            self.panel2.add(self.button2)
+            self.panel3.add(self.outputText)
+    
+            self.frame.contentPane.add(self.panel1, awt.BorderLayout.PAGE_START)
+            self.frame.contentPane.add(self.panel2, awt.BorderLayout.CENTER)
+            self.frame.contentPane.add(self.panel3, awt.BorderLayout.PAGE_END)
+    
+        def start(self):
+            self.frame.visible=1
+    
+        def printMessage(self,event):
+            print "Print Text!"
+            self.text = self.textField.getText()
+            self.outputText.append(self.text)
+    
+        def clearMessage(self, event):
+            self.outputText.text = ""
+
+At this time, the application is ready to build using Ant.
+In order to run the build, simply traverse into the directory that contains *build.xml* and initiate the *ant* command.
+The resulting JAR can be run using the following syntax:
+
+java -jar JythonSwingApp.jar
+
+In some situations, such as deploying via web start, this JAR file will also need to be signed.
+There are many resources online that explain the signing of JAR files that topic will not be covered in this text.
+The JAR is now ready to be deployed and used on other machines.
+This method will be a good way to package an application for distribution via the Java Store.
+
+Mobile
+======
+
+Mobile applications are the way of the future.
+At this time, there are a couple of different options for developing mobile applications using Jython.
+One way to develop mobile applications using Jython is to make use of the JavaFX API from Jython.
+Since JavaFX is all Java behind the scenes, it would be fairly simple to make use of the JavaFX API using Jython code.
+However, this technique is not really a production-quality result in my opinion for a couple of reasons.
+First, the JavaFX scripting language makes GUI development quite easy.
+While it is possible (see http://wiki.python.org/jython/JythonMonthly/Articles/December2007/2 for more details), the translation of JavaFX API using Jython would not be as easy as making use of the JavaFX script language.
+The second reason this is not feasible at the time of this writing is that JavaFX is simply not available on all mobile devices at this time.
+It is really just becoming available to the mobile world at this time and will take some time to become acclimated.
+
+Another way to develop mobile applications using Jython is to make use of the Android operating system which is available via Google.
+Android is actively being used on mobile devices today, and its use is continuing to grow.
+Although in early stages, there is a project known as *Jythonroid* that is an implementation of Jython for the Android Dalvik Virtual Machine.
+Unfortunately, it was not under active development at the time of this writing, although some potential does exist for getting the project on track.
+
+If you are interested in mobile development using Jython, please pay close attention to the two technologies discussed in this section.
+They are the primary deployment targets for Jython in the mobile world.
+As for the *Jythonroid* project, it is open source and available to developers.
+Interested parties may begin working on it again to make it functional and bring it up to date with the latest Android SDK.
+
+Summary
+=======
+
+Deploying Jython applications is very much like Java application deployment.
+For those of you who are familiar with Java application servers, deploying a Jython application should be a piece of cake.
+On the contrary, for those of you who are not familiar with Java application deployment this topic may take a bit of getting used to.
+In the end, it is easy to deploy a Jython web or client application using just about any of the available Java application servers that are available today.
+
+Deploying Jython web applications is universally easy to do using the WAR file format.
+As long as *jython.jar* is either in the application server classpath or packaged along with the web application, Jython servlets should function without issue.
+We also learned that it is possible to deploy a JAR file containing a Jython GUI application via Java web start technology.
+Using a JNLP deployment file is quite easy to do; the trick to deploying Jython via a JAR file is to set the file up correctly.
+Once completed, an HTML page can be used to reference the JNLP and initiate the download of the JAR to the client machine.
+
+Lastly, this section discussed use of the Google App Engine for deploying Jython servlet applications.
+While the Google App Engine environment is still relatively new at the time of this writing, it is an exceptional deployment target for any Python or Java application developer.
+Using a few tricks with the object factory technique, it is possible to deploy Jython servlets and use them directly or via a JSP file on the App Engine.
+Stay tuned for more deployment targets to become available for Jython in the coming months and years.
+As cloud computing and mobile devices are becoming more popular, the number of deployment targets will continue to grow and Jython will be more useful with each one.

ExceptionHandlingDebug.rst

-Chapter 7:  Exception Handling and Debugging - *Final v1.0*
-+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+Chapter 7:  Exception Handling and Debugging
+++++++++++++++++++++++++++++++++++++++++++++
 
 Any good program makes use of a language’s exception handling mechanisms. There
 is no better way to frustrate an end-user then by having them run into an issue

GUIApplications.rst

+Chapter 16: GUI Applications
+++++++++++++++++++++++++++++
+
+The C implementation of Python comes with Tkinter for writing Graphical User Interfaces (GUIs).
+The GUI toolkit that you get automatically with Jython is Swing, which is included with the Java Platform by default.
+Similar to CPython, there are other toolkits available for writing GUIs in Jython.
+Because Swing is available on any modern Java installation, we will focus on the use of Swing GUIs in this chapter.
+
+Swing is a large subject, and can’t be fully covered in a single chapter.
+In fact, there are entire books devoted to Swing.
+We will provide an introduction to Swing, but only enough to describe the use of Swing from Jython.
+For in-depth coverage of Swing, one of the many books or web tutorials, like the Swing tutorial at java.sun.com/docs/books/tutorial/uiswing provided by Sun Microsystems, should be used.
+
+
+Using Swing from Jython has a number of advantages over the use of Swing in Java.
+For example, bean properties are less verbose in Jython, and binding actions in Jython is much less verbose (in Java you have to use anonymous classes, and in Jython you can pass a function).
+
+Let’s start with a simple Swing application in Java, and then we will look at the same application in Jython.
+See Listing 16-1.
+
+Listing 16-1.
+::
+    
+    import java.awt.event.ActionEvent;
+    import java.awt.event.ActionListener;
+    import javax.swing.JButton;
+    import javax.swing.JFrame;
+    
+    public class HelloWorld {
+    
+        public static void main(String[] args) {
+            JFrame frame = new JFrame("Hello Java!");
+            frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
+            frame.setSize(300, 300);
+            JButton button = new JButton("Click Me!");
+            button.addActionListener(
+                new ActionListener() {
+                    public void actionPerformed(ActionEvent event) {
+                        System.out.println("Clicked!");
+                    }
+                }
+            );
+            frame.add(button);
+            frame.setVisible(true);
+        }
+    }
+
+This simple application draws a JFrame that is completely filled with a JButton.
+When the button is pressed, “Click Me!” prints to the command line.
+See Figure 16-1.