Commits

ianb  committed 79e8e4e

Added a do-it-yourself framework doc for webob

  • Participants
  • Parent commits ed3a79b

Comments (0)

Files changed (3)

File docs/do-it-yourself.txt

+Another Do-It-Yourself Framework
+================================
+
+.. contents::
+
+Introduction and Audience
+-------------------------
+
+It's been over two years since I wrote the `first version of this tutorial <http://pythonpaste.org/do-it-yourself-framework.html>`_.  I decided to give it another run with some of the tools that have come about since then (particularly `WebOb <>`_).
+
+Sometimes Python is accused of having too many web frameworks.  And it's true, there are a lot.  That said, I think writing a framework is a useful exercise.  It doesn't let you skip over too much without understanding it.  It removes the magic.  So even if you go on to use another existing framework (which I'd probably advise you do), you'll be able to understand it better if you've written something like it on your own.
+
+This tutorial shows you how to create a web framework of your own, using WSGI and WebOb.  No other libraries will be used.
+
+What Is WSGI?
+-------------
+
+At its simplest WSGI is an interface between web servers and web applications.  We'll explain the mechanics of WSGI below, but a higher level view is to say that WSGI lets code pass around web requests in a fairly formal way.  That's the simplest summary, but there is more -- WSGI lets you add annotation to the request, and adds some more metadata to the request.
+
+WSGI more specifically is made up of an *application* and a *server*.  The application is a function that receives the request and produces the response.  The server is the thing that calls the application function.
+
+A very simple application looks like this:
+
+.. code-block::
+
+    >>> def application(environ, start_response):
+    ...     start_response('200 OK', [('Content-Type', 'text/html')])
+    ...     return ['Hello World!']
+
+The ``environ`` argument is a dictionary with values like the environment in a CGI request.  The header ``Host:``, for instance, goes in ``environ['HTTP_HOST']``.  The path is in ``environ['SCRIPT_NAME']`` (which is the path leading *up to* the application), and ``environ['PATH_INFO']`` (the remaining path that the application should interpret).
+
+We won't focus much on the server, but we will use WebOb to handle the application.  WebOb in a way has a simple server interface.  To use it you create a new request with ``req = webob.Request('http://localhost/test')``, and then call the application with ``resp = req.get_response(app)``.  For example:
+
+.. code-block::
+
+    >>> from webob import Request
+    >>> req = Request.blank('http://localhost/test')
+    >>> resp = req.get_response(application)
+    >>> print resp
+    200 OK
+    Content-Type: text/html
+    <BLANKLINE>
+    Hello World!
+
+This is an easy way to test applications, and we'll use it to test the framework we're creating.
+
+About WebOb
+-----------
+
+WebOb is a library to create a request and response object.  It's centered around the WSGI model.  Requests are wrappers around the environment.  For example:
+
+.. code-block::
+
+    >>> req = Request.blank('http://localhost/test')
+    >>> req.environ['HTTP_HOST']
+    'localhost:80'
+    >>> req.host
+    'localhost:80'
+    >>> req.path_info
+    '/test'
+
+Responses are objects that represent the... well, response.  The status, headers, and body:
+
+.. code-block::
+
+    >>> from webob import Response
+    >>> resp = Response(body='Hello World!')
+    >>> resp.content_type
+    'text/html'
+    >>> resp.content_type = 'text/plain'
+    >>> print resp
+    200 OK
+    Content-Length: 12
+    content-type: text/plain; charset=UTF-8
+    <BLANKLINE>
+    Hello World!
+
+Responses also happen to be WSGI applications.  That means you can call ``resp(environ, start_response)``.  Of course it's much less *dynamic* than a normal WSGI application.
+
+These two pieces solve a lot of the more tedious parts of making a framework.  They deal with parsing most HTTP headers, generating valid responses, and a number of unicode issues.
+
+Serving Your Application
+------------------------
+
+While we can test the application using WebOb, you might want to serve the application.  Here's the basic recipe, using the `Paste <>`_ HTTP server:
+
+.. code-block::
+
+    if __name__ == '__main__':
+        from paste import httpserver
+        httpserver.serve(app, host='127.0.0.1', port=8080)
+
+you could also use `wsgiref <>`_ from the standard library, but this is mostly appropriate for testing as it is single-threaded:
+
+.. code-block::
+
+    if __name__ == '__main__':
+        from wsgiref import simple_server
+        server = make_server('127.0.0.1', 8080, app)
+        server.serve_forever()
+
+Making A Framework
+------------------
+
+Well, now we need to start work on our framework.
+
+Here's the basic model we'll be creating:
+
+* We'll define routes that point to controllers
+
+* We'll create a simple framework for creating controllers
+
+Routing
+-------
+
+We'll use explicit routes using URI templates (minus the domains) to match paths.  We'll add a little extension that you can use ``{name:regular expression}``, where the named segment must then match that regular expression.  The matches will include a "controller" variable, which will be a string like "module_name:function_name".  For our examples we'll use a simple blog.
+
+So here's what a route would look like:
+
+.. code-block::
+
+    app = Router()
+    app.add_route('/', controller='controllers:index')
+    app.add_route('/{year:\d\d\d\d}/',
+                  controller='controllers:archive')
+    app.add_route('/{year:\d\d\d\d}/{month:\d\d}/',
+                  controller='controllers:archive')
+    app.add_route('/{year:\d\d\d\d}/{month:\d\d}/{slug}',
+                  controller='controllers:view')
+    app.add_route('/post', controller='controllers:post')
+
+To do this we'll need a couple pieces:
+
+* Something to match those URI template things.
+* Something to load the controller
+* The object to patch them together (``Router``)
+
+Routing: Templates
+~~~~~~~~~~~~~~~~~~
+
+To do the matching, we'll compile those templates to regular expressions.  We'll actually use a regular expression to match the variables.  We use ``re.VERBOSE`` so that the regular expression can span lines (whitespace is ignored) and we can add comments on the lines.
+
+Another thing most people don't know about is ``re.escape()``.  This escapes any special regular expression characters.  So ``re.escape('.*')`` becomes ``'\\.\\*'``.
+
+.. code-block::
+
+    >>> import re
+    >>> var_regex = re.compile(r'''
+    ...     \{          # The exact character "{"
+    ...     (\w+)       # The variable name (restricted to a-z, 0-9, _)
+    ...     (?::([^}]+))? # The optional :regex part
+    ...     \}          # The exact character "}"
+    ...     ''', re.VERBOSE)
+    >>> def template_to_regex(template):
+    ...     regex = ''
+    ...     last_match = None
+    ...     for match in var_regex.finditer(template):
+    ...         if last_match is None:
+    ...             regex += re.escape(template[:match.start()])
+    ...         else:
+    ...             regex += re.escape(template[last_match.end():match.start()])
+    ...         expr = match.group(2) or '[^/]+'
+    ...         expr = '(?P<%s>%s)' % (match.group(1), expr)
+    ...         regex += expr
+    ...         last_match = match
+    ...     if not last_match:
+    ...         # There were no {} groups
+    ...         regex = re.escape(template)
+    ...     else:
+    ...         regex += re.escape(template[match.end():])
+    ...     regex = '^%s$' % regex
+    ...     return regex
+
+To test it we can try some translations.  You could put these directly in the docstring of the ``template_to_regex`` function and use `doctest <http://python.org/doc/current/lib/module-doctest.html>`_ to test that.  But I'm using doctest to test *this* document, so I can't put a docstring doctest inside the doctest itself.  Anyway, here's what a test looks like:
+
+.. code-block::
+
+    >>> print template_to_regex('/a/static/path')
+    ^\/a\/static\/path$
+    >>> print template_to_regex('/{year:\d\d\d\d}/{month:\d\d}/{slug}')
+    ^\/(?P<year>\d\d\d\d)\/(?P<month>\d\d)\/(?P<slug>[^/]+)$
+
+Routing: controller loading
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To load controllers we have to import the module, then get the function out of it.  We'll use the ``__import__`` builtin to import the module.  The return value of ``__import__`` isn't very useful, but it puts the module into ``sys.modules``, a dictionary of all the loaded modules.
+
+Also, some people don't know how exactly the string method ``split`` works.  It takes two arguments -- the first is the character to split on, and the second is the maximum number of splits to do.  We want to split on just the first ``:`` character, so we'll use a maximum number of splits of 1.
+
+.. code-block::
+
+    >>> import sys
+    >>> def load_controller(string):
+    ...     module_name, func_name = string.split(':', 1)
+    ...     __import__(module_name)
+    ...     module = sys.modules[module_name]
+    ...     func = getattr(module, func_name)
+    ...     return func
+
+Routing: putting it together
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Now, the ``Router`` class.  The class has the ``add_route`` method, and also a ``__call__`` method.  That ``__call__`` method makes this object a WSGI application itself.  So the application looks at ``PATH_INFO`` (also known as ``req.path_info``).
+
+We put the variables matched from the regular expression, and any extra keyword arguments passed to ``add_route`` into ``req.urlvars``.  This is actually ``environ['wsgiorg.routing_args']``, a `standard location for this routing information <>`_.
+
+Lastly, if nothing matches at all, we want to return a 404 Not Found result.  WebOb comes with applications to return error results.  ``exc.HTTPNotFound()`` is one that returns a not found message.  We could include more information, but for now we'll be lazy and not give much information.
+
+Using ``return controller(environ, start_response)`` basically forwards the request on to that controller.  The controller is then a WSGI application of its own.
+
+.. code-block::
+
+    >>> from webob import Request
+    >>> from webob import exc
+    >>> class Router(object):
+    ...     def __init__(self):
+    ...         self.routes = []
+    ...
+    ...     def add_route(self, template, controller, **vars):
+    ...         if isinstance(controller, basestring):
+    ...             controller = load_controller(controller)
+    ...         self.routes.append((re.compile(template_to_regex(template)),
+    ...                             controller,
+    ...                             vars))
+    ...
+    ...     def __call__(self, environ, start_response):
+    ...         req = Request(environ)
+    ...         for regex, controller, vars in self.routes:
+    ...             match = regex.match(req.path_info)
+    ...             if match:
+    ...                 req.urlvars = match.groupdict()
+    ...                 req.urlvars.update(vars)
+    ...                 return controller(environ, start_response)
+    ...         return exc.HTTPNotFound()(environ, start_response)
+
+Controllers
+-----------
+
+Controllers are themselves just WSGI applications, but we'll want to set up something to make them friendlier to write.
+
+To do that we'll write a `decorator <>`_.  A decorator is a function that wraps another function.  The decorated function will be a WSGI application, but it will call a function with a signature like ``controller_func(req, **urlvars)``.  The controller function will return a response object (which, remember, is a WSGI application on its own).
+
+If you remember ``webob.exc.HTTPNotFound``, those objects can also be used as exceptions.  Unless you are willing to restrict yourself to Python 2.5+, you'll have to do ``raise exc.HTTPNotFound().exception`` because the actual objects like ``HTTPNotFound`` don't work as exceptions.
+
+We'll also let the function return just a string, and in that case create a standard response object (200 OK, with a Content-Type of text/html).
+
+.. code-block::
+
+    >>> from webob import Request, Response
+    >>> from webob import exc
+    >>> def controller(func):
+    ...     def replacement(environ, start_response):
+    ...         req = Request(environ)
+    ...         try:
+    ...             resp = func(req, **req.urlvars)
+    ...         except exc.HTTPException, e:
+    ...             resp = e
+    ...         if isinstance(resp, basestring):
+    ...             resp = Response(body=resp)
+    ...         return resp(environ, start_response)
+    ...     return replacement
+
+You use this controller like:
+
+.. code-block::
+
+    >>> @controller
+    ... def index(req):
+    ...     return 'This is the index'
+
+Putting It Together
+-------------------
+
+Now we'll show a basic application.  Just a hello world application for now.  Note that this document is the module ``__main__``.
+
+.. code-block::
+
+    >>> @controller
+    ... def hello(req):
+    ...     if req.method == 'POST':
+    ...         return 'Hello %s!' % req.params['name']
+    ...     elif req.method == 'GET':
+    ...         return '''<form method="POST">
+    ...             You're name: <input type="text" name="name">
+    ...             <input type="submit">
+    ...             </form>'''
+    >>> hello_world = Router()
+    >>> hello_world.add_route('/', controller=hello)
+
+Now let's test that application:
+
+.. code-block::
+
+    >>> req = Request.blank('/')
+    >>> resp = req.get_response(hello_world)
+    >>> print resp
+    200 OK
+    content-type: text/html; charset=UTF-8
+    Content-Length: 131
+    <BLANKLINE>
+    <form method="POST">
+                You're name: <input type="text" name="name">
+                <input type="submit">
+                </form>
+    >>> req.method = 'POST'
+    >>> req.body = 'name=Ian'
+    >>> resp = req.get_response(hello_world)
+    >>> print resp
+    200 OK
+    content-type: text/html; charset=UTF-8
+    Content-Length: 10
+    <BLANKLINE>
+    Hello Ian!
+
+    
+Another Controller
+------------------
+
+There's another pattern that might be interesting to try for a controller.  Instead of a function, we can make a class with methods like ``get``, ``post``, etc.  The ``urlvars`` will be used to instantiate the class.
+
+We could do this as a superclass, but the implementation will be more elegant as a wrapper, like the decorator is a wrapper.  Python 3.0 will add `class decorators <>`_ which will work like this.
+
+We'll allow an extra ``action`` variable, which will define the method (actually ``action_method``, where ``_method`` is the request method).  If no action is given, we'll use just the method (i.e., ``get``, ``post``, etc).
+
+.. code-block::
+
+    >>> def rest_controller(cls):
+    ...     def replacement(environ, start_response):
+    ...         req = Request(environ)
+    ...         try:
+    ...             instance = cls(req, **req.urlvars)
+    ...             action = req.urlvars.get('action')
+    ...             if action:
+    ...                 action += '_' + req.method.lower()
+    ...             else:
+    ...                 action = req.method.lower()
+    ...             try:
+    ...                 method = getattr(instance, action)
+    ...             except AttributeError:
+    ...                 raise exc.HTTPNotFound("No action %s" % action)
+    ...             resp = method()
+    ...             if isinstance(resp, basestring):
+    ...                 resp = Response(body=resp)
+    ...         except exc.HTTPException, e:
+    ...             resp = e
+    ...         return resp(environ, start_response)
+    ...     return replacement
+
+Here's the hello world:
+
+.. code-block::
+
+    >>> class Hello(object):
+    ...     def __init__(self, req):
+    ...         self.request = req
+    ...     def get(self):
+    ...         return '''<form method="POST">
+    ...             You're name: <input type="text" name="name">
+    ...             <input type="submit">
+    ...             </form>'''
+    ...     def post(self):
+    ...         return 'Hello %s!' % self.request.params['name']
+    >>> hello = rest_controller(Hello)
+
+We'll run the same test as before:
+
+.. code-block::
+
+    >>> hello_world = Router()
+    >>> hello_world.add_route('/', controller=hello)
+    >>> req = Request.blank('/')
+    >>> resp = req.get_response(hello_world)
+    >>> print resp
+    200 OK
+    content-type: text/html; charset=UTF-8
+    Content-Length: 131
+    <BLANKLINE>
+    <form method="POST">
+                You're name: <input type="text" name="name">
+                <input type="submit">
+                </form>
+    >>> req.method = 'POST'
+    >>> req.body = 'name=Ian'
+    >>> resp = req.get_response(hello_world)
+    >>> print resp
+    200 OK
+    content-type: text/html; charset=UTF-8
+    Content-Length: 10
+    <BLANKLINE>
+    Hello Ian!
+
+Conclusion
+----------
+
+Well, that's a framework.  Ta-da!
+
+Of course, this doesn't deal with some other stuff.  In particular:
+
+* Configuration
+* Templates
+* Making your routes debuggable
+* Exception catching and other basic infrastructure
+* Database connections
+* Form handling
+* Authentication
+
+But, for now, that's outside the scope of this document.
+

File docs/index.txt

 
 * `JSON-RPC server and client example <jsonrpc-example.html>`_
 
+* `A Do-It-Yourself Framework <do-it-yourself.html>`_
+
 .. contents::
 
 .. comment:
 docs = docs/index.txt docs/license.txt docs/differences.txt docs/file-example.txt
        docs/news.txt docs/reference.txt docs/comment-example.txt
        docs/wiki-example.txt docs/jsonrpc-example.txt
+       docs/do-it-yourself.txt
 title = WebOb
 modules = webob
           webob.acceptparse