Commits

David Cramer committed 727fc62

Basic packaging so it can be installed via PIP

  • Participants
  • Parent commits 8e3d433

Comments (0)

Files changed (41)

LICENSE.txt

-Copyright 2009 Jonas Obrist. All rights reserved.
-
-Redistribution and use in source and binary forms, with or without modification, are
-permitted provided that the following conditions are met:
-
-   1. Redistributions of source code must retain the above copyright notice, this list of
-      conditions and the following disclaimer.
-
-   2. Redistributions in binary form must reproduce the above copyright notice, this list
-      of conditions and the following disclaimer in the documentation and/or other materials
-      provided with the distribution.
-
-THIS SOFTWARE IS PROVIDED BY JONAS OBRIST ``AS IS'' AND ANY EXPRESS OR IMPLIED
-WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
-FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL JONAS OBRIST OR
-CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
-CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
-SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
-ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
-NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
-ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-The views and conclusions contained in the software and documentation are those of the
-authors and should not be interpreted as representing official policies, either expressed
-or implied, of Jonas Obrist.

README

-################################################################################
-#
-# What is this?
-#
-################################################################################
-
-django-bbcode is a django application which was written to render BBCode syntax
-to HTML using python and django. However it does not restrict you to BBCode like
-syntax, you can use it to parse anything you like into anything you like.
-
-################################################################################
-#
-# Quickstart
-#
-################################################################################
-
-For those in a hurry:
-
-1.) Put django-bbcode into your python path into a folder called 'bbcode'.
-2.) Add 'bbcode' to your INSTALLED_APPS in django
-3.) Optional: Add/edit bbcode tags
-4.) Put {% load bbcode %} into the template where you want to render stuff.
-5.) Put {% bbcode varname %} where you want the stuff, here called 'varname', to
-    be rendered.
-6.) Done.
-
-################################################################################
-#
-# Slowstart
-#
-################################################################################
-
-For those who want to do it right:
-
-1.) Put django-bbcode into your python path into a folder called 'bbcode'.
-2.) Add 'bbcode' to your INSTALLED_APPS in django
-3.) Make sure all bbcode you have is valid. This can be done by using 
-    bbcode.validate(content, namespaces) (namespaces is optional) with the
-    content you want to save. This will either return None if no errors were
-    found or a list of errors that occured.
-4.) Put {% load bbcode %} into the template where you want to render stuff.
-5.) Put {% bbcode varname namespace1 "namespace2" %} where you want the stuff,
-    here called 'varname', to be rendered. Namespaces are optional. 'namespace1'
-    is a template variable holding either a string or a list (or tuple) of
-    strings. Those namespaces are used to filter the tags which are used to
-    render the content. You can also give it hardcoded strings like
-    '"namespace2"'. To exclude a namespace you can prefix the name with a 'no-'.
-    For example: "no-mynamespace". For more information on namespaces look
-    below.
-6.) Done.
-
-################################################################################
-#
-# What are those so called 'namespaces'?
-#
-################################################################################
-
- - "Namespaces are one honking great idea -- let's do more of those!" - PEP0020
-
-I totally agree with that quote from the Zen of Python, thus I included
-namespaces to django-bbcode. Namespaces are used to dynamcially filter what tags
-are used to render a context. The main reason I introduced this feature was to
-turn off smilies in my project for users who disable smilies (like me, I hate
-those silly graphics). In this case I'd just use the namespace "no-smilies".
-
-By default every tag gets 3 namespaces: "__all__" (which holds all tags), the
-django app name (for all builtin tags: "bbcode") and the name of the module the
-tag is defined in (for smilies: "smilies",...). Additionaly you can add
-namespaces to tags by giving their class an attribute 'namespace' which holds a
-list of strings. Those namespaces will be used additionally to the default ones.
-
-You can then use those namespaces in the bbcode template tag as additional 
-arguments after 'content'. It allows you to give template variables holding 
-lists or tuples of strings or a single string. You can also put hardcoded 
-namespaces there (wrapped in single or double quotes).
-
-################################################################################
-#
-# How do I write a custom tag?
-#
-################################################################################
-
-Basically where you wanna start is by reading the examples (builtin tags). A tag
-is created by subclassing the TagNode class (or some of the specialized base 
-classes such as ReplaceTagNode, ArgumentTagNode, MultiArgumentTagNode,....).
-
-Every tag class needs a couple of attributes:
-
-open_pattern: a compiled regular expression which matches the opening tag.
-close_pattern: a compiled regular expression which matches the ending tag.
-parse: a function which returns a string.
-
-For each match of open_pattern/close_pattern pairs an instance of the class will
-be created. Each class gets the parent node, the regular expression match object
-and the full content as arguments.
-Also each instance of a tag class as an attribute called 'nodes' which is the
-list of child-tags (tags nested within this tag).
-
-################################################################################
-#
-# What are soft exceptions?
-#
-################################################################################
-
-Python exceptions stop the flow of code, thus I wrote the SoftExceptionManager
-(SEM). If a tag was used wrong, this will not stop the parser, but create an
-entry in the SEM. This way if several independent errors occur you will get them
-at once instead of one after another. If you use bbcode.validate (which you 
-really should!) and the content fails to parse correctly, you will get all 
-errors from the SEM. The bbcode.parse function also returns a tuple with the
-parsed content and the list of errors as items.
-
-Each error in the SEM is a tuple of (line_number, error_text).

__init__.py

-"""
-BB Code parser by Jonas 'Ojii' Obrist (c) 2009
-
-USAGE:
-
-Parsing:
-
-parsed, errors = bbcode.parse(content, strict=True)
-
-This might raise a bbocde.PaserError if strict is True (default). Otherwise on a
-ParserError the content is returned unparsed and errors contains the reason.
-
-Validation:
-
-errors = bbcode.validate(content)
-
-Returns errors caused by parsing the code or an empty sequence.
-
-Extending:
-
-Subclassing bbcode.TagNode and bbcode.register the class adds new BB Code Tags.
-Each node must have an opening and closing pattern (open_pattern, close_pattern)
-and push, pushed, pull and close methods. For further information read the doc
-strings of the TagNode class.
-"""
-import re
-import cgi
-
-try:
-    from django.utils.translation import ugettext as _
-except ImportError:
-    _ = lambda x: x
-
-AUTODISCOVERED = False
-
-LINEFEED_PATTERN = re.compile('\n\s*\n', re.MULTILINE)
-def convert_linefeeds(content):
-    content = LINEFEED_PATTERN.sub('<br /><br />', content)
-    return content.replace('\n', '<br />')
-
-
-class UnmatchablePseudoPattern(object):
-    """
-    A class which should look like a compiled regular expression but never match.
-    """
-    def match(self, content):
-        return False
-    
-    def search(self, content):
-        return False
-    
-    def finditer(self, content):
-        return iter([])
-    
-    def sub(self, replacement, content):
-        return content
-
-
-class patterns:
-    """
-    This is a class for namespacing reasons
-    """
-    no_argument = r'\[%s\]'
-    self_closing_tag = r'\[%s\s*/\]'
-    single_argument = r'\[%s(\]|="?(?P<argument>[^\]]+)"?\])'
-    argument = r'( (\w+)=([^\] ]+))?'
-    closing = r'\[/%s\]'
-    unmatchable = UnmatchablePseudoPattern()
-     
-def get_tag_name(klass):
-    """
-    Convert a class to tagname
-    """
-    return klass.tagname if hasattr(klass, 'tagname') else klass.__name__.lower()
-
-class NeedsSubclassingError(Exception): pass
-class ParserError(Exception): pass
-
-
-class SoftException(object):
-    def __init__(self, lineno, message):
-        self.lineno = lineno
-        self.message = message
-        
-    def __str__(self):
-        return '<span class="bbcode-error lineno">Line %s:</span> <span class="bbcode-error message">%s</span>' % (self.lineno, self.message)
-    __unicode__ = __str__
-
-
-class SoftExceptionManager(object):
-    """
-    Allows 'soft exceptions'. Soft exceptions are exceptions which don't break
-    the flow of the code but are rather stored in a list and can then be told
-    given to the user.
-    """
-    def __init__(self):
-        self.exceptions = []
-        self.line_number = 1
-        
-    def set_line_number(self, number):
-        """
-        Update the line number
-        """
-        self.line_number = number
-        
-    def soft_raise(self, exception):
-        """
-        Soft raise an exception. Stores the line number the exception occured
-        and the exception message. If deployed in django it will make the 
-        message i18n ready.
-        """
-        self.exceptions.append(SoftException(self.line_number, _(exception)))
-        
-    def pull(self):
-        """
-        Pulls all exception since initialization or last pull. Resets exception
-        list.
-        """
-        old = self.exceptions
-        self.exceptions = []
-        return old
-
-sem = SoftExceptionManager()
-soft_raise = sem.soft_raise
-
-class VariableScope(dict):
-    def add(self, name, value):
-        dict.__setitem__(self, str(name), str(value))
-        
-    def resolve(self, context):
-        context = context.strip('"')
-        for var, value in dict.iteritems(self):
-            context = context.replace('$%s$' % var, value)
-        return context
-    
-    def lazy_resolve(self, context):
-        class Lazy:
-            def __init__(self, resolver, context):
-                self.resolver = resolver
-                self.context = context
-                
-            def __int__(self):
-                self.context = self.resolver(self.context)
-                return int(self.context)
-                
-            def __getattr__(self, attr):
-                self.context = self.resolver(self.context)
-                return self.context.__getattribute__(attr)
-        return Lazy(self.resolve, context)
-
-
-class Node(object):
-    """
-    This is the baseclass for all objects in a BBCode Parse Tree.
-    To understand Nodes it is important to understand the Tree.
-    Each Parse Tree has one, and only one, head node. This node has child nodes
-    and those children have child nodes themselves. This continues until there
-    are no more child nodes. In a standard Parse Tree the last leaves of a 
-    branch are instances of TextNode, however since empty TextNodes are not kept
-    in the Tree, they might also be missing.
-    
-    When the Parse Tree is generated the nodes get 'pushed', 'appended', 'pulled'
-    and 'closed'. Only TextNodes can be appended to a node's nodelist. When a
-    new child node is found it is 'pushed' and becomes the current node. When a
-    node cannot be closed correctly it is 'pulled', which means it's unparsed
-    contents are added to it's parent. Usually this causes a ParserError, which
-    means the Tree is not parseable. When a node is finished parsing it's
-    'closed' which normally returns the parent.
-    """
-    name = 'node'
-    
-    is_text_node = False
-    
-    def __init__(self, parent, match, fullcontent, context=None):
-        """
-        Normal nodes take their parent node as first argument, the regular
-        expression match as second argument and the full context as third
-        argument.
-        """
-        self.start = match.start()
-        self.fullcontent = fullcontent
-        self.raw_content = ''
-        self.parent = parent
-        self.match = match
-        self.nodes = []
-        self.context = context # for django only
-        # copy the variable scope
-        self.variables = parent.variables
-        
-    def soft_raise(self, errmsg):
-        soft_raise(errmsg)
-        return self.raw_content
-    
-    def append(self, text):
-        """
-        Adds a text node to the node
-        """
-        self.nodes.append(TextNode(self, text))
-    
-    def push(self, nodeklass, match, fullcontent):
-        """
-        Adds a nested tag node and returns that node
-        """
-        node = nodeklass(self, match, fullcontent, self.context)
-        self.nodes.append(node)
-        return node.pushed()
-    
-    def pushed(self):
-        """
-        Normal Nodes return themselves when being pushed. Self closing nodes
-        can overwrite this method to handle this in another fashion.
-        """
-        return self
-    
-    def pull(self, end):
-        """
-        Pulls all text nodes and returns the parent
-        """
-        self.parent.nodes.append(TextNode(self.fullcontent[self.start:end]))
-        return self.parent
-    
-    def close(self, end):
-        """
-        When closing the node just return the parent.
-        """
-        self.end = end
-        self.raw_content = self.fullcontent[self.start:end]
-        return self.parent
-    
-    def parse(self):
-        """
-        Parses the node. This is also responsible to parse child nodes. Should
-        return a string and fail silently.
-        """
-        raise NeedsSubclassingError
-        
-
-class HeadNode(Node):
-    """
-    The head node of the BBCode parse tree.
-    """
-    name = 'head'
-    def __init__(self, raw_content, context=None):
-        self.raw_content = raw_content
-        self.nodes = []
-        self.context = context
-        self.variables = VariableScope()
-    
-    def pull(self, end):
-        raise ParserError, "Cannot pull from headnode, invalid BBCode Tree"
-    
-    def close(self, end):
-        raise ParserError, "Cannot close headnode, invalid BBCode Tree"
-    
-    def parse(self):
-        content = ''
-        failed  = []
-        for node in self.nodes:
-            content += node.parse()
-        return content
-    
-    
-class TextNode(Node):
-    smilie_pattern = re.compile(':(?P<name>\w+):')
-    is_text_node = True
-    def __init__(self, parent, text):
-        self.text = text
-        self.variables = parent.variables
-        self.parent = parent
-        self.raw_content = text
-        self.nodes = []
-        
-    def append(self, text):
-        raise TypeError, "TextNode does not support appending"
-    
-    def push(self, node):
-        raise TypeError, "TextNode does not support pushing"
-    
-    def pull(self, end):
-        raise TypeError, "TextNode does not support pulling"
-    
-    def close(self, end):
-        raise TypeError, "TextNode does not support closing"
-    
-    def __repr__(self):
-        return '<TextNode instance "%s">' % self.text
-    
-    def parse(self):
-        """
-        Return cgi-escaped content
-        """
-        return cgi.escape(self.variables.resolve(self.text))
-    
-    def __str__(self):
-        return 'TextNode: %r' % self.text
-        
-    
-class TagNode(Node):
-    @staticmethod
-    def open_pattern():
-        raise NeedsSubclassingError
-    @staticmethod
-    def close_pattern():
-        raise NeedsSubclassingError
-    
-    def parse_inner(self):
-        """
-        Shortcut for parsing all inner nodes and return their combined contents.
-        """
-        inner = ''
-        for node in self.nodes:
-            inner += node.parse()
-        return inner
-    
-    def __str__(self):
-        return self.__class__.__name__
-    
-    
-class ReplaceTagNode(TagNode):
-    """
-    A specialized TagNode subclass with a predefined parse method. It allows
-    easy creation of simple bbcode - html replacement tags. [tag] becomes <tag>
-    and [/tag] becomes </tag>. These tags do not take any arguments and parse
-    all inner content.
-    Requires an explicit 'tagname' attribute, otherwise the lowered class name
-    will be used as tagname
-    """
-    def __init__(self, parent, match, content, context):
-        """
-        Implicitly set tag name if not available.
-        """
-        if not hasattr(self, 'tagname'):
-            self.tagname = self.__class__.__name__.lower()
-        TagNode.__init__(self, parent, match, content, context)
-        
-    def parse(self):
-        return '<%s>%s</%s>' % (self.tagname, self.parse_inner(), self.tagname)
-    
-    def __str__(self):
-        return 'ReplaceTagNode: %s' % self.__class__.__name__
-    
-    
-class ArgumentTagNode(TagNode):
-    """
-    TagNode which takes one (or no) argument. Open pattern must have a named
-    group 'argument'.
-    """
-    def __init__(self, parent, match, content, context):
-        TagNode.__init__(self, parent, match, content, context)
-        arg = match.group('argument')
-        self.argument = self.variables.lazy_resolve(arg.strip('"') if arg else '')
-        
-    def __str__(self):
-        return '%s (%s)' % (self.__class__.__name__, self.argument)
-
-
-class _MultiArgs(dict):
-    """
-    Dictionary-like class which allows items to be accessed via attributes.
-    """
-    def __getattr__(self, attr):
-        return dict.__getitem__(self, attr)
-        
-        
-class MultiArgumentTagNode(TagNode):
-    """
-    TagNode which takes multiple (or no) arguments. Must have an attribute
-    _arguments which holds key, value pairs of the arguments and their defaults.
-    Open pattern should use bbcode.patterns.argument as argument matching
-    expression.
-    """
-    _arguments   = []
-    def __init__(self, parent, match, content, context):
-        TagNode.__init__(self, parent, match, content, context)
-        args = match.groups()
-        kwargs = dict(self._arguments)
-        for index, value in enumerate(filter(bool, args)):
-            if not index or not index % 3:
-                continue
-            if not (index + 1) % 3:
-                kwargs[args[index - 1]] = self.variables.lazy_resolve(value)
-        self.arguments = _MultiArgs(kwargs)
-        
-    def __str__(self):
-        args = []
-        for key, value in self.arguments.iteritems():
-            args.append('%s: %s' % (key, value))
-        return '%s (%s)' % (self.__class__.__name__, ', '.join(args))
-        
-        
-class SelfClosingTagNode(TagNode):
-    """
-    A tag which is self closed.
-    """
-    close_pattern = patterns.unmatchable
-    
-    def __init__(self, parent, match, content, context):
-        self.start = match.start()
-        self.context = context
-        self.fullcontent = content
-        self.raw_content = content[match.start():match.end()]
-        self.parent = parent
-        self.match = match
-        self.nodes = []
-        self.variables = parent.variables
-    
-    def pushed(self):
-        """
-        A self closing node returns it's parent. Thus it will never have child
-        nodes!
-        """
-        return self.parent
-    
-    def __str__(self):
-        return 'SelfClosingTag: %s' % self.__class__.__name__
-    
-    
-class AutoDict(dict):
-    def __init__(self, default_thing=set, *args, **kwargs):
-        self.__default_thing = default_thing
-        dict.__init__(self, *args, **kwargs)
-
-    def __getitem__(self, item):
-        if not dict.__contains__(self, item):
-            dict.__setitem__(self, item, self.__default_thing() if callable(self.__default_thing) else self.__default_thing)
-        return dict.__getitem__(self, item)
-    
-    
-class Library(object):
-    """
-    The core of the BBCode parser. Keeps track of all bbcode tags and text
-    parsers. Also handles building BBCode Parse Trees and the automated help
-    generation.
-    """
-    name_pat1 = re.compile('([a-z0-9])([A-Z])')
-    name_pat2 = re.compile('(.)([A-Z][a-z]+)')
-    
-    def __init__(self):
-        self.names = AutoDict(None)
-        self.raw_names = {}
-        self.tags = AutoDict(set)
-        self.klasses = AutoDict(None)
-    
-    def convert(self, name):
-        """
-        Convert a class name to something a bit more readable
-        """
-        return  self.name_pat1.sub(r'\1 \2', self.name_pat2.sub(r'\1 \2', name))
-    
-    def dsparse(self, docs):
-        """
-        Parse docstrings
-        """
-        content, errors = parse(docs, strict=False, auto_discover=True)
-        return content
-    
-    def get_default_namespaces(self, klass):
-        bits = klass.__module__.split('.')
-        return (bits[-1], bits[-3], klass.__name__.lower())
-        
-    def register(self, klass):
-        """
-        Register a BBCode Tag Node
-        """
-        # Add the class to their namespaces.
-        if hasattr(klass, 'namespaces'):
-            for ns in klass.namespaces:
-                self.tags[ns].add(klass)
-                if not hasattr(klass, 'not_in_all') or not klass.not_in_all:
-                    self.tags['__all__'].add(klass)
-        elif not hasattr(klass, 'not_in_all') or not klass.not_in_all:
-            self.tags['__all__'].add(klass)
-        if not hasattr(klass, 'namespaces'):
-            setattr(klass, 'namespaces', [])
-        d_namespaces = self.get_default_namespaces(klass)
-        for default in d_namespaces:
-            self.tags[default].add(klass)
-        for ns in reversed(d_namespaces):
-            klass.namespaces.insert(0, ns)
-        # Register documentation
-        docstrings = klass.__doc__
-        if hasattr(klass, 'tagname'):
-            tagname = klass.tagname
-        else:
-            tagname = klass.__name__.lower()
-        if docstrings:
-            if hasattr(klass, 'verbose_name'):
-                verbose_name = klass.verbose_name
-            else:
-                verbose_name = self.convert(klass.__name__)
-            self.names[tagname] = {'docs': docstrings.strip(),
-                                   'name': verbose_name,
-                                   'class': klass}
-            self.klasses[klass] = self.names[tagname]
-        self.raw_names[klass.__name__] = klass
-        
-    def add_namespace(self, klass, *namespaces):
-        """
-        Add a tag to a namespace or several namespaces
-        """
-        if isinstance(klass, TagNode):
-            for namespace in namespaces:
-                self.tags[namespace].add(klass)
-        elif isinstance(klass, basestring):
-            if klass in self.raw_names:
-                self.add_namespace(self.raw_names[klass], *namespaces)
-            elif klass in self.names:
-                self.add_namespace(self.names[klass]['class'], *namespaces)
-                
-    def remove_namespace(self, klass, *namespaces):
-        """
-        Remove a tag from a namespace or several namespaces
-        """
-        if isinstance(klass, TagNode):
-            for namespace in namespaces:
-                if klass in self.tags[namespace]:
-                    self.tags[namespace].remove(klass)
-        elif isinstance(klass, basestring):
-            if klass in self.raw_names:
-                self.add_namespace(self.raw_names[klass], *namespaces)
-            elif klass in remove_namespace.names:
-                self.remove_namespace(self.names[klass]['class'], *namespaces)
-                
-    def set_not_in_all(self, klass, flag=True):
-        """
-        Set 'not_in_all' for a tag.
-        """
-        if flag:
-            self.remove_namespace(klass, '__all__')
-        else:
-            self.add_namespace(klass, '__all__')
-            
-    def get_help(self, *tags):
-        """
-        Get help for a tag or for all tags.
-        
-        Returns a dictionary with keys 'name', 'tag', 'docstring'.
-        """
-        if not tags:
-            tags = self.get_tags()
-        help_objects = []
-        for tag in tags:
-            if issubclass(tag, Node):
-                obj = self.klasses[tag]
-                if obj is None:
-                    continue
-            else:
-                obj = self.names[tag]
-                if obj is None:
-                    continue
-            help_objects.append({'name': obj['name'],
-                                 'docstring': parse(obj['docs'], strict=False, auto_discover=True)[0],
-                                 'obj': obj['class']})
-        return help_objects
-    
-    def get_tags(self, namespaces=None):
-        """
-        Get a list of tag classes for the namespaces
-        """
-        if namespaces is None:
-            namespaces = get_default_namespaces()
-        tags = set()
-        exclude = []
-        include = []
-        # Split the 'namespaces' into exclude and include namespaces
-        for ns in namespaces:
-            if ns.startswith('no-'):
-                _ns = ns[3:]
-                if _ns in self.tags:
-                    exclude.append(_ns)
-            elif ns in self.tags:
-                include.append(ns)
-        # Include first
-        if not include or '__all__' in include:
-            tags = set(self.tags['__all__'])
-        else:
-            if 'base' in include:
-                tags = set(self.tags['__all__'])
-            for ns in include:
-                tags = tags.union(self.tags[ns])
-        # Then exclude
-        for ns in exclude:
-            tags = tags.difference(self.tags[ns])
-        return tags
-    
-    def get_taglist(self, content, namespaces=None):
-        """
-        Get the tag-match list of a content for given namespaces
-        """
-        if namespaces is None:
-            namespaces = get_default_namespaces()
-        tags = self.get_tags(namespaces)
-        # Build tag list
-        taglist = []
-        for tagklass in tags:
-            op = tagklass.open_pattern
-            if callable(op):
-                op = op()
-            i = 1
-            for match in op.finditer(content):
-                i += 1
-                taglist.append((match.start(), match, tagklass, True))
-            cp = tagklass.close_pattern
-            if callable(cp):
-                cp = cp()
-            for match in cp.finditer(content):
-                taglist.append((match.start(), match, tagklass, False))
-        # Sort by position
-        return sorted(taglist)
-    
-    def get_parse_tree(self, content, namespaces=None, context=None):
-        """
-        Prepare content for parsing.
-        Returns a HeadNode instance
-        """
-        if namespaces is None:
-            namespaces = get_default_namespaces()
-        taglist = self.get_taglist(content, namespaces)
-        
-        # Get headnode
-        headnode = HeadNode(content, context)
-        
-        lastpos = 0
-        currentnode = headnode
-        # Loop over tag matches
-        for pos, match, tagklass, opener in taglist:
-            start, end = match.span()
-            # Prevent tags matching within other tags (eg AutoDetectURL)
-            if start < lastpos:
-                continue
-            # Append text between last tag and this one
-            text = content[lastpos:start]
-            if text:
-                currentnode.append(text)
-            # Set new position
-            lastpos = end
-            # Get line number for soft exceptions
-            lineno = content[:start].count('\n') + 1
-            sem.set_line_number(lineno)
-            # if opener, push new node
-            if opener:
-                currentnode = currentnode.push(tagklass, match, content)
-            # else close the tag
-            else:
-                # pull all unclosed child tags of the current node
-                while tagklass != currentnode.__class__:
-                    try:
-                        currentnode = currentnode.pull(end)
-                    except ParserError:
-                        sem.soft_raise("BBCode could not be parsed. There are probably unclosed or uneven tags!")
-                        raise ParserError, "Failed to find matching opening tag for closing tag '%s' in line %s."  % (get_tag_name(tagklass), lineno)
-                # close the node
-                currentnode = currentnode.close(end)
-        text = content[lastpos:]
-        if text:
-            headnode.append(text)
-        # Return the head node
-        return headnode
-    
-    def get_visual_parse_tree(self, content, namespaces=None, indent=4):
-        if namespaces is None:
-            namespaces = get_default_namespaces()
-        def recurse(nodes, level, indent):
-            cindent = level * indent
-            sindent = ' ' * cindent
-            next = level + 1
-            l = []
-            for node in nodes:
-                l.append('%s-%s' % (sindent, str(node)))
-                l += recurse(node.nodes, next, indent)
-            return l
-        try:
-            head = self.get_parse_tree(content, namespaces)
-        except ParserError:
-            return '-Parse Error'
-        visuals = ['-HeadNode']
-        visuals += recurse(head.nodes, 1, indent)
-        return '\n'.join(visuals)
-    
-    def validate(self, content, namespaces=None, auto_discover=False):
-        """
-        Validates a given content and returns the errors or an empty sequence.
-        """
-        if namespaces is None:
-            namespaces = get_default_namespaces()
-        if auto_discover:
-            autodiscover()
-        try:
-            headnode = self.get_parse_tree(content, namespaces)
-        except ParserError:
-            return sem.pull()
-        parsed = headnode.parse()
-        return sem.pull()
-
-
-lib = Library()
-register = lib.register
-validate = lib.validate
-get_help = lib.get_help
-get_visual = lib.get_visual_parse_tree
-
-def get_default_namespaces():
-    from django.conf import settings
-    if hasattr(settings, 'BBCODE_DEFAULT_NAMESPACES'):
-        return settings.BBCODE_DEFAULT_NAMESPACES
-    return ['__all__']
-    
-def parse(content, namespaces=None, strict=True, auto_discover=False,
-          context=None):
-    """
-    Parse a content with the BBCodes
-    """
-    if auto_discover:
-        autodiscover()
-    if namespaces is None:
-        namespaces = get_default_namespaces()
-    # Fix windows linefeeds
-    content = content.replace('\r','')
-    # Get head node
-    if strict:
-        head = lib.get_parse_tree(content, namespaces, context)
-    else:
-        try:
-            head = lib.get_parse_tree(content, namespaces, context)
-        except ParserError:
-            return convert_linefeeds(content), sem.pull()
-    # parse BB Codes
-    content = head.parse()
-    # Replace linefeeds
-    content = convert_linefeeds(content)
-    return content, sem.pull()
-    
-def autodiscover():
-    """
-    Automatically register all bbcode tags. This searches the 'bbtags' modules
-    of all INSTALLED_APPS if available.
-    """
-    global AUTODISCOVERED
-    if AUTODISCOVERED:
-        return
-    import imp
-    from django.conf import settings
-    import os
-
-    for app in settings.INSTALLED_APPS:
-        try:
-            module = __import__(app, {}, {}, [app.split('.')[-1]])
-            app_path = module.__path__
-        except AttributeError:
-            continue
-        try:
-            imp.find_module('bbtags', app_path)
-        except ImportError:
-            continue
-        for f in os.listdir(os.path.join(os.path.dirname(os.path.abspath(module.__file__)), 'bbtags')):
-            mod_name, ext = os.path.splitext(f)
-            if ext == '.py':
-                __import__("%s.bbtags.%s" % (app, mod_name))
-    AUTODISCOVERED = True

bbcode/LICENSE.txt

+Copyright 2009 Jonas Obrist. All rights reserved.
+
+Redistribution and use in source and binary forms, with or without modification, are
+permitted provided that the following conditions are met:
+
+   1. Redistributions of source code must retain the above copyright notice, this list of
+      conditions and the following disclaimer.
+
+   2. Redistributions in binary form must reproduce the above copyright notice, this list
+      of conditions and the following disclaimer in the documentation and/or other materials
+      provided with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY JONAS OBRIST ``AS IS'' AND ANY EXPRESS OR IMPLIED
+WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL JONAS OBRIST OR
+CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
+ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+The views and conclusions contained in the software and documentation are those of the
+authors and should not be interpreted as representing official policies, either expressed
+or implied, of Jonas Obrist.
+################################################################################
+#
+# What is this?
+#
+################################################################################
+
+django-bbcode is a django application which was written to render BBCode syntax
+to HTML using python and django. However it does not restrict you to BBCode like
+syntax, you can use it to parse anything you like into anything you like.
+
+################################################################################
+#
+# Quickstart
+#
+################################################################################
+
+For those in a hurry:
+
+1.) Put django-bbcode into your python path into a folder called 'bbcode'.
+2.) Add 'bbcode' to your INSTALLED_APPS in django
+3.) Optional: Add/edit bbcode tags
+4.) Put {% load bbcode %} into the template where you want to render stuff.
+5.) Put {% bbcode varname %} where you want the stuff, here called 'varname', to
+    be rendered.
+6.) Done.
+
+################################################################################
+#
+# Slowstart
+#
+################################################################################
+
+For those who want to do it right:
+
+1.) Put django-bbcode into your python path into a folder called 'bbcode'.
+2.) Add 'bbcode' to your INSTALLED_APPS in django
+3.) Make sure all bbcode you have is valid. This can be done by using 
+    bbcode.validate(content, namespaces) (namespaces is optional) with the
+    content you want to save. This will either return None if no errors were
+    found or a list of errors that occured.
+4.) Put {% load bbcode %} into the template where you want to render stuff.
+5.) Put {% bbcode varname namespace1 "namespace2" %} where you want the stuff,
+    here called 'varname', to be rendered. Namespaces are optional. 'namespace1'
+    is a template variable holding either a string or a list (or tuple) of
+    strings. Those namespaces are used to filter the tags which are used to
+    render the content. You can also give it hardcoded strings like
+    '"namespace2"'. To exclude a namespace you can prefix the name with a 'no-'.
+    For example: "no-mynamespace". For more information on namespaces look
+    below.
+6.) Done.
+
+################################################################################
+#
+# What are those so called 'namespaces'?
+#
+################################################################################
+
+ - "Namespaces are one honking great idea -- let's do more of those!" - PEP0020
+
+I totally agree with that quote from the Zen of Python, thus I included
+namespaces to django-bbcode. Namespaces are used to dynamcially filter what tags
+are used to render a context. The main reason I introduced this feature was to
+turn off smilies in my project for users who disable smilies (like me, I hate
+those silly graphics). In this case I'd just use the namespace "no-smilies".
+
+By default every tag gets 3 namespaces: "__all__" (which holds all tags), the
+django app name (for all builtin tags: "bbcode") and the name of the module the
+tag is defined in (for smilies: "smilies",...). Additionaly you can add
+namespaces to tags by giving their class an attribute 'namespace' which holds a
+list of strings. Those namespaces will be used additionally to the default ones.
+
+You can then use those namespaces in the bbcode template tag as additional 
+arguments after 'content'. It allows you to give template variables holding 
+lists or tuples of strings or a single string. You can also put hardcoded 
+namespaces there (wrapped in single or double quotes).
+
+################################################################################
+#
+# How do I write a custom tag?
+#
+################################################################################
+
+Basically where you wanna start is by reading the examples (builtin tags). A tag
+is created by subclassing the TagNode class (or some of the specialized base 
+classes such as ReplaceTagNode, ArgumentTagNode, MultiArgumentTagNode,....).
+
+Every tag class needs a couple of attributes:
+
+open_pattern: a compiled regular expression which matches the opening tag.
+close_pattern: a compiled regular expression which matches the ending tag.
+parse: a function which returns a string.
+
+For each match of open_pattern/close_pattern pairs an instance of the class will
+be created. Each class gets the parent node, the regular expression match object
+and the full content as arguments.
+Also each instance of a tag class as an attribute called 'nodes' which is the
+list of child-tags (tags nested within this tag).
+
+################################################################################
+#
+# What are soft exceptions?
+#
+################################################################################
+
+Python exceptions stop the flow of code, thus I wrote the SoftExceptionManager
+(SEM). If a tag was used wrong, this will not stop the parser, but create an
+entry in the SEM. This way if several independent errors occur you will get them
+at once instead of one after another. If you use bbcode.validate (which you 
+really should!) and the content fails to parse correctly, you will get all 
+errors from the SEM. The bbcode.parse function also returns a tuple with the
+parsed content and the list of errors as items.
+
+Each error in the SEM is a tuple of (line_number, error_text).

bbcode/__init__.py

+"""
+BB Code parser by Jonas 'Ojii' Obrist (c) 2009
+
+USAGE:
+
+Parsing:
+
+parsed, errors = bbcode.parse(content, strict=True)
+
+This might raise a bbocde.PaserError if strict is True (default). Otherwise on a
+ParserError the content is returned unparsed and errors contains the reason.
+
+Validation:
+
+errors = bbcode.validate(content)
+
+Returns errors caused by parsing the code or an empty sequence.
+
+Extending:
+
+Subclassing bbcode.TagNode and bbcode.register the class adds new BB Code Tags.
+Each node must have an opening and closing pattern (open_pattern, close_pattern)
+and push, pushed, pull and close methods. For further information read the doc
+strings of the TagNode class.
+"""
+import re
+import cgi
+
+try:
+    from django.utils.translation import ugettext as _
+except ImportError:
+    _ = lambda x: x
+
+AUTODISCOVERED = False
+
+LINEFEED_PATTERN = re.compile('\n\s*\n', re.MULTILINE)
+def convert_linefeeds(content):
+    content = LINEFEED_PATTERN.sub('<br /><br />', content)
+    return content.replace('\n', '<br />')
+
+
+class UnmatchablePseudoPattern(object):
+    """
+    A class which should look like a compiled regular expression but never match.
+    """
+    def match(self, content):
+        return False
+    
+    def search(self, content):
+        return False
+    
+    def finditer(self, content):
+        return iter([])
+    
+    def sub(self, replacement, content):
+        return content
+
+
+class patterns:
+    """
+    This is a class for namespacing reasons
+    """
+    no_argument = r'\[%s\]'
+    self_closing_tag = r'\[%s\s*/\]'
+    single_argument = r'\[%s(\]|="?(?P<argument>[^\]]+)"?\])'
+    argument = r'( (\w+)=([^\] ]+))?'
+    closing = r'\[/%s\]'
+    unmatchable = UnmatchablePseudoPattern()
+     
+def get_tag_name(klass):
+    """
+    Convert a class to tagname
+    """
+    return klass.tagname if hasattr(klass, 'tagname') else klass.__name__.lower()
+
+class NeedsSubclassingError(Exception): pass
+class ParserError(Exception): pass
+
+
+class SoftException(object):
+    def __init__(self, lineno, message):
+        self.lineno = lineno
+        self.message = message
+        
+    def __str__(self):
+        return '<span class="bbcode-error lineno">Line %s:</span> <span class="bbcode-error message">%s</span>' % (self.lineno, self.message)
+    __unicode__ = __str__
+
+
+class SoftExceptionManager(object):
+    """
+    Allows 'soft exceptions'. Soft exceptions are exceptions which don't break
+    the flow of the code but are rather stored in a list and can then be told
+    given to the user.
+    """
+    def __init__(self):
+        self.exceptions = []
+        self.line_number = 1
+        
+    def set_line_number(self, number):
+        """
+        Update the line number
+        """
+        self.line_number = number
+        
+    def soft_raise(self, exception):
+        """
+        Soft raise an exception. Stores the line number the exception occured
+        and the exception message. If deployed in django it will make the 
+        message i18n ready.
+        """
+        self.exceptions.append(SoftException(self.line_number, _(exception)))
+        
+    def pull(self):
+        """
+        Pulls all exception since initialization or last pull. Resets exception
+        list.
+        """
+        old = self.exceptions
+        self.exceptions = []
+        return old
+
+sem = SoftExceptionManager()
+soft_raise = sem.soft_raise
+
+class VariableScope(dict):
+    def add(self, name, value):
+        dict.__setitem__(self, str(name), str(value))
+        
+    def resolve(self, context):
+        context = context.strip('"')
+        for var, value in dict.iteritems(self):
+            context = context.replace('$%s$' % var, value)
+        return context
+    
+    def lazy_resolve(self, context):
+        class Lazy:
+            def __init__(self, resolver, context):
+                self.resolver = resolver
+                self.context = context
+                
+            def __int__(self):
+                self.context = self.resolver(self.context)
+                return int(self.context)
+                
+            def __getattr__(self, attr):
+                self.context = self.resolver(self.context)
+                return self.context.__getattribute__(attr)
+        return Lazy(self.resolve, context)
+
+
+class Node(object):
+    """
+    This is the baseclass for all objects in a BBCode Parse Tree.
+    To understand Nodes it is important to understand the Tree.
+    Each Parse Tree has one, and only one, head node. This node has child nodes
+    and those children have child nodes themselves. This continues until there
+    are no more child nodes. In a standard Parse Tree the last leaves of a 
+    branch are instances of TextNode, however since empty TextNodes are not kept
+    in the Tree, they might also be missing.
+    
+    When the Parse Tree is generated the nodes get 'pushed', 'appended', 'pulled'
+    and 'closed'. Only TextNodes can be appended to a node's nodelist. When a
+    new child node is found it is 'pushed' and becomes the current node. When a
+    node cannot be closed correctly it is 'pulled', which means it's unparsed
+    contents are added to it's parent. Usually this causes a ParserError, which
+    means the Tree is not parseable. When a node is finished parsing it's
+    'closed' which normally returns the parent.
+    """
+    name = 'node'
+    
+    is_text_node = False
+    
+    def __init__(self, parent, match, fullcontent, context=None):
+        """
+        Normal nodes take their parent node as first argument, the regular
+        expression match as second argument and the full context as third
+        argument.
+        """
+        self.start = match.start()
+        self.fullcontent = fullcontent
+        self.raw_content = ''
+        self.parent = parent
+        self.match = match
+        self.nodes = []
+        self.context = context # for django only
+        # copy the variable scope
+        self.variables = parent.variables
+        
+    def soft_raise(self, errmsg):
+        soft_raise(errmsg)
+        return self.raw_content
+    
+    def append(self, text):
+        """
+        Adds a text node to the node
+        """
+        self.nodes.append(TextNode(self, text))
+    
+    def push(self, nodeklass, match, fullcontent):
+        """
+        Adds a nested tag node and returns that node
+        """
+        node = nodeklass(self, match, fullcontent, self.context)
+        self.nodes.append(node)
+        return node.pushed()
+    
+    def pushed(self):
+        """
+        Normal Nodes return themselves when being pushed. Self closing nodes
+        can overwrite this method to handle this in another fashion.
+        """
+        return self
+    
+    def pull(self, end):
+        """
+        Pulls all text nodes and returns the parent
+        """
+        self.parent.nodes.append(TextNode(self.fullcontent[self.start:end]))
+        return self.parent
+    
+    def close(self, end):
+        """
+        When closing the node just return the parent.
+        """
+        self.end = end
+        self.raw_content = self.fullcontent[self.start:end]
+        return self.parent
+    
+    def parse(self):
+        """
+        Parses the node. This is also responsible to parse child nodes. Should
+        return a string and fail silently.
+        """
+        raise NeedsSubclassingError
+        
+
+class HeadNode(Node):
+    """
+    The head node of the BBCode parse tree.
+    """
+    name = 'head'
+    def __init__(self, raw_content, context=None):
+        self.raw_content = raw_content
+        self.nodes = []
+        self.context = context
+        self.variables = VariableScope()
+    
+    def pull(self, end):
+        raise ParserError, "Cannot pull from headnode, invalid BBCode Tree"
+    
+    def close(self, end):
+        raise ParserError, "Cannot close headnode, invalid BBCode Tree"
+    
+    def parse(self):
+        content = ''
+        failed  = []
+        for node in self.nodes:
+            content += node.parse()
+        return content
+    
+    
+class TextNode(Node):
+    smilie_pattern = re.compile(':(?P<name>\w+):')
+    is_text_node = True
+    def __init__(self, parent, text):
+        self.text = text
+        self.variables = parent.variables
+        self.parent = parent
+        self.raw_content = text
+        self.nodes = []
+        
+    def append(self, text):
+        raise TypeError, "TextNode does not support appending"
+    
+    def push(self, node):
+        raise TypeError, "TextNode does not support pushing"
+    
+    def pull(self, end):
+        raise TypeError, "TextNode does not support pulling"
+    
+    def close(self, end):
+        raise TypeError, "TextNode does not support closing"
+    
+    def __repr__(self):
+        return '<TextNode instance "%s">' % self.text
+    
+    def parse(self):
+        """
+        Return cgi-escaped content
+        """
+        return cgi.escape(self.variables.resolve(self.text))
+    
+    def __str__(self):
+        return 'TextNode: %r' % self.text
+        
+    
+class TagNode(Node):
+    @staticmethod
+    def open_pattern():
+        raise NeedsSubclassingError
+    @staticmethod
+    def close_pattern():
+        raise NeedsSubclassingError
+    
+    def parse_inner(self):
+        """
+        Shortcut for parsing all inner nodes and return their combined contents.
+        """
+        inner = ''
+        for node in self.nodes:
+            inner += node.parse()
+        return inner
+    
+    def __str__(self):
+        return self.__class__.__name__
+    
+    
+class ReplaceTagNode(TagNode):
+    """
+    A specialized TagNode subclass with a predefined parse method. It allows
+    easy creation of simple bbcode - html replacement tags. [tag] becomes <tag>
+    and [/tag] becomes </tag>. These tags do not take any arguments and parse
+    all inner content.
+    Requires an explicit 'tagname' attribute, otherwise the lowered class name
+    will be used as tagname
+    """
+    def __init__(self, parent, match, content, context):
+        """
+        Implicitly set tag name if not available.
+        """
+        if not hasattr(self, 'tagname'):
+            self.tagname = self.__class__.__name__.lower()
+        TagNode.__init__(self, parent, match, content, context)
+        
+    def parse(self):
+        return '<%s>%s</%s>' % (self.tagname, self.parse_inner(), self.tagname)
+    
+    def __str__(self):
+        return 'ReplaceTagNode: %s' % self.__class__.__name__
+    
+    
+class ArgumentTagNode(TagNode):
+    """
+    TagNode which takes one (or no) argument. Open pattern must have a named
+    group 'argument'.
+    """
+    def __init__(self, parent, match, content, context):
+        TagNode.__init__(self, parent, match, content, context)
+        arg = match.group('argument')
+        self.argument = self.variables.lazy_resolve(arg.strip('"') if arg else '')
+        
+    def __str__(self):
+        return '%s (%s)' % (self.__class__.__name__, self.argument)
+
+
+class _MultiArgs(dict):
+    """
+    Dictionary-like class which allows items to be accessed via attributes.
+    """
+    def __getattr__(self, attr):
+        return dict.__getitem__(self, attr)
+        
+        
+class MultiArgumentTagNode(TagNode):
+    """
+    TagNode which takes multiple (or no) arguments. Must have an attribute
+    _arguments which holds key, value pairs of the arguments and their defaults.
+    Open pattern should use bbcode.patterns.argument as argument matching
+    expression.
+    """
+    _arguments   = []
+    def __init__(self, parent, match, content, context):
+        TagNode.__init__(self, parent, match, content, context)
+        args = match.groups()
+        kwargs = dict(self._arguments)
+        for index, value in enumerate(filter(bool, args)):
+            if not index or not index % 3:
+                continue
+            if not (index + 1) % 3:
+                kwargs[args[index - 1]] = self.variables.lazy_resolve(value)
+        self.arguments = _MultiArgs(kwargs)
+        
+    def __str__(self):
+        args = []
+        for key, value in self.arguments.iteritems():
+            args.append('%s: %s' % (key, value))
+        return '%s (%s)' % (self.__class__.__name__, ', '.join(args))
+        
+        
+class SelfClosingTagNode(TagNode):
+    """
+    A tag which is self closed.
+    """
+    close_pattern = patterns.unmatchable
+    
+    def __init__(self, parent, match, content, context):
+        self.start = match.start()
+        self.context = context
+        self.fullcontent = content
+        self.raw_content = content[match.start():match.end()]
+        self.parent = parent
+        self.match = match
+        self.nodes = []
+        self.variables = parent.variables
+    
+    def pushed(self):
+        """
+        A self closing node returns it's parent. Thus it will never have child
+        nodes!
+        """
+        return self.parent
+    
+    def __str__(self):
+        return 'SelfClosingTag: %s' % self.__class__.__name__
+    
+    
+class AutoDict(dict):
+    def __init__(self, default_thing=set, *args, **kwargs):
+        self.__default_thing = default_thing
+        dict.__init__(self, *args, **kwargs)
+
+    def __getitem__(self, item):
+        if not dict.__contains__(self, item):
+            dict.__setitem__(self, item, self.__default_thing() if callable(self.__default_thing) else self.__default_thing)
+        return dict.__getitem__(self, item)
+    
+    
+class Library(object):
+    """
+    The core of the BBCode parser. Keeps track of all bbcode tags and text
+    parsers. Also handles building BBCode Parse Trees and the automated help
+    generation.
+    """
+    name_pat1 = re.compile('([a-z0-9])([A-Z])')
+    name_pat2 = re.compile('(.)([A-Z][a-z]+)')
+    
+    def __init__(self):
+        self.names = AutoDict(None)
+        self.raw_names = {}
+        self.tags = AutoDict(set)
+        self.klasses = AutoDict(None)
+    
+    def convert(self, name):
+        """
+        Convert a class name to something a bit more readable
+        """
+        return  self.name_pat1.sub(r'\1 \2', self.name_pat2.sub(r'\1 \2', name))
+    
+    def dsparse(self, docs):
+        """
+        Parse docstrings
+        """
+        content, errors = parse(docs, strict=False, auto_discover=True)
+        return content
+    
+    def get_default_namespaces(self, klass):
+        bits = klass.__module__.split('.')
+        return (bits[-1], bits[-3], klass.__name__.lower())
+        
+    def register(self, klass):
+        """
+        Register a BBCode Tag Node
+        """
+        # Add the class to their namespaces.
+        if hasattr(klass, 'namespaces'):
+            for ns in klass.namespaces:
+                self.tags[ns].add(klass)
+                if not hasattr(klass, 'not_in_all') or not klass.not_in_all:
+                    self.tags['__all__'].add(klass)
+        elif not hasattr(klass, 'not_in_all') or not klass.not_in_all:
+            self.tags['__all__'].add(klass)
+        if not hasattr(klass, 'namespaces'):
+            setattr(klass, 'namespaces', [])
+        d_namespaces = self.get_default_namespaces(klass)
+        for default in d_namespaces:
+            self.tags[default].add(klass)
+        for ns in reversed(d_namespaces):
+            klass.namespaces.insert(0, ns)
+        # Register documentation
+        docstrings = klass.__doc__
+        if hasattr(klass, 'tagname'):
+            tagname = klass.tagname
+        else:
+            tagname = klass.__name__.lower()
+        if docstrings:
+            if hasattr(klass, 'verbose_name'):
+                verbose_name = klass.verbose_name
+            else:
+                verbose_name = self.convert(klass.__name__)
+            self.names[tagname] = {'docs': docstrings.strip(),
+                                   'name': verbose_name,
+                                   'class': klass}
+            self.klasses[klass] = self.names[tagname]
+        self.raw_names[klass.__name__] = klass
+        
+    def add_namespace(self, klass, *namespaces):
+        """
+        Add a tag to a namespace or several namespaces
+        """
+        if isinstance(klass, TagNode):
+            for namespace in namespaces:
+                self.tags[namespace].add(klass)
+        elif isinstance(klass, basestring):
+            if klass in self.raw_names:
+                self.add_namespace(self.raw_names[klass], *namespaces)
+            elif klass in self.names:
+                self.add_namespace(self.names[klass]['class'], *namespaces)
+                
+    def remove_namespace(self, klass, *namespaces):
+        """
+        Remove a tag from a namespace or several namespaces
+        """
+        if isinstance(klass, TagNode):
+            for namespace in namespaces:
+                if klass in self.tags[namespace]:
+                    self.tags[namespace].remove(klass)
+        elif isinstance(klass, basestring):
+            if klass in self.raw_names:
+                self.add_namespace(self.raw_names[klass], *namespaces)
+            elif klass in remove_namespace.names:
+                self.remove_namespace(self.names[klass]['class'], *namespaces)
+                
+    def set_not_in_all(self, klass, flag=True):
+        """
+        Set 'not_in_all' for a tag.
+        """
+        if flag:
+            self.remove_namespace(klass, '__all__')
+        else:
+            self.add_namespace(klass, '__all__')
+            
+    def get_help(self, *tags):
+        """
+        Get help for a tag or for all tags.
+        
+        Returns a dictionary with keys 'name', 'tag', 'docstring'.
+        """
+        if not tags:
+            tags = self.get_tags()
+        help_objects = []
+        for tag in tags:
+            if issubclass(tag, Node):
+                obj = self.klasses[tag]
+                if obj is None:
+                    continue
+            else:
+                obj = self.names[tag]
+                if obj is None:
+                    continue
+            help_objects.append({'name': obj['name'],
+                                 'docstring': parse(obj['docs'], strict=False, auto_discover=True)[0],
+                                 'obj': obj['class']})
+        return help_objects
+    
+    def get_tags(self, namespaces=None):
+        """
+        Get a list of tag classes for the namespaces
+        """
+        if namespaces is None:
+            namespaces = get_default_namespaces()
+        tags = set()
+        exclude = []
+        include = []
+        # Split the 'namespaces' into exclude and include namespaces
+        for ns in namespaces:
+            if ns.startswith('no-'):
+                _ns = ns[3:]
+                if _ns in self.tags:
+                    exclude.append(_ns)
+            elif ns in self.tags:
+                include.append(ns)
+        # Include first
+        if not include or '__all__' in include:
+            tags = set(self.tags['__all__'])
+        else:
+            if 'base' in include:
+                tags = set(self.tags['__all__'])
+            for ns in include:
+                tags = tags.union(self.tags[ns])
+        # Then exclude
+        for ns in exclude:
+            tags = tags.difference(self.tags[ns])
+        return tags
+    
+    def get_taglist(self, content, namespaces=None):
+        """
+        Get the tag-match list of a content for given namespaces
+        """
+        if namespaces is None:
+            namespaces = get_default_namespaces()
+        tags = self.get_tags(namespaces)
+        # Build tag list
+        taglist = []
+        for tagklass in tags:
+            op = tagklass.open_pattern
+            if callable(op):
+                op = op()
+            i = 1
+            for match in op.finditer(content):
+                i += 1
+                taglist.append((match.start(), match, tagklass, True))
+            cp = tagklass.close_pattern
+            if callable(cp):
+                cp = cp()
+            for match in cp.finditer(content):
+                taglist.append((match.start(), match, tagklass, False))
+        # Sort by position
+        return sorted(taglist)
+    
+    def get_parse_tree(self, content, namespaces=None, context=None):
+        """
+        Prepare content for parsing.
+        Returns a HeadNode instance
+        """
+        if namespaces is None:
+            namespaces = get_default_namespaces()
+        taglist = self.get_taglist(content, namespaces)
+        
+        # Get headnode
+        headnode = HeadNode(content, context)
+        
+        lastpos = 0
+        currentnode = headnode
+        # Loop over tag matches
+        for pos, match, tagklass, opener in taglist:
+            start, end = match.span()
+            # Prevent tags matching within other tags (eg AutoDetectURL)
+            if start < lastpos:
+                continue
+            # Append text between last tag and this one
+            text = content[lastpos:start]
+            if text:
+                currentnode.append(text)
+            # Set new position
+            lastpos = end
+            # Get line number for soft exceptions
+            lineno = content[:start].count('\n') + 1
+            sem.set_line_number(lineno)
+            # if opener, push new node
+            if opener:
+                currentnode = currentnode.push(tagklass, match, content)
+            # else close the tag
+            else:
+                # pull all unclosed child tags of the current node
+                while tagklass != currentnode.__class__:
+                    try:
+                        currentnode = currentnode.pull(end)
+                    except ParserError:
+                        sem.soft_raise("BBCode could not be parsed. There are probably unclosed or uneven tags!")
+                        raise ParserError, "Failed to find matching opening tag for closing tag '%s' in line %s."  % (get_tag_name(tagklass), lineno)
+                # close the node
+                currentnode = currentnode.close(end)
+        text = content[lastpos:]
+        if text:
+            headnode.append(text)
+        # Return the head node
+        return headnode
+    
+    def get_visual_parse_tree(self, content, namespaces=None, indent=4):
+        if namespaces is None:
+            namespaces = get_default_namespaces()
+        def recurse(nodes, level, indent):
+            cindent = level * indent
+            sindent = ' ' * cindent
+            next = level + 1
+            l = []
+            for node in nodes:
+                l.append('%s-%s' % (sindent, str(node)))
+                l += recurse(node.nodes, next, indent)
+            return l
+        try:
+            head = self.get_parse_tree(content, namespaces)
+        except ParserError:
+            return '-Parse Error'
+        visuals = ['-HeadNode']
+        visuals += recurse(head.nodes, 1, indent)
+        return '\n'.join(visuals)
+    
+    def validate(self, content, namespaces=None, auto_discover=False):
+        """
+        Validates a given content and returns the errors or an empty sequence.
+        """
+        if namespaces is None:
+            namespaces = get_default_namespaces()
+        if auto_discover:
+            autodiscover()
+        try:
+            headnode = self.get_parse_tree(content, namespaces)
+        except ParserError:
+            return sem.pull()
+        parsed = headnode.parse()
+        return sem.pull()
+
+
+lib = Library()
+register = lib.register
+validate = lib.validate
+get_help = lib.get_help
+get_visual = lib.get_visual_parse_tree
+
+def get_default_namespaces():
+    from django.conf import settings
+    if hasattr(settings, 'BBCODE_DEFAULT_NAMESPACES'):
+        return settings.BBCODE_DEFAULT_NAMESPACES
+    return ['__all__']
+    
+def parse(content, namespaces=None, strict=True, auto_discover=False,
+          context=None):
+    """
+    Parse a content with the BBCodes
+    """
+    if auto_discover:
+        autodiscover()
+    if namespaces is None:
+        namespaces = get_default_namespaces()
+    # Fix windows linefeeds
+    content = content.replace('\r','')
+    # Get head node
+    if strict:
+        head = lib.get_parse_tree(content, namespaces, context)
+    else:
+        try:
+            head = lib.get_parse_tree(content, namespaces, context)
+        except ParserError:
+            return convert_linefeeds(content), sem.pull()
+    # parse BB Codes
+    content = head.parse()
+    # Replace linefeeds
+    content = convert_linefeeds(content)
+    return content, sem.pull()
+    
+def autodiscover():
+    """
+    Automatically register all bbcode tags. This searches the 'bbtags' modules
+    of all INSTALLED_APPS if available.
+    """
+    global AUTODISCOVERED
+    if AUTODISCOVERED:
+        return
+    import imp
+    from django.conf import settings
+    import os
+
+    for app in settings.INSTALLED_APPS:
+        try:
+            module = __import__(app, {}, {}, [app.split('.')[-1]])
+            app_path = module.__path__
+        except AttributeError:
+            continue
+        try:
+            imp.find_module('bbtags', app_path)
+        except ImportError:
+            continue
+        for f in os.listdir(os.path.join(os.path.dirname(os.path.abspath(module.__file__)), 'bbtags')):
+            mod_name, ext = os.path.splitext(f)
+            if ext == '.py':
+                __import__("%s.bbtags.%s" % (app, mod_name))
+    AUTODISCOVERED = True

bbcode/bbtags/__init__.py

Empty file added.

bbcode/bbtags/advanced.py

+from bbcode import *
+import re
+
+class Hidden(TagNode):
+    """
+    Defines a text to be hidden. The visibility of the text can be toggled using a button.
+    
+    Usage:
+    
+    [code lang=bbdocs linenos=0][hidden]Secret content[/hidden][/code]
+    """
+    num = 0
+    open_pattern = re.compile(patterns.no_argument % 'hidden')
+    close_pattern = re.compile(patterns.closing % 'hidden')
+        
+    def parse(self):
+        Hidden.num += 1
+        return '<p><input type="button" onclick="toggle(\'hidden_%s\');" value="Toggle" /></p><div style="display:none" id="hidden_%s">%s</div>' % (self.num, self.num, self.parse_inner())
+    
+register(Hidden)

bbcode/bbtags/brainfuck.py

+from bbcode import *
+import re
+import cgi
+
+def parseout(bfcode):
+    try:
+        output = parsebf(bfcode)
+    except (UnknownLanguageCommand, DataPointerError, UnevenSquareBracketsError, ValueError, NotImplementedError), e:
+        return e.message
+    return output
+
+def parsebf(bfcode):
+    code_end = len(bfcode)
+    instruction_pointer = 0
+    data_pointer = 0
+    cells = [0]
+    output = ''
+    def jump(pointer, opener, closer, direction):
+        start = pointer + 1
+        opened = 1
+        while opened:
+            if direction == '+':
+                pointer += 1
+            else:
+                pointer -= 1
+            if pointer == code_end or pointer < 0:
+                raise UnevenSquareBracketsError, "Uneven square brackets (@%s)" % start
+            if bfcode[pointer] == opener:
+                opened += 1
+            elif bfcode[pointer] == closer:
+                opened -= 1
+        return pointer
+    while instruction_pointer < code_end:
+        current = bfcode[instruction_pointer]
+        verbose_pointer = instruction_pointer + 1
+        if current == '>':
+            data_pointer += 1
+            if len(cells) == data_pointer:
+                cells.append(0)
+        elif current == '<':
+            if data_pointer == 0:
+                raise DataPointerError, "Data pointer cannot be zero (@%s)" % verbose_pointer
+            data_pointer -= 1
+        elif current == '+':
+            if cells[data_pointer] == 255:
+                raise ValueError, "Byte cannot exceed 255 (@%s)" % verbose_pointer
+            cells[data_pointer] += 1
+        elif current == '-':
+            if cells[data_pointer] == 0:
+                raise ValueError, "Byte cannot be negative (@%s)" % verbose_pointer
+            cells[data_pointer] -= 1
+        elif current == '.':
+            output += chr(cells[data_pointer])
+        elif current == ',':
+            raise NotImplementedError, "Input (',') is not implemented yet (@%s)" % verbose_pointer
+        elif current == '[':
+            if cells[data_pointer] == 0:
+                instruction_pointer = jump(instruction_pointer, '[',']','+')
+        elif current == ']':
+            if cells[data_pointer] != 0:
+                instruction_pointer = jump(instruction_pointer, ']','[','-')
+        else:
+            raise UnknownLanguageCommand, "Unknown language command: '%s' (@%s)" % (current, verbose_pointer)
+        instruction_pointer += 1
+    return output
+
+class Brainfuck(SelfClosingTagNode):
+    """
+    Executes a brainfuck statement
+    
+    Usage:
+    
+    [code lang=bbdocs linenos=0][brainfuck]++++++++++[>+++++++>++++++++++>+++>+<<<<-]>++.>+.+++++++..+++.>++.<<+++++++++++++++.>.+++.------.--------.>+.>.[/brainfuck][/code]
+    
+    Note: this brainfuck implementation does not support the , command.
+    """
+    open_pattern = re.compile(r'\[brainfuck\](?P<bfcode>[+-\[\].><]+)\[/brainfuck\]')
+    
+    def parse(self):
+        bfcode = self.match.group('bfcode')
+        parsed = cgi.escape(parseout(bfcode))
+        return """<p style="font-weight: bold;">Brainfuck</p>
+                  <code class="code">%s</code>
+                  <p style="font-weight: bold;">Output</p>
+                  <pre class="code">%s</pre>""" % (bfcode, parsed)
+        
+    
+register(Brainfuck)

bbcode/bbtags/functional.py

+"""
+This module only handles how to define variables, their storage/resolving is in
+the main module!
+"""
+
+from bbcode import *
+import re
+
+inner_re = re.compile('(?P<name>\w+)\s*=\s*(?P<value>.+)')
+
+
+class BBStyleVariableDefinition(TagNode):
+    """
+    Stores a value in a variable.
+    
+    Usage:
+        [code lang=bbdocs linenos=0][def]varname=value[/def][/code]
+        
+    The stored variable can be used in many other tags. Variables are wrapped in
+    dollar signes when used.
+    
+    Example:
+    
+        [code lang=bbdocs linenos=0][def]myvar=http://www.mysite.com[/def]
+[url=$myvar$/someimg.png]super cool picture[/url][/code]
+    """
+    open_pattern = re.compile(patterns.no_argument % 'def')
+    close_pattern = re.compile(patterns.closing % 'def')
+    
+    def parse(self):
+        inner = ''
+        for node in self.nodes:
+            if not node.is_text_node:
+                soft_raise("def tag cannot have nested tags")
+                return self.raw_content
+            else:
+                inner += node.raw_content
+        match = inner_re.match(inner)
+        if not match:
+            soft_raise("invalid syntax in define tag: inner must be 'name = value'")
+            return self.raw_content
+        name = match.groupdict()['name']
+        value = match.groupdict()['value']
+        real_value = self.variables.resolve(value)
+        self.variables.add(name, real_value)
+        return ''
+    
+    
+class BBStyleArguments(TagNode):
+    """
+    Sets default arguments for all tags within this tag.
+    
+    Usage:
+    
+        [code lang=bbdocs linenos=0][args arg1=val1]
+...
+[/args][/code]
+        
+    Example:
+    
+        [code lang=bbdocs linenos=0][args align=right]
+[img]http://www.mysite.com/1.png[/img]
+[img]http://www.mysite.com/2.png[/img]
+[/args][/code]
+        
+    This would align both images 'right'.
+    """
+    open_pattern = re.compile('\[args(?P<args>(=[^\]]+)| ([^\]]+))\]')
+    close_pattern = re.compile(patterns.closing % 'args')
+    verbose_name = 'Arguments'
+    
+    def __init__(self, parent, match, content, context):
+        TagNode.__init__(self, parent, match, content, context)
+        arg = match.group('args')
+        self.args = self.variables.lazy_resolve(arg.strip('"') if arg else '')
+    
+    def parse(self):
+        # get the arguments
+        if self.args.startswith('='):
+            return self.parse_single(self.args[1:])
+        else:
+            return self.parse_multi(dict(map(lambda x: x.split('='), filter(bool, self.args.split(' ')))))
+            
+    def parse_multi(self, argdict):
+        def recurse(nodes, argdict):
+            for node in nodes:
+                if hasattr(node, 'arguments'):
+                    for key, value in node.arguments.iteritems():
+                        if key in argdict:
+                            node.arguments[key] = argdict[key]
+                if node.nodes:
+                    recurse(node.nodes, argdict)
+        recurse(self.nodes, argdict)
+        inner = ''
+        for node in self.nodes:
+            inner += node.parse()
+        return inner
+    
+    def parse_single(self, arg):
+        def recurse(nodes, argument):
+            for node in nodes:
+                if hasattr(node, 'argument'):
+                    node.argument = argument
+                if node.nodes:
+                    recurse(node.nodes, argument)
+        recurse(self.nodes, arg)
+        inner = ''
+        for node in self.nodes:
+            inner += node.parse()
+        return inner
+    
+
+class BBStyleRange(MultiArgumentTagNode):
+    """
+    A very basic numerical loop. Useful for inserting lots of numbered pictures.
+    
+    Usage:
+    
+        [code lang=bbdocs linenos=0][range start=1 end=16 name=index zeropad=3]
+...
+[/range][/code]
+        
+    Arguments:
+    
+        start: the first number in the loop
+        end: the last number in the loop
+        name: the name of the variable to assign the number to within the loop
+        zeropad: enables zeropadding. eg. 1 with zeropad 3 becomes 001.
+        
+    Example:
+    
+        [code lang=bbdocs linenos=0][range end=10]
+[img]http://www.mysite.com/img_$index$.png[/img]
+[/range][/code]
+        
+        is the equivalent to:
+    
+        [code lang=bbdocs linenos=0][img]http://www.mysite.com/img_001.png[/img]
+[img]http://www.mysite.com/img_002.png[/img]
+[img]http://www.mysite.com/img_003.png[/img]
+[img]http://www.mysite.com/img_004.png[/img]
+[img]http://www.mysite.com/img_005.png[/img]
+[img]http://www.mysite.com/img_006.png[/img]
+[img]http://www.mysite.com/img_007.png[/img]
+[img]http://www.mysite.com/img_008.png[/img]
+[img]http://www.mysite.com/img_009.png[/img]
+[img]http://www.mysite.com/img_010.png[/img][/code]
+    """
+    _arguments = {
+        'start': '1',
+        'end': '',
+        'name': 'index',
+        'zeropad': '3'
+    }
+    
+    @staticmethod
+    def open_pattern():
+        pat = r'\[range'
+        for arg in BBStyleRange._arguments:
+            pat += patterns.argument
+        pat += r'\]'
+        return re.compile(pat)
+    
+    close_pattern = re.compile(patterns.closing % 'range')
+    verbose_name = 'Range'
+    
+    def parse(self):
+        if not self.arguments.end:
+            return self.soft_raise('Range tag requires an end argument')
+        if not self.arguments.start.isdigit() or not self.arguments.end.isdigit():
+            return self.soft_raise('Range arguments must be digits')
+        if not self.arguments.zeropad.isdigit():
+            return self.soft_raise('Range argument zeropad must be digit')
+        start = int(str(self.arguments.start))
+        end   = int(str(self.arguments.end))
+        zeropad = int(str(self.arguments.zeropad))
+        if start < 0 or end < start:
+            return self.soft_raise('Range arguments start must be positive and end must be bigger than start')
+        output = ''
+        for i in range(start, end + 1):
+            self.variables.add(self.arguments.name, '%%0%si' % zeropad % i)
+            output += self.parse_inner()
+        return output
+register(BBStyleArguments)
+register(BBStyleVariableDefinition)
+register(BBStyleRange)

bbcode/bbtags/lists.py

+from bbcode import *
+import re
+
+
+class OL(MultiArgumentTagNode):
+    """
+    Creates an ordered list.
+    
+    Usage:
+    
+    [code lang=bbdocs linenos=0][ol]
+  [*] First item
+  [*] Second item
+[/ol][/code]
+    """
+    _arguments = {'css': '',
+                  'itemcss': ''}
+    
+    @staticmethod
+    def open_pattern():
+        pat = r'\[ol'
+        for arg in OL._arguments:
+            pat += patterns.argument