find a way to create static html dumps

Issue #9 resolved
Thomas Waldmann
repo owner created an issue

moin 1.9 had "moin dump" to create static html dumps, but moin2 has no such feature yet.

Problem: if one wants to load such html directly from the filesystem into a browser (without using a web server), the URLs in the html need to be relative. also, the filenames need to get some appropriate extension, because there is no web server to give a content-type header.

I tried Frozen-Flask a while ago, but was unable to create relative URLs. So we could use this just for html dumps that are served at some specific URL.

Comments (6)

  1. RogerHaase

    I gave Frozen-Flask a try on Windows 4 years after the above comments. Short answer is Frozen-Flask looks hopeless.

    Last release is Version 0.11 on 2013-06-13. There are a few patches added on github and 10 open issues: https://github.com/SimonSapin/Frozen-Flask.

    Problems:

    1. On moin 1.9, the "export dump" command created .html files for wiki pages. Frozen-Flask hooks into the url_for method and tries to create pages for everything, including the +admin, +misc, etc resulting in unhandled 403 return codes. I had to add code to Frozen-Flask to ignore the 403's. There is no option to ignore files matching a regex. The files created do not have a .html suffix.

    2. Seems poorly tested on Windows, yielding some permission denied errors trying to write a file. A guess is an attempt was made to open the file twice.

    3. On the few occasion I somehow managed to process the Home, moin, creole files, a directory named Home, moin, creole was created with an index.html file within each.

    4. Links to any URL with an embedded + are broken.

    To get Frozen-Flask "working", I did the following:

    To make.py, add this near line 378:

    def cmd_dump(self, *args):
        """create a directory of html files for each item within the wiki"""
        if wiki_exists():
            command = '{0}moin dump'.format(ACTIVATE)
            with open(BACKUPWIKI, 'w') as messages:
                result = subprocess.call(command, shell=True, stderr=messages, stdout=messages)
            if result == 0:
                print 'Success: wiki was dumped to {0}'.format(filename)
            else:
                print 'Important messages from {0} are shown below. Do "{1} log backup" to see complete log.'.format(BACKUPWIKI, M)
                search_for_phrase(BACKUPWIKI)
                print '\nError: attempt to backup wiki failed.'
        else:
            print 'Error: cannot dump wiki because it has not been created.'
    

    To /script/init.py, add this near line 38:

    from MoinMoin.script.maint import moindump
    manager.add_command("dump", moindump.Dump())
    

    In /script/maint/ add a new file moindump.py:

    """
    MoinMoin - use Frozen-Flask to create a static html dump of wiki items
    """
    
    import sys
    import os
    
    from flask.ext.frozen import Freezer
    from flask import current_app as app
    from flask.ext.script import Command, Option
    
    from MoinMoin.app import create_app
    
    
    wiki_config = os.path.dirname(os.path.abspath(__file__)) + '/../../../wikiconfig_local.py'
    app = create_app(wiki_config)
    
    
    freezer = Freezer(app)
    
    
    @freezer.register_generator
    def product_url_generator():
        # URLs as strings
        yield '/Home/'
        yield '/creole/'
        yield '/moin/'
    
    
    from MoinMoin import log
    logging = log.getLogger(__name__)
    
    
    class Dump(Command):
    
        def run(self):
            freezer.freeze()
    

    In wikiconfig_editme.py, add a few config parameters:

    FREEZER_BASE_URL = 'http://localhost:8080'
    FREEZER_DESTINATION_IGNORE = []
    FREEZER_DESTINATION = '../FreezerQQQ'
    FREEZER_REMOVE_EXTRA_FILES = True
    FREEZER_IGNORE_404_NOT_FOUND = True
    FREEZER_RELATIVE_URLS = False
    

    Run it by doing ./m dump

  2. RogerHaase

    https://codereview.appspot.com/261900043/

    • needs a CMS-like theme and a way to override current theme
      • app.cfg.theme_default = u'modernized' has no effect
      • copy theme.css
    • should all raw data be copied to +get directory, or only images, zip, targz, ...?
    • append ".html" to all html files and fix resulting broken links
    • fix links to subitems
      • <a href="Home/subitem"> becomes <a href="Home%2fsubitem">
      • but do not change <a class=" moin-https" href="https://moinmo.in/FrontPage">
    • the "svg" image does not display, but works if renamed to svg.svg and link is changed accordingly
  3. Log in to comment