-moin strongly relies on indexes that accelerate access to item metadata and data.
+moin strongly relies on indexes that accelerate access to item metadata and
+data and makes it possible to have simple backends, because the index layer
+is doing all the hard and complex work.
Indexes are used internally for many operations like item lookup, history,
iterating over items, search (also for interactive searches), etc..
-So, you need to configure indexing correctly first.
+moin won't be able to start with damaged, inaccessible or non-existing indexes.
+So, you need to configure and initialize indexing correctly first.
moin will automatically update the index when items are created, updated, deleted,
-destroyed or renamed (via the storage api of moin):
-* when users change content via the user interface
-* when xml is unserialized to load items into the backend
-In case you have items changing without usage of the backend api or in case
-your index gets damaged or lost, you need to manually do an index build -
-or moin won't be able to work correctly.
+destroyed or renamed (via the storage api of moin, indexing layer or above).
-Your wiki config needs ``index_dir`` and ``index_dir_tmp`` to point to different
-directories. They have default values and most likely you don't need to change
+Your wiki config needs ``index_dir`` to point to a writable directory - a fast,
+local filesystem is preferred.
But if you want, try something like::
- index_dir = "/path/to/moin-2.0/wiki/index"
- index_dir_tmp = "/path/to/moin-2.0/wiki/tmp_build"
-**Note:** Paths MUST BE absolute.
+* The path MUST be absolute.
+* Moin will use `index_dir`.temp location also, if you build an index at
+the so-called `temporary location`.
moin index script reference
-You can use the ``moin index`` script to build, update, clean, move and monitor
+You can use the ``moin index-*`` group of script commands to manage indexes.
-MoinMoin uses 2 indexes: ``latest-revs`` (index stores only current revisions)
-and ``all-revs`` (index stores all revisions).
+Many of the script commands for index management support a `--tmp` option to use
+the temporary index location. This is useful if you want to do index operations
+in parallel to a running wiki which is still using the index at the normal
-**Note:** If you see <indexname> below, use one of ``latest-revs``, ``all-revs``
+Creates an empty, but valid index.
-Let's see what you can do with that stuff.
+**Note:** the moin wsgi application needs an index to successfully start up.
+As the moin index-* script commands are also based on the moin wsgi application,
+this can lead to "which came first, the chicken or the egg?" like problems.
-Index all revisions of all items to the index located in ``index_dir_tmp`` (we
-use this separate location so that index building does not affect the index
-your wiki engine is currently using).
+To solve this, the moin command has a ``-i`` (``--index-create``) option that
+will trigger index creation on startup.
-If there is no index at that location yet, a new index will be built there.
-If there is already an index at that location, that index will be extended.
+Additionally, if the storage is also non-existant yet, one might also need
+``-s`` (``--storage-create``) to create an empty storage on startup.
+Process all revisions of this wiki and add the indexable documents to the index.
- moin index --for <indexname> --action build
+* For big wikis, this can take rather long, consider using --tmp.
+* index-build does NOT clear the index at the beginning.
+* index-build does not check the current contents of the index, you must not run
+ index-build multiple times for the same data / same wiki.
-**Note:** moin won't use this index until you have moved it to ``index_dir``.
+Compare an index to the current storage contents and update the index as
+needed (add, remove, update stuff) to reflect the current storage contents.
-Move indexes from ``index_dir_tmp`` to ``index_dir``.
+**Note:** You can use this after building at the tmp location, to also get
+the changes that happened to the wiki while building the index. You can run
+index-update multiple times to increasingly catch up.
+Destroy an index (nothing left at the respective location).
- moin index --for <indexname> --action move
+Move the index from the temporary location to the normal location.
-Update the index located in ``index_dir`` to reflect the current backend
-contents. Add new stuff, remove outdated stuff.
+Optimize an index, see Whoosh docs for more details.
+Output index contents in human readable form, e.g. for debugging purposes.
- moin index --for <indexname> --action update
-Create empty index in ``index_dir`` for given index (previous will be erased).
- moin index --for <indexname> --action clean
-Show contents of the index located in ``index_dir`` in human readable form.
-This is mostly used for debugging.
- moin index --for indexname --action show
-**Note:** field length limited to 40 chars.
-**Note:** fields without attribute ``stored=True`` are not displayed.
+**Note:** only fields with attribute ``stored=True`` can be displayed.
Building an index for a single wiki
-Build index at separate place, move it at right place:
- moin index --for both --action build
- moin index --for both --action move
+If your wiki is fresh and empty
+ moin index-create --storage-create --index-create
+ moin index-create -s -i # same, but shorter
+Storage and index is now initialized (both empty).
+If you add stuff to your wiki, the index will get updated on the fly.
+If your wiki has data and is shut down
+If index needs a rebuild for some reason (e.g. index lost, index damaged,
+incompatible upgrade, ...), use::
+ moin index-build # can take a while...
+If your wiki has data and should stay online
+ moin index-create -i --tmp
+ moin index-build --tmp # can take a while...
+ moin index-update --tmp # should be quicker, make sure we have 99.x%
+ # better shut down the wiki now (or at least make sure it is not changed)
+ moin index-update --tmp # make sure we have indexed ALL content - should be even quicker.
+ moin index-move # instantaneously
+ # start the wiki again (or allow changes now again)
+**Note:** Indexing puts load onto your server, so if you like to do regular
+index rebuilds, schedule them at some time when your server is not too busy
Building an index for a wiki farm
index_dir = "/path/to/wiki/index"
- index_dir_tmp = "/path/to/wiki/index_tmp"
wiki config for ``Engineering``::
interwikiname = u"Engineering"
index_dir = "/path/to/wiki/index"
- index_dir_tmp = "/path/to/wiki/index_tmp"
-Now do the initial index building:
+Now do the initial index building:
- moin index --for both --action build # in Sales virtual env
- moin index --for both --action build # in Engineering virtual env
- moin index --for both --action move # you can run it from any virtual env
+ moin index-create -i # create an empty index
+ # now ADD stuff from all your wikis:
+ moin index-build # with Sales wiki configuration
+ moin index-build # with Engineering wiki configuration
Now you should have a shared index for all these wikis.
**Note:** Do not build indexes for multiple wikis in parallel, this is not
-Building indexes while your wiki is running
-If you want to build an index while your wiki is running, you have to be
-careful not to miss any changes that happen while you build the index.
-``moin index --action build`` is made to not interfere with your running wiki.
-So you can run this in parallel without taking your wiki offline.
-Depending on the size of your wiki, index build can take rather long - but it
-doesn't matter as you don't have to take your wiki offline for this.
-But: if indexing takes rather long, it can easily happen that content that was
-already put into the index is updated afterwards in the online wiki. So we need
-to do a quick index update while the wiki is offline:
-Offline your wiki (or at least make it read-only, so no data in it changes).
-``moin index --action move`` to move indexes into place.
-``moin index --action update`` to add anything we might have missed otherwise.
-As this is not as much as doing a full index build, this should be rather quick
-(but still: it has to look at every item in your wiki, whether it has been
-updated after the initial index build).
-Put your wiki back online again.
-**Note:** Indexing puts load onto your server, so if you like to do regular
-index rebuilds, schedule them at some time when your server is not too busy