How the Plugin DB Will Work
The actual Plugin DB will have two components. The first is the Archive, which is a simple static repository containing plugin information and metadata. The second is the Browser, which will be on the Zine Web site and provide a browsable Web interface and a JSON API for querying the Plugin DB. Supporting both of these will be the module
zineweb.plugin, which is capable of loading and dumping Zine plugins independent of an actual Zine instance.
To facilitate the complex path manipulations that the Plugin DB will require, my Pathway library will be added to the list of dependencies for the Web site. This will not have anything to do with the Zine core.
This module is not actually connected to the rest of Zineweb, but this might as well be here. (A separate release later is possible.) Its main purpose is manipulating Zine plugins like
zine.pluginsystem without a Zine instance or even having Zine installed.
Unlike an earlier draft, the archive will have a single root for all versions of Zine. There are two special files in the root of the archive. The first is categories.txt, and the second is zineversions.txt. All directories under the archive root that have names matching the regular expression
[a-zA-Z_]\w* (in other words, Python identifiers) are assumed to be plugin directories.
This file contains the data for categories. It uses a custom format based on lines and designed for internationalization. Lines beginning and ending with straight brackets are headers, and must have a Python identifier in the middle. Subsequent lines under headers should start with a language code, followed by a colon. Next comes the verbose name for the category, and another colon. The rest of the line is a sentence describing the category. Blank lines, or lines starting with
#, are ignored. For example:
[themes] en: Themes: Change the appearance of your blog. l3: th3m35: ch4ng3 the 4pp34r4nc3 f j00 bl0g. [parsers] en: Parsers: Add support for writing posts in different markup languages. [misc] en: Miscellaneous: Do other cool stuff. l3: m15c: d0 0ther 1337 h4x0r 5tuff.
(In the example above, l3 stands for l33t.) 'en' is assumed to be the default language, and will be the only one until internationalization gets working.
This is much simpler. It is a simple text file, with one entry per line. This is the major and minor number of every extant zine version (0.1, 0.2, 0.3, etc). Any plugins declaring their support for any other version will be misfiled, so be diligent about this.
A plugin directory is merely a container for plugin versions. A file named versions.txt exists in the root, and is simply a sequence of lines. All non-blank lines represent a plugin version number. They should be in release order, as comparing version numbers is a pain in the neck. For every version in the versions.txt file, there should be a folder of the same name. This is the version directory. It should, of course, contain the actual plugin file, which should take the form import_name-version.plugin. There is also a metadata.txt file, which has the same format as Zine's metadata.txt, including the internationalized keys (though, again, internationalization probably won't actually be supported initially). The supported keys are:
- Name, Plugin URL, Description, Author, Author URL, Contributors, and Depends have the same semantics as they do in normal metadata.txt. (Version is not included because that is obtained from the directory name.)
- Preview is the path - WITHIN the version directory - to the preview image.
- Python Depends is a comma-separated list, similar to normal Depends, containing a list of Python modules/packages/whatever that the package depends on.
- License is the license (BSD, MIT, GPL, etc.) that the plugin is released under.
- Categories is the identifiers for each category, comma-separated.
- Zine Versions is a comma-separated list of Zine versions supported by the plugin. Only the number part should be included - any suffixes like -dev will be ignored.
(It would be preferable if Python Depends and Zine Versions were added to the official metadata.txt standard to facilitate extracting this data from the plugin. If extracting from metadata.txt, if a Preview Path field exists, the referred file is extracted and used as "Preview" here. Otherwise, the Preview entry is split on '::' and looked for in the plugin's 'shared' directory.)
Also in the version directory can be a file named description.html. This should be valid HTML 4, and is used as the long description on the plugin detail page if present. Finally, there can be an image referred to in the Preview field.
Traversing all this data on every request would be slow and stupid. There will be some sort of index command to generate an index of the plugins. This will be stored in the archive root, and named index.json. This collects as much of the data from the plugins into one place.
The "categories" key contains a dictionary mapping category identifiers to objects, where each key is a language code and each value is a length-2 list of [name, description].
The "plugins" key contains a dictionary mapping plugin import names to objects. Each plugin object has keys for each version, and the empty string key ("") holds the list of versions, in order. Every plugin version is simply its metadata, where every item in the metadata is represented by a dictionary mapping language code to value (no language code is "").
The "versions" key contains a list of Zine versions, in order.
The "versionmap" key contains an object. The keys of the object are Zine versions, and the values are objects mapping categories (or "" for all categories) to a list of [plugin_name, versions] entries.
The browser is accessed under the '/extend/plugins' heading of the Web site. When you go to that landing page, it has a brief description of the plugin archive. The sidebar has two lists of links - one of Zine versions (with an entry on top for "All"), and one of plugin categories (with an entry on top for "All"). The final URL map will look something like this:
- /extend/plugins/ to plugins_index
- /extend/plugins/all/ to plugins_list
- /extend/plugins/<zversion:version>/ to plugins_list
- /extend/plugins/all/<string:category> to plugins_list
- /extend/plugins/<zversion:version>/<string:category> to plugins_list
- /extend/plugins/<identifier:plugin>/ to plugin_detail
- /extend/plugins/<identifier:plugin>/<string:version> to plugin_detail
zversion and identifier are custom converters. zversion matches a number, a dot, and another number. identifier matches a valid Python identifier. The only possibility of overlap is if a plugin is named
all, but that shouldn't be a problem because "all" is a lame name for a plugin anyway.
The plugins_list endpoint accepts 'version' (for Zine version) and 'category' (for the plugin category) from the URL. If either is None, it is interpreted as "all". It simply displays the latest version of the matching plugins, in alphabetical order by plugin name. (Pagination may be implemented once there are enough plugins to worry about it.) Each entry in the list will have its name (which links to its detail page), matching version, description, categories, and supported Zine versions. There will also be a list of "other versions" that indicate other versions matching the search criteria.
The plugins_detail endpoint accepts 'plugin' (for plugin name) and 'version' (for the plugin version) from the URL. It displays pretty much all the available metadata, as well as the "description.html" if it is present. There is also a download link to the plugin file.
Browser JSON API
The Browser provides a JSON based API for obtaining plugin information from off the Plugin DB. It exposes this from the /extend/plugin-api URL family. All methods may end up supporting a ?lang=code query parameter that will send back the data in the given language code if available rather than English.
- /extend/plugin-api/: Describes the Plugin API.
- /extend/plugin-api/plugin-list: Returns a list of all plugins. You can pass
- /extend/plugin-api/versions: Returns a list of Zine versions.
- /extend/plugin-api/categories: Returns a list of categories.
- /extend/plugin-api/plugin/<identifier:plugin>/: Returns information on the available versions of the given plugin.
- /extend/plugin-api/plugin/<identifier:plugin>/<string:version>: Returns the metadata for the plugin version given.
Exact details on the Plugin API will come later, once I get the Archive and Browser working.
Zineweb finds its archive from the
ZINEWEB_ARCHIVE environment variable. It should have the base filesystem path to the archive directory, a semicolon, and the URL where the archive directory can be accessed. (If the base URL ends in a slash, it will be stripped.)
Once The Website Is Finished
Once the Website is finished and the Pocoo team have uploaded my changes to their server, development on the Plugin Database Plugin will begin. This will let plugins be downloaded directly from the Plugin DB.