audrid / docs / source / validators.rst

Full commit


Validators are a core part of the application, since most content is persisted in a document-oriented fashion. Validators are used to check all incoming data for sanity. There are four types of payloads:

  • Tasks
  • Pools
  • Exams
  • Audits

There are many validation libraries available for Python. We will use voluptuous.


The schema for a single answer task is the following:

single = Schema({
    required('kind'): 'single',
    required('id'): all(unicode, length(min=6, max=10)),
    required('task'): all(unicode, length(min=10, max=65535)),
    'answer': all(unicode, length(min=10, max=140)),
    'options': dict,

An example input would be:

    "id" : "text-0",
    "task" : "What is the answer to all questions?",
    "kind" : "single"

The answer key is specified as a optional key within the question. That way it is possible to define validation functions, that relate the form of the answer to the form of the question. E.g. a mapping task data structure should not be considered valid, if the answer contains keys that were not part of the question.

In the example above, the answer is specified as a unicode field with a length between 10 and 140 characters.

During a test, only the answers can be exchanged. The answer is temporary merged with the task datastructure to check for formal compliance. If the data structure validates, the answer is saved in the answers column of the Audit table. (Suggested serialization):

    "text-0" : "42"
    "geocap" : {
        "Berlin" : "Germany",
        "Paris" : "France"

The corresponding REST API entry points would be (example with curl):

$ curl -XGET localhost:8000/api/v2/audits/1/answers
$ curl -XGET localhost:8000/api/v2/audits/1/answers/geocap

As an example, we want to add the above answers to the system via curl:

$ curl -XPUT -H 'Content-Type: application/json' \
    http://localhost:8000/api/v2/audits/1/answers/geocap \
    -d '{"Berlin" : "Germany", "Paris" : "France"}'

$ curl -XPUT -H 'Content-Type: application/json' \
    http://localhost:8000/api/v2/audits/1/answers/text-0 \
    -d '42'

Every interaction with the answers subsystem is intercepted to ensure some rules:

  • modify answers only when the audits status is started (once the time for the exam is over, this gets set to finished by the client)
  • answers conform to specification
  • log everything

At the end, the answers field of the Audit table contains a dictionary, that contains one key for every task attempted.


Pools are mainly a collection of tasks. The schema for a pool is as follows:

pool = Schema({
    required('id'): all(unicode, length(min=2, max=36)),
    required('tasks'): unique([any(

Pools have an unicode ID, which is also part of the URL.

$ curl localhost:5000/api/v2/pools/econ101
$ curl localhost:5000/api/v2/pools/econ201

Each pool is versioned in the background. When an exam is created, it is a version of the pool, that is referenced, not the pool directly. This way it is possible to keep all revisions of pools in the system, which is important, because one might want to evolve a question pool while retaining historical records for the exams already performed. It is possible to access previous versions of a pool via API:

$ curl localhost:5000/api/v2/pools/econ101/versions

This also provides data to perform quality assurance (QA) since it is traceable who edit which question when and optionally why (through a comment field).


Exams are pools, that made public for examination.


Audits combine exams and users. This is where all the answers are stored.