soil /

Filename Size Date modified Message
4.4 KB
set_context properly calls destructor
274 B
Shrink the mmapped file on windows,
112 B
added credits, license
1.1 KB
added credits, license
7.6 KB edited online with Bitbucket
26 B
add SOIL_EXPORT to new function,
730 B
- moved includes to soil subdir (use #include <soil/soil.h> from other projects)
1.1 KB
some more fixes needed to work with liminal.

Soil is an efficient C implementation of a memory heap with reflection services, mapped to an auto-expanding memory-mapped file, similar to a persistent application-centric swap file.

The Soil implementation is not finished yet; Linux/mmap based platforms are fully supported, preliminary Windows support has been added. I'm still adding major features which break the file format, so it's probably not a good idea to use Soil productively yet.

Here's a sneak peek of what usage looks like:

// an example declaration for an entity
typedef struct {
    float x,y;
    char name[32];
} Entity;

// open/create volume, reserve 1 GB of addressing space
soil_volume_t *volume = soil_open("demo.soil", 1<<30);
// select volume as current context

// allocate entity and init
Entity *ent = (Entity *)soil_alloc(sizeof(Entity));
memset(ent, 0, sizeof(*ent));
ent->x = 0.0f;
ent->y = 0.5f;
sprintf(ent->name, "Testitem");

Key-Value Store

To allow you finding the root allocations of your program, Soil provides a dictionary (short TOC for table of contents) that resolves a hashed string (usually some sort of namespace) to a block address. The dictionary is used as a key->value store for bootstrapping and resolving types.

// store entity in TOC so we can find it later on
soil_set_key(soil_hash_string("root_entity"), ent);

// close volume. all touched data is flushed back to disk,
// addresses into the volume are now invalid.

// re-open and use
volume = soil_open("test.soil", 1<<30);

// retrieve our pointer
ent = (Entity *)soil_get_key(soil_hash_string("root_entity"));
// ensure it's still our item
assert(!strcmp(ent->name, "Testitem"));

// and free the entity

The advantage of a memory-mapped heap is that you can stop and re-start your program and continue with the same heap you left off with.

Reflection Services

To make data introspection possible, allocated blocks may optionally be tagged with the hashed name of a schema, a so-called "spec"; a spec provides enough information about the data layout of a block to automate instantiation, conversion, copying, visualization or any other kind of editing, comparable to the reflection services the data models of dynamic languages are offering.

// import specs from a volume generated by the soilc
// command-line tool.
soil_volume_t *type_volume = soil_open("types.soil",0);

// types.soil knows about our Entity type, so 
// let's generate a new entity with spec; this one is already
// initialized.
ent = soil_salloc((soil_spec_struct_t *)

If a spec changes and is re-imported, all blocks using the spec are updated to the new format.

C Schema Language

The schema language to declare specs is C. Specs are described as C structs, and can be generated using the soilc command-line tool, which parses C header files, generates spec blocks and stores them in a Soil volume.

// soilc is able to parse this file and create a spec volume 
// from it; C enum and function declarations will be ignored.

// provided by soil, allows declarations to work
// for C compilers and soilc.
#include <soil_spec.h>

// Entity will be added to the spec volume
typedef struct SEntity {
    // traits allow to extend field declarations
    // with contextual information
    float x,y;
    // name is z-terminated
    char name[32];
} Entity;

Managed Pointers

Blocks support a simple reference type that ensures pointers can never become invalid; to use it, replace your pointers with the soil_ref_t structure. Soil updates all references to a block when it is deleted, and allows to query which references point to a particular block.

// assuming soil_ref_t has a registered spec in this volume:

// allocate a sole reference; it could also be part of a struct.
soil_ref_t *ref = (soil_ref_t *)soil_salloc(
    (soil_spec_struct_t *)soil_get_key(soil_hash_string("soil_ref_t")));

// point the reference to entity
soil_ref(ref, ent);
assert(soil_deref(ref) == ent);

// ref should now be the first and only block pointing to ent
void *iter;
assert(soil_get_first_ref(ent, &iter) == ref);
assert(soil_get_next_ref(&iter) == NULL);

// free ent

// ref has implicitly been updated
assert(soil_deref(ref) == NULL);


There is also a more advanced type of reference, sources and sinks, which allow the construction and inspection of more complex one-to-many pointer graphs and trees.

// an example declaration for a node;
typedef struct {
    // two ingoing ports
    soil_sink_t inputs[2];
    // one outgoing port
    soil_source_t output;
} Node;

// we assume that the spec for this type is already imported.
soil_spec_struct_t *NodeSpec = (soil_spec_struct_t *)soil_get_key(soil_hash_string("Node"));

// allocate two new nodes
Node *node1 = (Node *)soil_salloc(NodeSpec);
Node *node2 = (Node *)soil_salloc(NodeSpec);

// link both nodes
soil_link(&node2.inputs[0], &node1.output);
// nodes are now linked
assert(soil_get_source(&node2.inputs[0]) == &node1.output);
assert(soil_get_first_sink(&node1.output) == &node2.inputs[0]);

Runtime Contexts

Soil supports attaching runtime contexts and event handlers, allowing soil blocks to be used as the model part of a MVC application structure. Modules can independently register event callbacks and application resources with blocks that allow them to track whenever a block is instantiated, copied, touched or deleted.

// allocate owner token used as placeholder
static int g_entity_owner;

// event handler for entity events
void on_entity_event(const soil_event_t *event, void *owner, void *context) {
    switch(event->type) {
    // a new entity is being created; this is the only event
    // that will be triggered for specs.
    case SOIL_EVENT_ALLOC: {
        // create an app-specific runtime resource for entity
        DataCache *cache = datacache_new((Entity*)event->ptr);

        // register this handler for the entity,
        // and attach the runtime resource
            on_entity_event, &g_entity_owner, cache);    
    } break;
    // an entity is being deleted
    case SOIL_EVENT_FREE: {
        // delete the app-specific runtime resource
        datacache_delete((DataCache *)context);
    } break;
    // soil_touch() has been called on an entity
    case SOIL_EVENT_DIRTY: {
        // invalidate the cache
        datacache_invalidate((DataCache *)context);
    } break;
    // a block is being referenced
    case SOIL_EVENT_INCREF: break;
    // a block is no longer being referenced
    case SOIL_EVENT_DECREF: break;
    // a source in the block has changed
    case SOIL_EVENT_SOURCE: break;
    // a sink in the block has changed
    case SOIL_EVENT_SINK: break;
    default: break;

void attach_context() {
    // register the handler for the Entity spec type;
    // it will be notified whenever a block with this 
    // spec is being allocated.
        on_entity_event, &g_entity_owner, NULL);