The Saturnalia database is optimized for aggregating, storing, and displaying a large number of scalar samples for a large number of different metrics (ie: sources of periodic measurement).
The database (like other components) is written with a Crash-Only Design. The design includes optimizations for storage size and read/write latency from disk.
FIXME: Make this overview more comprehensive.
Here is a quick overview of some existing optimizations:
Minimized Disk Seeks
FIXME: Define chunks and tiles.
List of chunks and there time ranges. Chunks are 100M in size so on a 1T disk we will only have 10,000 chuncks before the disk is full. If each chunk index record is a 100 byte file name + two int64s 116 bytes per entry or 1.6M max for the hole thing on a full 1T disk.
Chunks, finding tiles:
- Hash of metrics to tile index. Assume two seeks to find the right entry in the has table.
- entry index is a array of time with one entry for each tile in the time range covered by this chunk. See to find tiles is one.
- One more seek to get the tiles.
In this way we have four seeks to get to a tile.