Use SQLite as Formaline back-end

Issue #871 new
Erik Schnetter created an issue

Use SQLite as Formaline back-end instead of storing the information in ASCII files.

Keyword:

Comments (3)

  1. Roland Haas
    • removed comment

    Using SQLite (or any non-ascii data storage) requires extra tools to be present to access the information. Therefore my preference would be to avoid them unless the amount or size of data clearly requires it.

  2. Erik Schnetter reporter
    • removed comment

    ASCII data also require tools to access the data. Opening a file in an editor is fine, but is not useful for any kind of post-processing. SQLite is a standard tool that a really easy to install and use. XML would be another approach, but is significantly more difficult to process. (For example, to modify the state written into a file, one has to read the file, parse all entries, modify the internal data structure, and then write out the file again -- presumably much slower than just a call to SQLite.)

    What kinds of analysis post-processing are you performing on Formaline's output? Are you using particular tools? Have you experimented with saving or combining these output files?

  3. Roland Haas
    • removed comment

    I am not doing anything with the Formaline output other than looking at it. I am speaking out of my experience when Firefox switched from storing cookies in a plain text file to storing them in an sqlite database (and I wanted to transfer cookies from firefox to wget). Yes the tools are in principle available but they are still harder to learn than an editor or a set of perl/awk/python regular expressions to extract a single item. Doing large amounts of data manipulation might be quite different. Jus speaking for myself: I have no experience at all using sqlite (either its command line tools or SQL itself) but feel confident parsing even moderately complicated ascii files with tools I feel comfortable with. So my point was not so much as to what is technically better (a structured database clearly beats an unstructured ASCII file) but more of what I myself would expect user to be able to work with. I have no real strong preference since I can just continue to more or less ignore these file the way I do right now :-)

  4. Log in to comment