Generate Report from CoverageData?

Issue #405 new
Will Bond
created an issue

I am working with coverage trying to create a small system for myself to aggregate data from multiple operating systems and then generate a report from it. It appears with 4.0 (I've only been working with that) that I can export coverage data via CoverageData and StringIO to save the results in a SQLite database.

The API for CoverageData seems to cover the ability to do handy things like write the data to a file object, read it back in, combine multiple files, map paths, etc. However, it doesn't appear that a report can be generated directly from the CoverageData. Additionally, it doesn't appear that a Coverage object can load a CoverageData object in, only data filenames.

Is there an alternate way (without writing to disk) that I can seed a Coverage object with a CoverageData object (of merged data), or is that possibly an API that could be added?

Comments (9)

  1. Ned Batchelder repo owner

    From your description, it seems like existing .coverage files should be what you want. You can move them around, and then combine them with .combine(), and generate reports from the combined data files. This is all available from the command line if you want.

    Maybe I'm missing an unusual requirement of yours?

  2. Will Bond reporter

    I am using CoverageData with StringIO to store coverage data in a SQLite database. Hence the desire to not write coverage data to the filesystem. CoverageData allows me to load, save and combine without touching the filesystem, but the Coverage class doesn't seem to be able to work with a file object instead of a filename.

  3. Ned Batchelder repo owner

    Just for my own curiosity, can you explain more about the big picture? Are you trying to capture coverage per-test or something like that?

    I see your point, it would be useful to able to add data to a Coverage object from an existing CoverageData object.

  4. Will Bond reporter

    Sure, the big picture is that I am building a tool to capture and aggregate coverage data for Sublime Text packages.

    Since Sublime Text runs on Windows, OS X and Linux, and many packages interact with the system in some way, I wanted to build a simple user experience for aggregating coverage results. Rather than dealing with all sorts of .coverage files from different machines and different packages I was testing, I decided that storing all of the coverage run data in a single SQLite database that I sync across all of my VMs using Dropbox would be a good place to start.

    The coverage data from each run is stored along with the git commit hash of the code being measured, the platform being run on and the version of Sublime Text. Using a few simple SQL statements, I can easily find all of the runs for a given package and commit. Rather than writing all of these coverage results to disk and then loading them back in, I query the database and use StringIO to load the data into CoverageData and merge it all together.

    For now, I am writing the final CoverageData object to disk and then loading it via Coverage in order to have something working. It just seemed like a logical extension of the CoverageData/Coverage API to push data into Coverage via CoverageData, rather than just pull it out.

  5. Ned Batchelder repo owner

    Thanks. I'd be concerned about a SQLite file being synched by Dropbox: will it do the right thing if the db is updated in more than one place at once? Also, this points out that PathAliases needs to be documented...

  6. Will Bond reporter

    Well, in my situation I'll be running the tests by hand on each machine - I don't currently have a way to automate all of this from within the Sublime Text UI, and the Python environment there is a bit different than a command line (for instance, no TTY on stdin, sublime APIs, etc). So I'll be switching from one machine to the other to invoke the tests, and can confirm that the DB has synced before I invoke the tests.

    In a more robust environment where I could automate things, I imagine I would use something like Postgres, and that would be ok since I would be maintaining the whole VM infrastructure anyway.

    I did notice the PathAliases class referenced in the CoverageData documentation, hence I figured you meant for it to be public, but I was unable to find any docs for it. I guess I'm so used to digging into source for various libraries that it didn't strike me to open an issue to add docs for that.

  7. Log in to comment