1. Ned Batchelder
  2. coverage.py


coverage.py / TODO.txt

Coverage TODO

* 3.1

- Python 3.1 support.
- bitvector in trace extension.

* Speed

+ C extension collector
- Ignore certain modules
- Record linenos rather than (file,lineno) pairs in tracer.
x Tricky swapping of collector like figleaf, pycov, et al. (Don't need to do
    this with C collector).
- Seems like there should be a faster way to manage all the line number sets in
- If tracing, canonical_filename_cache overlaps with should_trace_cache.  Skip
    canonical_filename_cache. Maybe it isn't even worth it...

* Accuracy

- Record magic number of module to ensure code hasn't changed
- Record version of coverage data file, so we can update what's stored there.
- Record options in coverage data file, so multiple runs are certain to make
    sense together.
- Do I still need the lines in annotate_file that deal specially with "else"?

* Power

- API for getting coverage data.
- Instruction tracing instead of line tracing.
- Path tracing (how does this even work?)
- Branch coverage
    Titus' idea:
        1: if a:
        2:     b = 2
        3: c = 3
    if the coverage data shows 1,2,3, it was if-then.  if it's 1,3, then the
    missing else was executed.
- Count execution of lines
- Track callers of functions (ala std module trace)
- Method/Class/Module coverage reporting.
- .coverage files that can be kept separate, rather than accumulated.

* Convenience

- Why can't you specify execute (-x) and report (-r) in the same invocation?
    Maybe just because -x needs the rest of the command line?
- How will Coverage package install over coverage.py module?
- Support 2.3 - 3.0?
    - Explicitly set pickle protocol to 2.
- Use a .coveragerc file to control Coverage without the programmatic API.
- An inference mode that marks lines as executed if they "must have been" executed:
    class definitions, etc, when coverage is started after the class is defined.
- Different categories of exclude pragma? So you can enable and disable them
    from the command line, to reconsider exclusions.
- Reporting on files never touched by coverage (package completeness)
- Why can't a morf also be a string, the name of a module?
- ignore by module as well as file?

* Beauty

+ HTML report
    - Package navigation.
    - Rolled-up statistics.
    - Syntax coloring in HTML report.
    + Dynamic effects in HTML report.
    + Footer in reports pointing to coverage home page.
    + Baseline grid for linenumber font.
    + Separate out css and HTML.
    - Does it work right with utf-8 source files? http://www.python.org/dev/peps/pep-0263/
    - Colored bars indicating coverage per file.

* Community

+ New docs, rather than pointing to Gareth's
    + Min python version is 2.3.
    - Three phases of work:
        - Collection
        - Analysis
        - Reporting
    - Distinction between:
        - ignore (files not to collect)
        - exclude (lines not to report as missed)
        - omit (files not to report)
    - Changes from coverage 2.x:
        - Bare "except:" lines now count as executable code.
        - Double function decorators: all decorator lines count as executable code.
    - Document the .coverage file format.
    + HTML reporting.
+ Be sure --help text is complete (-i is missing).
+ Host the project somewhere with a real bug tracker: bitbucket.org
+ Point discussion to TIP
- PEP 8 compliance?
- References between pages are off:
    - They have <em> tags around them.
    - They use #anchors that don't survive the px->html conversion.

* Programmability

+ Don't use sys.exit in CoverageScript.
+ Remove singleton
    + Initialization of instance variables in the class.

* Installation

- pip can't install it: it reads the coverage html page, and finds the kit link,
    but then can't handle the root-relative link.

* Modernization

+ Decide on minimum supported version
    + 2.3
    + Get rid of the basestring protection
    + Use enumerate
    + Use sets instead of dicts
- Switch from getopt to optparse.
+ Get rid of the recursive nonsense.
+ Docstrings.
+ Remove huge document-style comments.
- Better names:
    + self.cache -> self.cache_filename -> CoverageData.filename
    + self.usecache -> CoverageData.use_file
- More classes:
    - Module munging
    + Coverage data files
+ Why are some imports at the top of the file, and some in functions?
+ Get rid of sys.exitfunc use.
+ True and False (with no backward adaptation: the constants are new in 2.2.1)
+ Get rid of compiler module
    + In analyzing code
    + In test_coverage.py
+ Style:
    + lineno
    + filename

* Correctness

- What does -p (parallel mode) mean with -e (erase data)?

* Tests

+ Switch to a real test runner, like nose.
+ Test both the C trace function and the Python trace function.
- Tests about the .coverage file.
- Tests about the --long-form of arguments.
+ Tests about overriding the .coverage filename.
- Tests about parallel mode.
+ Tests about assigning a multi-line string.
- Tests about tricky docstrings.
- Coverage test Coverage!
- Tests that tracing stops after calling stop()
- More intensive thread testing.
x Tests about the "import __main__" in cmdline.py
- What happens if the -x script raises an exception?
- Test that the kit has all the proper contents.

* Release checklist

- Test on:
    - Windows
    - Ubuntu
    - Mac
    - Pythons 2.3, 2.4, 2.5, 2.6
- Version number in coverage/__init__.py:__version__
- Update CHANGES, including release date.
- Update docs
    - Generate new sample_html to get the latest, incl footer version number:
        cd C:\ned\cog\trunk
        rmdir/s/q htmlcov
        coverage -e -x cogapp\test_cogapp.py CogTestsInMemory
        coverage -b -i -d htmlcov
        copy htmlcov\*.* C:\ned\coverage\trunk\doc\sample_html
    - Build and publish docs:
        $ make px publish
- Kits:
    - source .tgz and windows .exe for each Python
        - $ allkits.cmd
- Update PyPi:
    - $ make pypi
    - Visit http://pypi.python.org/pypi?%3Aaction=pkg_edit&name=coverage and show/hide the proper versions.
    - Upload kits
- Tag the tree
    - hg tag -m "Coverage 3.0.1" coverage-3.0.1
- Update nedbatchelder.com
    - Blog post?
- Update bitbucket:
    - Issue tracker should get new version number in picker.