Parallelize HTML report generation

Issue #249 closed
Brett Cannon created an issue

I don't know how feasible it is, but would it be at all possible to parallelize the creation of the HTML report in some way? Using something like concurrent.futures would simplify the work and help make it cross-version supported.

Comments (6)

  1. Ned Batchelder repo owner

    Sounds intriguing. concurrent.futures is new in 3.2, I guess I could use the backport on 2.7. I'm not sure if the reporting process is i/o- or cpu-bound, which would affect whether or not we could parallelize it effectively.

  2. Brett Cannon reporter

    You could even stub out the API since it's fairly simple when in the map() case.

    In terms of measuring where the bottleneck is, trying to generate an HTML report from the stdlib is a good test case; it currently takes several hours to generate an HTML report.

  3. Ned Batchelder repo owner

    Brett, is this still true that it takes several hours to generate the HTML report for the stdlib? In another ticket filed around the same time, you thought it may have been user error.

  4. Brett Cannon reporter

    Not sure really since I hacked around it by using PyPy and tweaking coverage to generate coverage for Python 3 source while using a Python 2 interpreter to generate the report.

  5. Log in to comment