(From DRMacIver) Here's a reproduction for the problem we discussed on IRC.
I've attached a file that shows significant performance loss running under coverage and modest performance loss under 4.0a5 vs 3.7.1
These are run against 32a214a141bbc81bd624890216899261745f8588 of firstname.lastname@example.org:DRMacIver/hypothesis.git. Note that if you run against 1.2.1 this test has significantly different performance characteristics (it will be much faster).
Command I used to run, from the root of a hypothesis checkout:
With coverage: time PYTHONPATH=src python -m coverage run --branch --rcfile=/dev/null test_coverage_perf.py
Without coverage: time PYTHONPATH=src python test_coverage_perf.py
Numbers are about as variable as you would expect from this extremely shonky benchmarking method, but roughly what I'm seeing is:
Without coverage: ~14s With coverage 3.7.1: ~35s With coverage 4.0a5: ~50s
This is running on CPython 3.3.6 on an oldish laptop running Linux Mint. Coverage was using CTracer in both instances.
Let me know if there's anything I can do to help with debugging this or if you need more information.