cannot cover multiprocessing.Pool

Issue #338 duplicate
Former user created an issue

Related to #117, I have a program that uses multiprocessing.Pool, but when I use coverage run or any other method of calling coverage, the multiprocessing function doesn't get listed in coverage.

I've done some looking around, and most solutions relate to multiprocessing.Process instead. For the record, I understand that a pool is a set of processes, but I'm not getting anything out of fixes to multiprocessing.Process.

I've included my test program. This program uses solutions that use fork() instead of multiprocessing.Pool, inspired by #56, but those have been commented out for purposes of this issue.

Comments (6)

  1. Former user Account Deleted

    No, it hasn't. I've struggled with it for almost a month and it hasn't done a thing.

  2. Ned Batchelder repo owner

    @mezarim The attached file, which is your program, combined with #117's monkeypatching, works for me. Can you try it with your program? I had forgotten earlier to use "coverage combine" before reporting.

    $ coverage run
    (75853, 0)
    (75853, 1)
    (75853, 9)
    (75853, 16)
    (75853, 25)
    (75853, 36)
    (75854, 4)
    (75853, 49)
    (75853, 81)
    (75854, 64)
    (75853, 100)
    (75853, 121)
    (75853, 169)
    (75854, 144)
    (75853, 196)
    (75853, 256)
    (75854, 225)
    (75853, 289)
    (75854, 324)
    (75853, 361) warning: No data was collected.
    $ coverage combine
    $ coverage report -m
    Name           Stmts   Miss  Cover   Missing
    --------------------------------------------      15      0   100%

    #!/usr/bin/env python
    import multiprocessing
    import os
    # monkey patch multiprocessing to enable  code coverage
    def coverage_multiprocessing_process(): # pragma: no cover
            import coverage as _coverage
        from coverage.collector import Collector
        from coverage import coverage
        # detect if coverage was running in forked process
        if Collector._collectors:
            original = multiprocessing.Process._bootstrap
            class Process_WithCoverage(multiprocessing.Process):
                def _bootstrap(self):
                    cov = coverage(data_suffix=True)
                        return original(self)
            return Process_WithCoverage
    ProcessCoverage = coverage_multiprocessing_process()
    if ProcessCoverage:
        multiprocessing.Process = ProcessCoverage
    def func(x):
        pid = os.getpid()
        return pid, x*x
    pool = multiprocessing.Pool(3)
    inputs = range(20)
    outputs = pool.imap_unordered(func, inputs)
    for i in outputs:
        print i
  3. Ned Batchelder repo owner

    If this works, I can figure out how best to add this support to, so that people don't have to manually monkey-patch multiprocessing.

  4. Former user Account Deleted

    I've just tested the modified monkey patch and it works! Thanks to you and schettino72 for your help.

  5. Log in to comment