Issue #149 resolved

coverage gevent looks broken

mardiros
created an issue

If you run the attachement test_gevent.py, coverage doesn't mark a few line as executed.

Comments (20)

  1. krw1243

    Hi Ned, wondering if you could describe what behavior the newbrough branch sacrifices for someone who doesn't have a deep understanding of coverage.py and gevent internals, so we can decide if this fix would be appropriate for our project (which uses gevent)?

  2. Vadim Fint

    This monkey patch fixed coverage to work with gevent for me:

    def patch_coverage_for_gevent():
        from coverage import collector as _collector, control as _control
        import collections as _collections
        import gevent as _gevent
        _PyTracer = _collector.PyTracer
    
        class DataStack(object):
            def __init__(self):
                self.__data = _collections.defaultdict(list)
    
            def __idx(self):
                return hash(_gevent.getcurrent())
    
            def pop(self):
                return self.__data[self.__idx()].pop()
    
            def append(self, value):
                return self.__data[self.__idx()].append(value)
    
        class PyTracer(_PyTracer):
            def __init__(self):
                _PyTracer.__init__(self)
                self.data_stack = DataStack()
    
        _collector.PyTracer = PyTracer
    
        # Patch coverage, so it uses timid simple tracer by default after monkey patching
        _coverage = _control.coverage
    
        class coverage(_coverage):
            def __init__(self, *args, **kwargs):
                kwargs.setdefault('timid', True)
                super(coverage, self).__init__(*args, **kwargs)
    
        __import__('pyjack').replace_all_refs(_coverage, coverage)
    
  3. Ned Batchelder repo owner

    Vadim, thanks, this will help as I evaluate solutions. Just one question: why did you use __import__ to import pyjack, instead of simply importing it?

  4. Vadim Fint

    "Import statement" at the end of meth looks ugly while "import statement" on top will break all monkey patching if pyjack is not available

    But anyway, this is more strange for me:

    (.venv)[trunk] mcsl .venv/lib/python2.7/site-packages/coverage % flake8 . | wc -l
    352
    

    You are right, it is strange what people value in code... :-

  5. Ned Batchelder repo owner

    Those are better reasons than "one line of code." :)

    And it's even worse on my machine for some reason:

    (coverage)~/coverage/trunk $ flake8 . | wc -l
        1874
    
  6. Brian Wylie

    The workbench project really needs gevent coverage https://github.com/SuperCowPowers/workbench. The project used to use nosetests and we pulled in https://github.com/newbrough/coverage.git. With that configuration we were getting about 94% coverage. We upgraded to pytest and the latest version complains/won't run with the github version of coverage above. So we switched to using the 'stock' coverage module and now we have a coveralls badge that is RED and kinda embarrassing :)

    Would it be possible to contract someone to put in the support for gevent? Elance contract perhaps?

    Happy to support this work because we'd really like to have our coverage badge not be red all the time, kinda erodes confidence in an embryonic project like ours.. :)

    Cheers, -bri

  7. Brian Wylie

    Hi Ned,

    I also tried Vadim's patch and it didn't work for me either. Vadim said he wasn't surprised as these things go I think it was more a quick patch on a particular version than meant to be anything more long lasting. I ran the test 'test_gevent.py' above.. and after some random tinkering noticed that I would occasionally get a thread KeyError exception thrown at the very end (was playing with gevent monkey patching). I found this thread on SO http://stackoverflow.com/questions/8774958/keyerror-in-module-threading-after-a-successful-py-test-run. I think it might be a good place to start... to be honest it's deeper than my knowledge, so I'll just have to hope the pointer helps. :)

    Replication Instructions:

    1) Add this as the first line of the script above

    from gevent import monkey; monkey.patch_all()
    

    2) Also note that newer versions of gevent no longer have the gevent.shutdown() call, so that can just be removed.

    Just wanted to say that the workbench project ^appears^ to be getting exactly correct coverage with coveragepy 4.0a0 version (Github).. it's a complicated project (client/server) with tons of gevent calls (using ZeroRPC).. so that is a data point that perhaps the 4.0a0 version of coverage IS working on gevent calls, there's just some weird corner case at the very end of the program... perhaps with this thread KeyError exception getting thrown.

  8. Log in to comment