Using Fudge decorators more than once breaks call detection

Issue #4 resolved
Jeff Forcier
created an issue

First: thanks for Fudge! Love the API.

I've found an issue and am not 100% sure if it's something that can be addressed in Fudge or if I have to structure my tests differently. FWIW I'm using Fudge 0.9.2 and Nose 0.10.4 to run my tests.

In a test module I have two (soon to be more) test functions. Both of them are wrapped with @with_fakes and @with_patched_object; and both calls to @with_patched_object create new Fake objects using expect_call and times_called().

See this only slightly abstracted example for what I mean: http://dpaste.com/23853/

The problem crops up when I have two+ tests like that example (both of which make use of expect_call/times_called()): my module loads, and -- before any tests are run! -- both Fake objects are created, loading their calls into the Registry's list of expected calls. Meaning the Registry's call list now holds all calls, one for each test.

Finally, the linchpin is that when the first test executes and its @with_fakes calls Registry.verify(), it's considering both expected calls, not just the one intended for that one test. Thus, I get failures, since the 'extra' calls weren't called or weren't called enough times or etc.

I can't think of a good way around this that does not involve forgoing use of the (very helpful) decorators. Am I missing something?

Comments (9)

  1. Jeff Forcier reporter

    FWIW, avoiding the decorators does avoid the issue, as expected -- i.e. anything where the Fake object is created at time of test run instead of module load, will work OK.

    Needed to tighten up my test code anyways; I've now got a single function that does clear_calls(), patch_object(), verify(), restore() etc (all surrounding my few lines of actual test code), and it's used once per test via Nose's tests-via-generators functionality. Works pretty well.

    At any rate, I think amending http://farmdev.com/projects/fudge/using-fudge.html#a-simple-test-case with a note about this "gotcha" might be wise -- something warning folks that combining the decorators with expects_call=True will be problematic, maybe.

  2. Kumar McMillan repo owner

    I see how that can be confusing. A few paragraphs down is an example of a complete module which suggests to use one Fake per test module. For what you're doing I've been just using a TestCase in my own tests and calling clear_expectations() in tearDown(). But maybe having a decorator that calls clear_expectations automatically isn't such a bad idea.

  3. Jeff Forcier reporter

    Well, I had experimented with adding clear_expectations() as teardown for the methods; the problem is that tests will still fail as the expected-call list has already been messed up before the first test is even run.

    In other words, this is sort of the opposite problem to what clear_expectations() was added for: it's not that a test is polluting the expectations list *while it runs* (which clear_expectations() handily solves) -- it's that said list is polluted *even before tests run*. There's simply no way around it but to ensure your Fakes are initialized at runtime instead, which is what I'm doing now.

  4. Kumar McMillan repo owner

    ohhh, right, I see. You could use the with statement :

    @with_fakes
    def test_one():
        with patched_context('package.module', 'my_function',
                Fake('my_function', expect_call=True).times_called(2)):
            function_which_calls_my_function()
            another_function_calling_my_function()
    
  5. Jeff Forcier reporter

    Oh hey, I totally forgot about the context manager. That will help somewhat, thanks for the reminder.

    Still noodling around and adding tests; I'll let you know if I come up with any bright ideas for how to modify Fudge to make this less of a pain point. Regardless, I do still think a single extra sentence/note in the docs might be a good idea.

  6. Jeff Forcier reporter

    D'oh, spoke too soon. Context manager doesn't actually help much; I need both patch restore and expectation clearing in a finally() block.

    This is what currently works for me (and the function is being used via Nose generator-driven tests, but I believe would also apply to an individual test method/function):

    def do_my_test(arguments, num_calls):
        # Clear Fudge call stack
        clear_calls()
        # Patch my_function with Fake obj set to expect arguments
        patched_func = patch_object('package.module', 'my_function',
            Fake('my_function', expect_call=True).times_called(num_calls)
        )
        try:
            # setup here
            for arg in arguments:
                package.module.my_function() # perform actual test here
            # Verify expected calls matches up with actual calls
            verify()
        finally:
            # Restore my_function()
            patched_func.restore()
            # Clear expectation stack
            clear_expectations()
    

    (How'd you get your paste to be highlighted/have line numbers?)

  7. Anonymous

    Thanks! I've not had the time to dig into this since (i.e. I just kept going with what I had :D) but I did notice 1.0 had come out recently. Will make myself a note to check out @patch sometime.

    Keep up the good work!

  8. Log in to comment