Different coverage percent

Issue #586 closed
Former user created an issue

Between gitlab CI and my local machine from the same source code. I'm using nose to discover test in django.

In my local machine

$ ./manage.py test
nosetests --with-coverage --cover-package=page,user_profile --cover-inclusive --verbosity=1
Creating test database for alias 'default'...
...........
Name                       Stmts   Miss  Cover   Missing
--------------------------------------------------------
page/__init__.py               0      0   100%
page/admin.py                  0      0   100%
page/apps.py                   3      0   100%
page/models.py                 0      0   100%
page/urls.py                   4      0   100%
page/views.py                  5      0   100%
user_profile/__init__.py       0      0   100%
user_profile/adapter.py        6      0   100%
user_profile/admin.py          3      3     0%   1-5
user_profile/apps.py           3      0   100%
user_profile/forms.py         51      0   100%
user_profile/models.py        10      8    20%   1-20, 30
user_profile/tasks.py         10      0   100%
user_profile/urls.py           4      0   100%
user_profile/views.py         34      0   100%
--------------------------------------------------------
TOTAL                        133     11    92%
----------------------------------------------------------------------
Ran 11 tests in 5.493s

OK
Destroying test database for alias 'default'...

In gitlab CI

$ ./manage.py test
...........
Name                       Stmts   Miss  Cover   Missing
--------------------------------------------------------
page/__init__.py               0      0   100%
page/admin.py                  0      0   100%
page/apps.py                   3      0   100%
page/models.py                 0      0   100%
page/urls.py                   4      0   100%
page/views.py                  5      0   100%
user_profile/__init__.py       0      0   100%
user_profile/adapter.py        6      2    67%   8-9
user_profile/admin.py          3      3     0%   1-5
user_profile/apps.py           3      0   100%
user_profile/forms.py         51      0   100%
user_profile/models.py        10      9    10%   1-30
user_profile/tasks.py         10      0   100%
user_profile/urls.py           4      0   100%
user_profile/views.py         34      0   100%
--------------------------------------------------------
TOTAL                        133     14    89%
----------------------------------------------------------------------
Ran 11 tests in 0.567s

OK
DEBUG from local_settings.py was not imported
settings from local_settings.py was not imported
nosetests --with-coverage --cover-package=page,user_profile --cover-inclusive --verbosity=1
Creating test database for alias 'default'...
Destroying test database for alias 'default'...

Comments (16)

  1. Ned Batchelder repo owner

    This looks to me like a difference in your environments. Notice that the lines run in models.py are different. Perhaps you have different versions of Django or your test runner plugins?

  2. Ned Batchelder repo owner

    Without a link to the repo, or some other way to reproduce the problem, I can't help. Feel free to re-open the issue if you have more information about what coverage.py is doing wrong.

  3. Ned Batchelder repo owner

    Do you have more information about how I can reproduce this? Have you examined the lists of the exact versions of all of the third-party packages in each environment?

  4. Former user Account Deleted

    @ned Both are running on python 3.6. I check the package version from pip freeze both have the same package. The output of the CI below

    DEBUG from local_settings.py was not imported
    settings from local_settings.py was not imported
    

    It's just the output from my settings.py. I think it should not impact the testing.

    About output below I think it executes the same way as in my local machine if the command nosetests is executed later how can the test report show up and also if the test database was created after test, some of tests would fail.

    nosetests --with-coverage --cover-package=page,user_profile --cover-inclusive --verbosity=1
    Creating test database for alias 'default'...
    Destroying test database for alias 'default'...
    

    Here is the link to the repo it requires some setup but if you know django you can use sqlite instead of postgres and disable whole site cache (using memcached) in the settings.py. All the package dependency is in requirement/dev file.

  5. Ned Batchelder repo owner

    Could you paste here the output of pip freeze in both environments? I really don't see how this can be coverage.py's fault. Also, your gitlab link doesn't have a repository tab in it?

  6. Ned Batchelder repo owner

    Your local machine is claiming that nothing is run in user_profile/models.py except for line 25. That's not possible. Be sure you are running in a clean environment, with no .coverage or .pyc files left from previous runs.

  7. Former user Account Deleted

    After delete .coverage and __pycache__ in user_profile it seem that those are the cause the result is now in sync with the CI.

    $ ./manage.py test
    nosetests --with-coverage --cover-package=page,user_profile --cover-inclusive --verbosity=1
    Creating test database for alias 'default'...
    ...........
    Name                       Stmts   Miss  Cover   Missing
    --------------------------------------------------------
    page/__init__.py               0      0   100%
    page/admin.py                  0      0   100%
    page/apps.py                   3      0   100%
    page/models.py                 0      0   100%
    page/urls.py                   4      0   100%
    page/views.py                  5      0   100%
    user_profile/__init__.py       0      0   100%
    user_profile/adapter.py        6      2    67%   8-9
    user_profile/admin.py          3      3     0%   1-5
    user_profile/apps.py           3      0   100%
    user_profile/forms.py         51      0   100%
    user_profile/models.py        10      9    10%   1-30
    user_profile/tasks.py         10      0   100%
    user_profile/urls.py           4      0   100%
    user_profile/views.py         34      0   100%
    --------------------------------------------------------
    TOTAL                        133     14    89%
    ----------------------------------------------------------------------
    Ran 11 tests in 5.450s
    
    OK
    Destroying test database for alias 'default'...
    

    One last question any advices to run clean test report every time on my local machine? Thank you :)

  8. Log in to comment