Issue #67 resolved

Unit test subprocess failure doesn't cause unit test failure

Franz Schrober
created an issue

I've just installed the wrong version of SDL2 to cause a failure during the tests. This was expected to also cause an failure code as result. This is the output:

$ python -B -m sdl2.test.util.runtests -s -t70
Executing tests from [audio_test.py]... OK
Executing tests from [clipboard_test.py]... OK
Executing tests from [cpuinfo_test.py]... OK
Executing tests from [endian_test.py]... OK
Executing tests from [error_test.py]... OK
Executing tests from [events_test.py]... OK
Executing tests from [filesystem_test.py]... OK
Executing tests from [hints_test.py]... OK
Executing tests from [joystick_test.py]... OK
Executing tests from [keyboard_test.py]... OK
Executing tests from [log_test.py]... OK
Executing tests from [mouse_test.py]... OK
Executing tests from [pixels_test.py]... OK
Executing tests from [platform_test.py]... OK
Executing tests from [power_test.py]... OK
Executing tests from [rect_test.py]... OK
Executing tests from [render_test.py]... OK
Executing tests from [rwops_test.py]... OK
Executing tests from [sdl2ext_algorithms_test.py]... OK
Executing tests from [sdl2ext_array_test.py]... OK
Executing tests from [sdl2ext_color_test.py]... OK
Executing tests from [sdl2ext_draw_test.py]... OK
Executing tests from [sdl2ext_ebs_test.py]... OK
Executing tests from [sdl2ext_events_test.py]... OK
Executing tests from [sdl2ext_font_test.py]... OK
Executing tests from [sdl2ext_gui_test.py]... OK
Executing tests from [sdl2ext_image_test.py]... OK
Executing tests from [sdl2ext_particles_test.py]... OK
Executing tests from [sdl2ext_pixelaccess_test.py]... OK
Executing tests from [sdl2ext_resources_test.py]... OK
Executing tests from [sdl2ext_sprite_test.py]... OK
Executing tests from [sdl2ext_test.py]... OK
Executing tests from [sdl2ext_window_test.py]... OK
Executing tests from [sdl_test.py]... OK
Executing tests from [sdlgfx_test.py]... OK
Executing tests from [sdlimage_test.py]... OK
Executing tests from [sdlmixer_test.py]... OK
Executing tests from [sdlttf_test.py]... OK
Executing tests from [shape_test.py]... OK
Executing tests from [surface_test.py]... OK
Executing tests from [syswm_test.py]... OK
Executing tests from [timer_test.py]... OK
Executing tests from [version_test.py]... ERROR
Running tests [ 6 / 6 ] ...           
======================================================================
-- Statistics --
======================================================================
Python:         /usr/bin/python
Options:
                'filename' = '/tmp/pysdl2/sdl2/test/util/../version_test.py'
                'subprocess' = 'False'
                'timeout' = '70'
                'random' = 'False'
                'seed' = 'None'
                'verbose' = 'False'
                'excludetags' = '['interactive']'
Time taken:     0.000 seconds
Tests executed: 6 
Tests OK:       5 
Tests SKIPPED:  0 
Tests ERROR:    0 
Tests FAILURE:  1 
Failures:

----------------------------------------------------------------------
FAILURE: test_SDL_GetVersion (sdl2.test.version_test.SDLVersionTest)
======================================================================
Traceback (most recent call last):
  File "/tmp/pysdl2/sdl2/test/version_test.py", line 22, in test_SDL_GetVersion
    self.assertEqual(v.patch, 3)
AssertionError: 2 != 3


Executing tests from [video_test.py]... OK
$ echo $?
0

As comparison without subprocesses:

$ python -B -m sdl2.test.util.runtests        
/tmp/pysdl2/sdl2/test/util/../sdl2ext_pixelaccess_test.py:55: ExperimentalWarning: 'pixels3d' is in an experimental state.
  nparray = sdl2ext.pixels3d(sprite)
Running tests [ 677 / 677 ] ...
======================================================================
-- Statistics --
======================================================================
Python:         /usr/bin/python
Options:
                'filename' = 'None'
                'subprocess' = 'False'
                'timeout' = '70'
                'random' = 'False'
                'seed' = 'None'
                'verbose' = 'False'
                'excludetags' = '['interactive']'
Time taken:     158.484 seconds
Tests executed: 677 
Tests OK:       468 
Tests SKIPPED:  208 
Tests ERROR:    0 
Tests FAILURE:  1 
Failures:

----------------------------------------------------------------------
FAILURE: test_SDL_GetVersion (sdl2.test.version_test.SDLVersionTest)
======================================================================
Traceback (most recent call last):
  File "/tmp/pysdl2/sdl2/test/util/../version_test.py", line 22, in test_SDL_GetVersion
    self.assertEqual(v.patch, 3)
AssertionError: 2 != 3
$ echo $?
1

Comments (1)

  1. Log in to comment