Test failure on Ubuntu build

Issue #155 resolved
Thomas Kluyver
created an issue

From the Launchpad buildbot that I'm trying to set up, there's a test failure like this:

BUGGY TEST RESULTS EVAL:
 {'test.image__save_gl_surface_test': {'errors': [('GL_ImageSave.test_image_save_works_with_opengl_surfaces',
                                                  'Traceback (most recent call last):\n  File "/build/buildd/pygame-1.9.2~pre~ubuntu+2250+2~raring1/test/image__save_gl_surface_test.py", line 29, in test_image_save_works_with_opengl_surfaces\n    screen = pygame.display.set_mode((640,480), OPENGL|DOUBLEBUF)\nerror: OpenGL not available\n',
                                                  '/build/buildd/pygame-1.9.2~pre~ubuntu+2250+2~raring1/test/image__save_gl_surface_test.py',
                                                  '29')],
                                      'failures': [],
                                      'num_tests': 1,
                                      'output': 'E\n======================================================================\nERROR: GL_ImageSave.test_image_save_works_with_opengl_surfaces\n----------------------------------------------------------------------\nTraceback (most recent call last):\n  File "/build/buildd/pygame-1.9.2~pre~ubuntu+2250+2~raring1/test/image__save_gl_surface_test.py", line 29, in test_image_save_works_with_opengl_surfaces\n    screen = pygame.display.set_mode((640,480), OPENGL|DOUBLEBUF)\nerror: OpenGL not available\n\n----------------------------------------------------------------------\nRan 1 test in 0.004s\n\nFAILED (errors=1)\n',
                                      'tests': {'GL_ImageSave.test_image_save_works_with_opengl_surfaces': {'error': ('GL_ImageSave.test_image_save_works_with_opengl_surfaces',
                                                                                                                      'Traceback (most recent call last):\n  File "/build/buildd/pygame-1.9.2~pre~ubuntu+2250+2~raring1/test/image__save_gl_surface_test.py", line 29, in test_image_save_works_with_opengl_surfaces\n    screen = pygame.display.set_mode((640,480), OPENGL|DOUBLEBUF)\nerror: OpenGL not available\n',
                                                                                                                      '/build/buildd/pygame-1.9.2~pre~ubuntu+2250+2~raring1/test/image__save_gl_surface_test.py',
                                                                                                                      '29'),
                                                                                                            'stderr': '',
                                                                                                            'stdout': '',
                                                                                                            'tags': ['slow',
                                                                                                                     'display'],
                                                                                                            'times': [0.0038089752197265625]}}}}
[?1000l[?25h[?0c

Traceback (most recent call last):
  File "run_tests.py", line 9, in <module>
    import test.__main__
  File "/build/buildd/pygame-1.9.2~pre~ubuntu+2250+2~raring1/test/__main__.py", line 131, in <module>
    run(*args, **kwds)
  File "/build/buildd/pygame-1.9.2~pre~ubuntu+2250+2~raring1/test/test_utils/run_tests.py", line 275, in run
    test_results = get_test_results(raw_return)
  File "/build/buildd/pygame-1.9.2~pre~ubuntu+2250+2~raring1/test/test_utils/test_runner.py", line 192, in get_test_results
    return eval(test_results.group(1))
  File "<string>", line 17
    [?1000l[?25h[?0c
    ^
SyntaxError: invalid syntax

The same happens on both i386 and x64 builds. The full log is available here

Comments (14)

  1. Thomas Kluyver reporter

    Experimenting on my own machine, that call fails messily when it's not in a graphical display. If there's a simple way to detect that case, we can skip the relevant module - it only has the one test.

    It also writes some junk directly to stdout, which causes the problem when the parent process tries to eval() the output. Maybe we should bracket the test results with markers at the start and end, instead of just the end.

  2. Thomas Kluyver reporter

    Is there a good way to check for this? Should the master test process just call pygame.display.set_mode() with the OPENGL flag, and see if there's an exception?

  3. René Dudfield

    Sounds like a good way to test if OpenGL is supported, and if so use that.

    I added issue #156 about the test runner failing because of prints... I saw this on the ubuntu travis CI build bots too. If you know a fix, feel free to add it :)

  4. René Dudfield

    I added an 'opengl' test tag to that test. So we can exclude all OpenGL using tests by adding that tag to the tests that use it.

    Then you can disable those tests via some test runner arguments...

    run_tests.py --exclude opengl,display
    

    Would that be possible with the launchpad builds?

    helpful env variables

    For the travis build I added the following env variables:

    • SDL_VIDEODRIVER=dummy
    • SDL_AUDIODRIVER=disk

    I hope they allow running more tests, and maybe they are useful for the launchpad build as well.

  5. Thomas Kluyver reporter

    I've made a couple of adjustments. We still have failures in:

    • Floating point arithmetic. I think numpy has something like almost_equal() to check floating point numbers, maybe we can use this.
    • Fonts
    • Audio (on the Launchpad build servers - they don't appear to have an audio device)
  6. René Dudfield
    • Floating point errors in the math module and tests need review. It could be as simple as using an epsilon (almost_equal) or it could be a problem with the code itself. Getting these errors in the travis ci build too.
    • looks like mostly sysfont issues, or perhaps a default font is not correct, and perhaps rendering differences (freetype can give different results depending on version and options used in compiling). Getting these errors in the travis ci build too.
    • Setting the environment variable: SDL_AUDIODRIVER=disk makes pygame use the disk based audio code. So it just dumps audio to a file, rather than sending it to a real audio driver. This fixed the audio test failures on travis ci.
  7. Thomas Kluyver reporter
    • changed status to open

    It looks like that was premature, I'm afraid - all the builds still failed. The recipe builds happen in two stages: the first prepares the source packages, and the second turns the source packages into binary packages, and runs the tests. It's the second part that fails.

    Here's a log of the test failures on one build (Python 2.7, Quantal 64 bit): https://gist.github.com/takluyver/8582859

  8. Log in to comment