Issue #154 open

Set up more build bots

Thomas Kluyver
created an issue

Resurrecting PyGame's custom buildbot is now issue #111, but there are now other services we could use for additional testing features.

  • ShiningPanda should be relatively easy to configure.
  • Travis and Launchpad won't import directly from Bitbucket, but if we wanted those features, we could set up a mirror.

Comments (27)

  1. illume

    There is a travis-ci setup here: https://travis-ci.org/illume/pygame

    It uses the .travis.yml file in the pygame repo here: https://bitbucket.org/pygame/pygame/src/86a0e5f7f37ce11b2594e32a384af9e17e45e2e3/.travis.yml?at=default

    It currently uses the github mirror of the pygame repo here: https://github.com/illume/pygame

    It's currently failing on some font tests, probably related to the offscreen rendering at a different bitdepth or because it is using some different font, or font library. There is also a problem with results eval in the test code... whatever that is.

  2. Thomas Kluyver reporter

    Unfortunately, Launchpad disabled Mercurial imports a couple of months ago - apparently it wasn't very reliable, and it didn't have enough users to be worth their while fixing.

  3. illume

    It seems the launchpad has stopped updating a few days ago... https://code.launchpad.net/~pygame/pygame/trunk

    So I'm not sure how stable even their git mirroring is? Maybe you know something else that I'm missing?

    We could try doing our own bzr mirror from the server? It would import from the server copy of the hg pygame repo, and export to a publically available bzr repo, which then launchpad could mirror from? Or we could even export to launchpad on the server?

  4. Thomas Kluyver reporter

    (Reply via tak...@gmail.com):

    Oh, I've done the same thing before - if you use the http git protocol, it succeeds for the first import, and then fails. I've set it up again with the git:// protocol, which works for other repositories I have mirrored there.

  5. illume

    Nice one!! It seems there is a successful build on there now :) I also got an email alert from one of the earlier build failures which is nice.

    Also, I see we can download .deb files :)

  6. Thomas Kluyver reporter

    There is a successful build, but it's still not running the tests, for some reason. I'm trying another tweak.

    Yep, it gives us deb file downloads, plus a PPA. When it's set up, we'll be able to add the PPA and get PyGame updated daily, which is handy for dogfooding.

    (The last few failed amd64 builds are where I've cancelled them manually - the build queue for amd64 is longer, so i386 packages are built first)

  7. illume

    The git should be updating again now.

    I had to update the hg-git, and mecurial on the server. Also had to add "hg bookmark -r default master" into the dance.

    hg pull && hg update && hg bookmark -r default master && hg push git+ssh://git@github.com/illume/pygame.git

    Now travisci and launchpad are updating.

    We need to create python 3.x builds of pygame as well as the python 2 build(s).

  8. Paul Craven

    I forked your git code and played around with Travis-CI working on a 2.7 build. It seems to not throw some of the other errors the 2.6 version did, but still throws several.

    The first one is the test_set_grab_and_get_symmetric() text in event_test.py. The SDL_WM_GrabInput call returns a 0, so it wasn't able to grab input. I wonder if this is because we are running headless.

  9. Thomas Kluyver reporter

    On Launchpad, I'm seeing a bunch of failures with error messages like this:

    loading pygame.tests.event_test
    ]0;caca for ncursesError opening terminal: unknown.
    

    Also, a failure in filesystem_encode:

    Traceback (most recent call last):
      File "/build/buildd/pygame-1.9.2~pre~ubuntu+2381+9~ubuntu15.04.1/debian/python-pygame/usr/lib/python2.7/dist-packages/pygame/tests/compat_test.py", line 96, in test_filesystem_encode
        self.assertEqual(compat.filesystem_encode(upath),
      File "/build/buildd/pygame-1.9.2~pre~ubuntu+2381+9~ubuntu15.04.1/debian/python-pygame/usr/lib/python2.7/dist-packages/pygame/compat.py", line 69, in filesystem_encode
        return u.encode(sys.getfilesystemencoding(), filesystem_errors)
    UnicodeEncodeError: 'ascii' codec can't encode character u'\u212a' in position 2: ordinal not in range(128)
    

    In Debian build environments, the locale encoding is set to ascii, so Python claims that the filesystem can only handle ascii names. This is not really true - filenames on Linux can be arbitrary bytes, and nowadays almost everything will expect UTF-8. I have seen some compatibility layers that refuse to believe the system if it claims to have an ascii filesystem, and do this:

    fsencoding = sys.getfilesystemencoding()
    if fsencoding.lower() == 'ascii':
        fsencoding = 'utf-8'
    

    Build log is here: https://launchpadlibrarian.net/192526473/buildlog_ubuntu-vivid-amd64.pygame_1.9.2~pre~ubuntu%2B2381%2B9~ubuntu15.04.1_FAILEDTOBUILD.txt.gz

  10. Paul Craven

    I see fewer failures when running the tests on a local machine. It seems that there needs to be a fix in the automated build, or we perhaps some tests aren't possible on these automated build platforms.

  11. Thomas Kluyver reporter

    There may well be some things that can't be tested. For instance, I don't think the builds run in a terminal, so there are terminal related things that you can't test unless you create a pty in the tests.

    But the different conditions may also reveal genuine problems with the code that you don't encounter on your own machine, like the filesystem encoding issue I described above.

  12. illume

    Thomas Kluyver that sounds like an ok patch.

    Started to collect links to the different buildbots here: http://www.pygame.org/wiki/Hacking

    Here is Paul's AppVeyor page https://ci.appveyor.com/project/pvcraven/pygame/history ... and his code https://bitbucket.org/pcraven/pygame

    OS X server can do continuous integration. I did use it for some iOS projects, but not sure if it's ok for building python stuff. Seems like overkill for that perhaps. However, I'll have a bit more of a look at it to see if it's useful. It seems you can run it on a normal laptop, and don't need a dedicated box. https://developer.apple.com/library/ios/documentation/IDEs/Conceptual/xcode_guide-continuous_integration/200-Adopting_a_Continuous_Integration_Workflow/adopt_continuous_integration.html

  13. Log in to comment