Issue #300 open

OS X wheels, dmg and .zip builds (with travis?)

René Dudfield
created an issue

In order to make builds for OSX and to do continuous integration on OSX we will use the travisci infrastructure.

It currently supports 10.9, 10.10, and 10.11. Whilst pygame can still as of half a year ago compile on 10.4, it's hard to keep these machines accessible to people.

Some notes: - https://github.com/travis-ci/travis-ci/issues/2312 - https://docs.travis-ci.com/user/multi-os/ - https://docs.travis-ci.com/user/osx-ci-environment/#Homebrew

Most important is to get it building and running tests, and secondly to generate a .wheel for the latest OS X.

Comments (111)

  1. René Dudfield reporter

    Yeah, it's risky. I'll set up another one. I just wanted to quickly play with something.

    Still reviewing different projects to see how they do it. So far it seems like the best approach is to download the required pythons and build against those. It's what scikit-learn does amongst others. I can use the script they've already made, which is nice.

  2. Thomas Kluyver

    Yeah, I think the osx images only come with the Mac system Python, so it's necessary to download other versions somehow.

    Did you mean scikit-image? Scikit-learn doesn't appear to have any OSX testing configured on Travis. In any case, +1 to cribbing from people who've already worked this out! :-)

  3. Thomas Kluyver

    I have got tests running on the two versions of Python available through Homebrew - see pull request #66.

    However, there are a number of tests failing, and the fix is not obvious to me:

    ======================================================================
    ERROR: testSavePNG24 (pygame.tests.image_test.ImageModuleTest)
    see if we can save a png with color values in the proper channels.
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/Users/travis/build/takluyver/pygame/_test_env/lib/python3.5/site-packages/pygame/tests/image_test.py", line 214, in testSavePNG24
        pygame.image.save(surf, f_path)
    pygame.error: SavePNG: could not create png write struct
    
    ======================================================================
    ERROR: testSavePNG32 (pygame.tests.image_test.ImageModuleTest)
    see if we can save a png with color values in the proper channels.
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/Users/travis/build/takluyver/pygame/_test_env/lib/python3.5/site-packages/pygame/tests/image_test.py", line 183, in testSavePNG32
        pygame.image.save(surf, f_path)
    pygame.error: SavePNG: could not create png write struct
    
    ======================================================================
    ERROR: test_save (pygame.tests.image_test.ImageModuleTest)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/Users/travis/build/takluyver/pygame/_test_env/lib/python3.5/site-packages/pygame/tests/image_test.py", line 247, in test_save
        pygame.image.save(s, temp_filename)
    pygame.error: SavePNG: could not create png write struct
    
    ======================================================================
    ERROR: test_save_colorkey (pygame.tests.image_test.ImageModuleTest)
    make sure the color key is not changed when saving.
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/Users/travis/build/takluyver/pygame/_test_env/lib/python3.5/site-packages/pygame/tests/image_test.py", line 274, in test_save_colorkey
        pygame.image.save(s, temp_filename)
    pygame.error: SavePNG: could not create png write struct
    
    ======================================================================
    FAIL: testLoadPNG (pygame.tests.image_test.ImageModuleTest)
    see if we can load a png with color values in the proper channels.
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/Users/travis/build/takluyver/pygame/_test_env/lib/python3.5/site-packages/pygame/tests/image_test.py", line 81, in testLoadPNG
        self.assertEquals(pixel_x1_y1, greyish_pixel)
    AssertionError: (109, 120, 129, 140) != (110, 120, 130, 140)
    
    ======================================================================
    ERROR: test_save_unicode_path (pygame.tests.imageext_test.ImageextModuleTest)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/Users/travis/build/takluyver/pygame/_test_env/lib/python3.5/site-packages/pygame/tests/imageext_test.py", line 65, in test_save_unicode_path
        imageext.save_extended(im, temp_file)
    pygame.error: SavePNG: could not create png write struct
    
    ======================================================================
    ERROR: test_load (pygame.tests.mixer_music_test.MixerMusicModuleTest)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/Users/travis/build/takluyver/pygame/_test_env/lib/python3.5/site-packages/pygame/tests/mixer_music_test.py", line 44, in test_load
        pygame.mixer.music.load(umusfn)
    pygame.error: Unrecognized music format
    

    Full log

  4. René Dudfield reporter

    Hi. I'm working on this again.

    This issue seems to be about missing libraries not being installed. eg, libpng.

    For the wheel versions, the other main issue is installing 'Framework' builds of the dependencies. Previously this was done into a shared folder. Which of course means this doesn't work for virtualenv based things. It also can require permissions to install. The solution is to bundled the dependencies into the installed pygame folder for the wheel version. Either the .so files or the framework files. Bundling in the dependencies probably also makes sense for the dmg.

  5. Thomas Kluyver

    It says PNG : found when configuring it - do you think that's erroneous, or that it's finding libpng at compile time but not at runtime?

    For wheels, we essentially have to bundle any non-Python dependencies (at least for now), because no code runs at install time. For OSX, the delocate tool can help us do this bundling. For reference, auditwheel is a similar tool for Linux, which I've already used in my attempts on #295.

  6. PanderMusubi

    My experience on OSX is from more than ten years ago. So you will have to tell me which commands to run to get the information you need. I can find all the libpng files for you and list those.

  7. René Dudfield reporter

    For the png/etc errors it was picking up the wrong headers. Using the X11 stuff.

    I've got some test builds going here: https://travis-ci.org/illume/pygameplay/

    There's still two issues left, and I haven't done the X11 stuff properly yet. I'm thinking of not using it at all on OSX.

    ======================================================================
    
    FAIL: testLoadPNG (pygame.tests.image_test.ImageModuleTest)
    
    see if we can load a png with color values in the proper channels.
    
    ----------------------------------------------------------------------
    
    Traceback (most recent call last):
    
      File "/Users/travis/build/illume/pygameplay/_test_env/lib/python2.7/site-packages/pygame/tests/image_test.py", line 81, in testLoadPNG
    
        self.assertEquals(pixel_x1_y1, greyish_pixel)
    
    AssertionError: (109, 120, 129, 140) != (110, 120, 130, 140)
    
    ======================================================================
    
    ERROR: test_load (pygame.tests.mixer_music_test.MixerMusicModuleTest)
    
    ----------------------------------------------------------------------
    
    Traceback (most recent call last):
    
      File "/Users/travis/build/illume/pygameplay/_test_env/lib/python2.7/site-packages/pygame/tests/mixer_music_test.py", line 44, in test_load
    
        pygame.mixer.music.load(umusfn)
    
    error: Unrecognized music format
    
  8. Thomas Kluyver

    I'm not rejecting the idea of splitting pygame up into pieces, but just to reiterate my position: for now I think we need to focus on getting a release out as soon as possible, so any architectural changes should be postponed until after that.

  9. PanderMusubi

    Agreed. If illume needs more testing the next days, let me know. For Linux it is working fine for me. I will go through more examples and let you know if I find anything. Do you who is the person that does the packaging for Debian/Raspbian/Ubuntu?

  10. Thomas Kluyver

    You can see info on the Debian packaging here: https://tracker.debian.org/pkg/pygame . Vincent Cheng appears to have been the uploader for recent pre-release packages going into experimental.

    Once it's in Debian, it should automatically wind up in Ubuntu and Raspian - but depending on how release cycles fall, it might take months to years before it's available to regular users through their distros.

  11. PanderMusubi

    Yes, Ubuntu doesn't do anything special with this packages, I just checked Vincent worked for the last time on it in 2014, so that is not so long ago. See there are also multiple packages:

    • python-pygame - SDL bindings for games development in Python
    • python-pygame-sdl2 - reimplementation of the Pygame API using SDL2

    Indeed it can take a long time, but quickly notifying packagers that there is a new version with all the information they need sometimes helps a lot. I have gotten packages repackaged and released in a few weeks.

  12. René Dudfield reporter

    Yes, communicating with packagers is very helpful. Also going through the different distros and taking their diffs/patches is helpful for all the other packagers.

    Concentrating on using getting pygame packaged as a wheel for OSX is my main goal for now. I'm trying to reuse the binaries that homebrew makes.

    There's a way to rewrite the linker paths: http://superuser.com/questions/282450/where-do-i-set-dyld-library-path-on-mac-os-x-and-is-it-a-good-idea Also another tool to rewrite things: https://github.com/tito/osxrelocator

  13. René Dudfield reporter

    ps. I disregarded statically linking things because then SDL(etc) would be linked into every .so file we have, and there are a lot of them.

    I tried to rewrite the install_name_tool -change /usr/local/opt/libpng/lib/libpng16.16.dylib @executable_path/libpng16.16.dylib imageext.so

    This at least works with libpng, and I was able to load png files ok this way.

    So now I need a tool that will go through all of the .so files, and make a set of .dylib files which need copying into the folder. Then it needs to copy them, and then rewrite all the paths in the .so files to have @executable_path in front of them.

  14. René Dudfield reporter

    One problem is that it does not include the i386 arch by default. So I need to build all the architectures with homebrew and get pygame to build with i386 as well.

    Found the issue with: delocate-wheel -w fixed_wheels --require-archs=intel -v pygame-1.9.2b1-cp27-cp27m-macosx_10_11_intel.whl

    This fails if both intel architectures are not available.

  15. René Dudfield reporter

    Seems portmidi, smpeg, and fluid-synth do not have a --universal flag with homebrew. It also makes installing the libs take much longer (because they have to be compiled from source).

    Really, only portmidi needs to be fixed I think. smpeg is unsupported by us now. fluid-synth is wonderful, but also not really required.

  16. Thomas Kluyver

    If it's going to be difficult, I'd be in favour of offering only 64-bit Mac wheels for the time being - in keeping with my 'reduce the problem until we actually have time to solve it' strategy. If people are clamouring for 32-bit Mac support, we can return to it, and possibly enlist them to help.

    This SO answer says that all Macs sold from mid 2007 are capable of running 64-bit software, so I wouldn't be surprised if there's little pushback.

    Do you want to upload the wheels you've got as part of the beta on PyPI, so Mac users can easily test them?

  17. René Dudfield reporter

    I don't think it's difficult to leave out portmidi for now. pygame loads without pygame.midi by default so for most people it will work fine (hopefully).

    I have compiled portmidi for both architectures before, it's not that hard. But yes, I will delay sending patches to homebrew and concentrate on getting travisci to upload the osx wheels.

    I've manually uploaded two .whl files I built on my own computer for testing.

  18. René Dudfield reporter

    Seems I accidentally fixed the music loading failing test.

    There is still the png loading issue.

    You can also see a png image loading issue here: Screen Shot 2016-07-17 at 15.21.03.png

    Looks like transparency is not there either.

  19. Thomas Kluyver

    Looks like transparency is not there either.

    That sounds like there's another problem besides the lack of transparency, but I'm not spotting the other problem in your screenshot.

  20. René Dudfield reporter
    (env2)15:58:30-rene~/dev/pygame/git/pygame (master)$ python -c "import pygame;print(pygame.image.load('examples/data/alien3.png'))"
    <Surface(80x71x32 SW)>
    (env2)15:58:36-rene~/dev/pygame/git/pygame (master)$ file examples/data/alien3.png
    examples/data/alien3.png: PNG image data, 80 x 71, 8-bit colormap, non-interlaced
    

    Seems that SDL_image is broken for color mapped images, as well as the greyscale images. I vaguely remember someone made this change to SDL_image on OSX so it uses the built in png loader (but not correctly it seems).

    Perhaps an older version still compiles. Will investigate when the change happened and try that.

  21. PanderMusubi

    I had the same black backgrounds in the sprites of aliens. Is that SDL or libpng problem?

    Do we need to report a bug or feature request at homebrew regarding the --universal flag?

  22. René Dudfield reporter

    Well, it seems to be passing all the tests now. Takes 32 minutes to do a build but. I guess this is another thing that could be improved in the future.

    I added the upload script, and I think it 'works'. It makes binaries for OS X 10.9.

  23. René Dudfield reporter

    I added the extra OSX builds (10.10, 10.11) in there. But they time out after 48 minutes. I guess the homebrew on there is slower or something. So I will have to figure out a way to cache the homebrew builds. I guess someone else has done this before though.

    Once I figure that out I'll create a pull request. Or maybe there is nothing 10.9 specific in there, and we can rename those files to be used on 10.10, 10.11.

  24. René Dudfield reporter

    Well, I just installed a 10.9 whl on a 10.11 machine. So I think we just need to make whl files for that, and they work for any later version. Trying a build again.

  25. Thomas Kluyver

    Thanks Matthew! Is it necessary to ensure that the extra libraries getting bundled were compiled with the same SDK?

    If we install Python.org Python but get other libraries from Homebrew, will extensions compiled on Python.org Python see that?

  26. Matthew Brett

    Yes, I think you will need to build the extra libraries with MACOSX_DEPLOYMENT_FLAG set to the same SDK. Well, specifically, the compiled libraries should be expected to work on that SDK or higher, so if you compile your extensions with MACOSX_DEPLOYMENT_FLAG=10.6 (the default for Python.org Python) and your libraries with MACOSX_DEPLOYMENT_FLAG=10.9, then you can expect your resulting application to work on 10.9, but it will likely crash (in the library code) for OSX < 10.9.

    You can get the libraries from homebrew, the problem being that these are usually (always?) x86_64 architecture, so in the highly unlikely event that someone tries to use your wheel on i386 on Python, you'll get a nasty crash. You might choose to ignore that problem, and that would be very reasonable. For example, I built h5py wheels that have been on pypi for more than a year, that crash on 32-bit, but no-one reported it. If you do want to avoid that you'll either need to build the libraries yourself, using dual arch flags (this often works) or, if that doesn't work, build the wheels and libraries twice, once for 32- and once for 64-bit, and then fuse them into one dual-arch wheel. Current build of h5py is an example : https://github.com/MacPython/h5py-wheels/blob/master/config.sh#L29

  27. Thomas Kluyver

    If the wheel tag specifies x86_64, pip will refuse to install it on i386 Python, won't it? So long as that's the case, I think we can keep things simple and stick to 64-bit for now. We can always come back and enhance this later.

  28. Matthew Brett

    The problem with that, is that pip will also refuse to install x86_64 wheels on system Python and Python.org Python, because these identify themselves as dual arch (intel). These two are a fairly large proportion of the OSX wheel market.

    So the choice is - lie about the compatibility by saying the wheel is dual arch when it isn't, on the basis that it's unlikely anyone's ever going to test the i386 architecture, or jump through some possibly significant hoops to make a dual arch wheel.

  29. Thomas Kluyver

    Matthew: ah, gotcha. That's a pain. If you went for a year without complaints, I'd probably still lean towards x86_64 masquerading as dual arch, but it's up to René Dudfield.

    Pander: Guessing, but I think it's the travis build itself. I know Travis kills jobs if there's no output for 10 minutes, and I imagine there's some kind of absolute timeout as well, though I thought it was a bit higher than 48 minutes. If it's the lack of output getting us, there's a travis_wait command we can use to keep it alive for a bit longer. But it would be nicer to have a solution that didn't involve 50-minute builds on each commit...

  30. Matthew Brett

    The other problem with the homebrew libs, as you were implying, is that they usually (always?) use the SDK of the OSX version (10.9 SDK on OSX 10.9), so, to use the homebrew libraries in a wheel (with delocate), you'd have to restrict the wheels to (the OSX version of the homebrew you compiled with) or later.

  31. René Dudfield reporter

    Thanks for all the words, tips and links.

    I set a few caches, and tried some parallel builds and test runs. Which still doesn't stop that font test which hangs. I'll investigate that next.

    Also added the travis_wait command to the before_install which used to take 2000+ seconds.

    Another thing I did was rearrange the builds so that they are in dependency order, so eg, sdl isn't installed (without universal) and then sdl is installed again with universal.

    I tried MACOSX_DEPLOYMENT_TARGET=10.6 but ran into an issue.

  32. René Dudfield reporter

    Oops. It went directly to the main branch. 2cee182f6ccfe708a78c5f15912d0ee7e7f5e1a3 I meant to make a pull request.

    There's a problem with adding environment variables from europython, I think travis is blocking the ip now. So I'll have to do that later. It worked building and uploading from my 'pygameplay' fork though.

    I'm not sure what to do with the versions though. Note the b version doesn't seem to require the --pre in pip install --pre pygame, just pip install pygame works.

  33. Thomas Kluyver

    Note the b version doesn't seem to require the --pre in pip install --pre pygame, just pip install pygame works.

    This might be because no there are no previous versions of pygame downloadable from PyPI; I have certainly seen it working with other projects.

  34. Aivar Annamaa

    I was not able to load png-s with pygame installed from wheel (pygame-1.9.2b6-cp35-cp35m-macosx_10_9_intel.whl)

    When I had older libng installed in /usr/local/lib OS X 10.11 then I got "libpng warning: Application built with libpng-1.6.23 but running with 1.5.4" and loading png failed with message about incompatible pnglib version. When I removed the versions from /usr/local/lib, then I got loading error "pygame.error: Failed loading libpng.dylib: dlopen(libpng.dylib, 2): image not found".

    > otool -L imageext.cpython-35m-darwin.so
    imageext.cpython-35m-darwin.so:
        @loader_path/.dylibs/libSDL-1.2.0.dylib (compatibility version 12.0.0, current version 12.4.0)
        @loader_path/.dylibs/libSDL_image-1.2.0.dylib (compatibility version 9.0.0, current version 9.4.0)
        @loader_path/.dylibs/libpng16.16.dylib (compatibility version 40.0.0, current version 40.0.0)
        @loader_path/.dylibs/libjpeg.8.dylib (compatibility version 13.0.0, current version 13.0.0)
        /System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa (compatibility version 1.0.0, current version 20.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1197.1.1)
    

    Seems that imageext loads the correct pnglib but SDL_image looks for it in the wrong places.

    I tried setting DYLD_LIBRARY_PATH to .dylib folder but this didn't help.

    When I installed libpng 1.6 to /usr/local/lib, then I was able to load png-s.

    Similar thing with jpg-s -- when I had correct version in /usr/local/lib, then everything worked, otherwise not.

  35. Aivar Annamaa

    By the way, currently there is no wheel for 32-bit OS X 10.6 (pip install pygame says "No matching distribution found for pygame" ). Does it mean that this system is not supported or the wheel is not there yet?

  36. René Dudfield reporter

    The problem is that Travisci, and modern xcode (well, 3 previous releases) does not support that.

    After the release I might go to the effort of installing an old OSX, and an old xcode to make wheels for older platforms. There's a possibility that the tools we use to make the wheels won't work however. Two days ago I got an old Mac laptop I can use for testing. So it's at least possible for me to do now.

    Can you please explain your situation more? Is it for a class room? If we hear there are more people using older OSX, then there might be more motivation for people.

  37. Thomas Kluyver

    I tried using a copy of the script we use to build SDL in the Linux docker images, but it failed to compile SDL. I'm rather out of my depth to fix this, and the push to Github/wait for Travis cycle makes testing hypotheses very slow. This needs someone with more knowledge of C and/or OSX to look into it.

  38. Tommy Sparber

    Hi,

    I made some improvements on building on macOS, it seems to build and test fine, but I did not yet test the wheels.

    I still need to clean it up a bit and will then create a PR here on bitbucket (if desired).

    Is building a universal binary still a real requirement? (32-Bit Macs are quite old, 10.6 is the latest running OS and the build process is rather slooooow). Is an universal build of fluid-synth really needed? (See https://github.com/Homebrew/homebrew-core/pull/6903)

    Installing and building using homebrew works quite well and is also my preferred method of obtaining pygame. I created an updated PR at https://github.com/Homebrew/homebrew-python/pull/360. Their bottling process makes the install also quite fast. I'm sure, that as soon as an 1.9.2 release is tagged the PR will be merged quite fast. (I will update the PR there then).

    Happy to help out :)

  39. Thomas Kluyver

    Thanks Tommy Sparber ! I'm not very familiar with mac stuff, but if it simplifies stuff I think it's fine to build for only 64-bit for now. We seem to be struggling to get a release out, so I'm in favour of aggressively confining what we try to do, so we can actually get there.

    If we're not confident the wheels are behaving themselves, I think we should make the release without Mac wheels - as you mention, people will still be able to get it from other sources like Homebrew.

  40. Tommy Sparber

    The wheels seem to build now, but I'm not having any luck installing them.

    • pip3 install https://transfer.sh/kXOOw/pygame-1.9.2b8-cp35-cp35m-macosx-10-11-x86-64.whl
    • pip install https://transfer.sh/q2eDK/pygame-1.9.2b8-cp27-cp27m-macosx-10-11-x86-64.whl
    • pip3 install https://transfer.sh/12zYzP/pygame-1.9.2b8-cp35-cp35m-macosx-10-11-intel.whl (universal)
    • pip install https://transfer.sh/161ykZ/pygame-1.9.2b8-cp27-cp27m-macosx-10-11-x86-64.whl (universal but not named intel??)

    each of them results in the following message using pip and pip3 from homebrew (python 2.x and python 3.x) on Sierra 10.12.1:

    ... is not a supported wheel on this platform.
    

    Renaming the files to end in -macosx_10_11_intel.whl (notice the underlines) makes them install. I think delocate has some issues to name the wheels correctly.

    any ideas?

  41. Tommy Sparber

    I figured out the naming issue: transfer.sh was replacing all the underscores to dashes. 😦

    The latest run now builds sucessfully, but the wheels are missing some dylibs. I added a test where all homebrew libs are uninstalled and the pygame wheel is installed into a new virtualenv. Results can be seen at:

    https://travis-ci.org/tsparber/pygame/builds/178390201

    https://travis-ci.org/tsparber/pygame/jobs/178390205#L2554

    The libpng, libjpg dylibs are included but with their versionnumber. dlopen expects no version number as far as I can tell. libvorbisfile is not included.

    Matthew Brett any idea on how to include some extra dylibs using delocate?

    Files to test:

    • pip3 install https://sparber.net/r/kARYs/pygame-1.9.2b8-cp35-cp35m-macosx_10_11_x86_64.whl (homebrew python3.5)
    • pip install https://sparber.net/r/bprGu/pygame-1.9.2b8-cp27-cp27m-macosx_10_11_x86_64.whl (homebrew python2.7)
    • pip install https://sparber.net/r/xqUun/pygame-1.9.2b8-cp27-cp27m-macosx_10_11_intel.whl (system python2.7)
  42. Joe Rickerby

    I think I've seen this problem before - I've actually managed to manually build an OS X wheel of an old version of pygame (in a bit of a hurry) to include in the Tingbot IDE. The problem is that SDL_image loads the PNG, JPEG and TIF libraries using SDL_LoadObject, which tries to find the libraries by resolving against some search paths and then doing dlopen on them.

    So, delocate isn't going to be able find those libraries since they're not included in the output of otool -L (it has bundled libpng because that library is referenced by libfreetype, but it's not finding that correctly either as it needs a symlink).

    I see above that there are some references to SDL_Image compilation flags - perhaps they would fix the issue? I think the relevant flags are --disable-jpg-shared and --disable-png-shared.

  43. Tommy Sparber

    Yes, those flags would (so I assume) statically link libpng and libjpeg. But this would not solve the issue of the missing libvorbisfile.dylib and if you say with libfreetype (haven't seen this myself yet).

    Also adding the compiler flags to SDL_image needs changes on homebrew (possible, but I would like to avoid further changes if other methods are required for libvorbisfile).

    Maybe creating the relative symlinks can be added to delocate.

  44. Joe Rickerby

    Sorry, I misspoke above - there's no problem with libfreetype, just that the libpng that is included by libfreetype isn't found by SDL_image.

    Adding the symlinks to the wheel might work, I just wish I could find something in the source that implies SDL is going to try and resolve library paths relative to itself... here's where I'm looking https://github.com/spurious/SDL-mirror/blob/SDL12/src/loadso/macosx/SDL_dlcompat.c#L444

    I might be missing something though.

    (I just had a look, the equivalent flag for SDL_mixer is --disable-music-ogg-shared)

  45. Tommy Sparber

    Thanks Joe Rickerby. The tests are now green 🎆

    I would be great if someone else could test these wheels:

    pip3 install https://sparber.net/r/1qUhn/pygame-1.9.2b8-cp35-cp35m-macosx_10_11_x86_64.whl
    pip install https://sparber.net/r/ZfoSd/pygame-1.9.2b8-cp27-cp27m-macosx_10_11_intel.whl
    

    If everything is fine, I'll create a PR for required travis changes.

  46. Tommy Sparber

    The osx_image: beta-xcode6.2 (with 10.9) will only be available until 28th Nov (=today). The osx_image:xcode6.1 (with 10.9) will be only image with 10.9 and is only available until 17.01.2017. See https://blog.travis-ci.com/2016-11-17-retiring-some-osx-images/. The next best option would be then osx_image: xcode6.4 with 10.10.

    Is it not possible to build on a later macOS for an earlier version? Would you reccomend to use MacPython? According to this table its platform tag would be macosx-10.6-intel.

    pip3 install https://sparber.net/r/lF0SV/pygame-1.9.2b8-cp35-cp35m-macosx_10_6_intel.whl (macpython on 10.11)
    pip install https://sparber.net/r/uXUzQ/pygame-1.9.2b8-cp27-cp27m-macosx_10_6_intel.whl (macpython on 10.11)
    
  47. Matthew Brett

    If the wheel is compatible, it should be fine. However, with your 2.7 wheel on 10.9:

    In [1]: import pygame
    ---------------------------------------------------------------------------
    ImportError                               Traceback (most recent call last)
    <ipython-input-1-4a415d16fbed> in <module>()
    ----> 1 import pygame
    
    /Users/mb312/.virtualenvs/test/lib/python2.7/site-packages/pygame/__init__.py in <module>()
        131 
        132 # first, the "required" modules
    --> 133 from pygame.base import *
        134 from pygame.constants import *
        135 from pygame.version import *
    
    ImportError: dlopen(/Users/mb312/.virtualenvs/test/lib/python2.7/site-packages/pygame/base.so, 2): Symbol not found: _AudioOutputUnitStart
      Referenced from: /Users/mb312/.virtualenvs/test/lib/python2.7/site-packages/pygame/.dylibs/libSDL-1.2.0.dylib
      Expected in: /System/Library/Frameworks/AudioToolbox.framework/Versions/A/AudioToolbox
     in /Users/mb312/.virtualenvs/test/lib/python2.7/site-packages/pygame/.dylibs/libSDL-1.2.0.dylib
    
  48. Matthew Brett

    Thanks - yes - that does import correctly at least. I guess MacPython is preferable to system Python, in the sense that it claims and might provide compatibility with versions back to 10.6.

    I can give you access to a 10.6 machine if that would help.

  49. Thomas Kluyver

    I see that those Homebrew PRs were merged, and it sounds like the options we need to use became the defaults. So am I right in thinking that rebuilding the wheels with the Travis CI setup we already have will make it work? Do we need to do anything to use the new Homebrew recipes?

    Is there anything else we want to change about the Mac builds before doing a release? It looks like the way we currently build them produces wheels for 10.9 and above.

  50. Thomas Kluyver

    I have persuaded Travis (after a bit of disagreement) to build wheels and upload them here:

    https://github.com/takluyver/pygame/releases/tag/1.9.2b9

    Mac users, could you test these? In particular, it would be good run the aliens example game and check that the PNG problem shown in the screenshot above is fixed.

    I see these wheels are now showing up as requiring OSX 10.11, presumably because of changes to the Travis defaults. It looks like we can get back to 10.10 easily enough (assuming that doesn't break the build). Is that sufficient? Or should we change more things to try to support older versions of OSX?

    Also, for some weird reason, the Python 3.5 wheel gets the intel arch tag, but the 2.7 wheel gets x86_64. Is there a better way to fix that than just renaming the 2.7 wheel?

  51. Thomas Kluyver

    It doesn't explicitly set the osx_image, and it looks like the Travis default is XCode 7.3, on OSX 10.11.

    There's a commented out line in the Travis file for MACOSX_DEPLOYMENT_TARGET=10.6. Is it that simple? How far back can we set the deployment target and expect it to work?

  52. Matthew Brett

    I think that MACOSX_DEPLOYMENT_TARGET=10.6 should compile (it's the default for Python.org builds, that most of us are using to build OSX wheels on travis).

    I'm guessing that the practical incompatibility is from the homebrew libraries you are pulling in, which presumably were compiled against, and depend on, the OSX version you're running on.

  53. Thomas Kluyver

    Many, but not all, of the Homebrew packages it installed were compiled from source. Does MACOSX_DEPLOYMENT_TARGET affect Homebrew builds? I don't know which piece looks at that variable.

    Unless there's an easy way to get those libraries in a format compatible with older versions, then I think building on Travis limits us to 10.10. The statistics I can find (here, here and here) suggest that a majority of Mac users are on >=10.10, but maybe 20-30% are still on older versions.

    Given that this release is already long overdue, is there anything simple we can do to make wheels for older OSX versions? If it's not simple, I'm inclined to say 10.10+ is good enough - users on older versions will still be able to install it, just not as a convenient wheel from PyPI.

  54. Thomas Kluyver

    That image seems to be refusing to download any SDL components - maybe something to do with the root SSL certificates it has? The error message is not very specific:

    ==> Downloading https://www.libsdl.org/release/SDL-1.2.15.tar.gz
    Error: Failed to download resource "sdl"
    Download failed: https://www.libsdl.org/release/SDL-1.2.15.tar.gz
    

    I'll try on the 10.10 image.

  55. Tommy Sparber

    Setting the deployment target does not work for hombrew without patching it. It is always overwritten. Even when using env=std. Setting it in the formula works or replacing the os detection in brew.sh seems also to work (or patch it somewhere else). I would also suggest to test the wheel after building it (remove everything from homebrew and install the wheel and repeat the tests). This shows for example missing dylibs.

    If the wheel is build using macpython it identifies itself as compatible to 10.6 (but there might be issues with older versions). Maybe the 10.10 built also work on 10.9? (did not test yet).

    Downloading the bottle on beta-xocde6.1 works. Maybe its a temporary issue with libsdl.org. Or disable the universal build. The wheel name contains the intel/x86_64 based on the python version/variant not if the contained dylibs are universal.

  56. Thomas Kluyver

    Maybe its a temporary issue with libsdl.org.

    I thought that, but the same thing occurred across 3 builds (the 2 it runs, plus I restarted one of them). I tried downloading the same URL while they were running, and it worked perfectly normally on my computer. And it's working once I switched to the xcode6.4 image. It could still be random failures on the server hosting the files, but I think some difference in the image downloading them is more likely.

    Or disable the universal build. The wheel name contains the intel/x86_64 based on the python version/variant not if the contained dylibs are universal.

    I was wondering about that. We're not building everything with the --universal flag (some packages reject it). So should we not try to build fat binaries of anything? Presumably it's only useful if all the libraries we use are fat? And I guess it would save quite a lot of time if there are 'bottles' of the x86_64 libraries.

  57. Matthew Brett

    Yes, please do rename the wheels so they get found by Python.org and system Python.

    Wheel installs fine, plays music, graphics look OK. I get the impression from the code that I should be able to use the keyboard to move the player sprite, but that does not work.

    Also (I guess this is a Mac / pygame thing?) the window does not get focus when I start it, and cannot be put to fullscreen.

    python3 -c 'import pygame.examples.aliens as g; g.main()'
    
  58. Thomas Kluyver

    Thanks Matthew! The focus issue is #203 - I've no clue where to start on fixing that, but it isn't something introduced (solely) by building wheels.

    You should be able to use the keyboard to control it - left/right arrows, space to fire. Apologies if this is obvious, but you weren't trying to use WASD, right?

    When you say it works as expected from IDLE, does that include keyboard controls, or just the focus issue?

    And just to double check, the graphics don't include the black boxes around sprites seen in the screenshot above, right?

  59. Matthew Brett

    Right on all counts - was trying the arrow keys and spacebar for the controls.

    These do work from Idle (is this using Tk?), as does the focus.

    The graphics look fine, and don't have the black boxes in the screenshot above.

  60. Matthew Brett

    Aha - and looking at issue #203 - I also only see the behavior when using a virtualenv Python - the Window gets focus and the keyboard works with a pip user install. I had to use a pip user install when testing Idle, so that was probably why it worked when the virtualenv OSX run did not. I guess this is the parallel problem to the matplotlib issues with non-framework builds and the OSX backend - see http://matplotlib.org/faq/virtualenv_faq.html#osx

  61. Thomas Kluyver

    Aha, thanks for the info! So, if I'm following all that correctly, it's all working correctly when using a framework build?

    I don't think it's using Tk - as far as I know, SDL includes the windowing features itself. But it does look like the same effect.

  62. Matthew Brett

    Yes, it's all working correctly with a framework build. For example:

    python3 -m venv venv  #  This makes a framework virtualenv
    pip install https://github.com/takluyver/pygame/releases/download/1.9.2b12/pygame-1.9.2b12-cp35-cp35m-macosx_10_9_x86_64.whl
    python -c 'import pygame.examples.aliens as p; p.main()'
    
  63. Matthew Brett

    That looks fine - I think that all you need to to is rename, and I just installed the wheel without problem. You can also write the tags inside the wheel, but as far as I know, pip does not use these for the install.

  64. Log in to comment