Windows wheels

Issue #3747 new
Frazer McLean created an issue

As mentioned in #3746, Windows wheels on PyPI would be useful.

I quickly set up wheel building on AppVeyor as a proof of concept. It's using my GitHub repository but it supports Bitbucket too.

https://ci.appveyor.com/project/RazerM/sqlalchemy/build/1.0.3

I'd be willing to put the wheels on PyPI for new releases, but it's worth discussing user expectations. Once wheels are available, users expect them to be available. As such, it's good to have wheels on PyPI as soon as a new release goes live, and we'd have a delay if someone else is uploading.

AppVeyor has a REST API, so I could create a script that downloads the build artifacts (i.e. the wheels) so that whomever does the release can also upload the wheels at the same time.

Comments (9)

  1. Mike Bayer repo owner

    this is very nice and thanks for this effort!

    Ideally, it would be scripted to both retrieve the wheels from appveyor and to push them to the corresponding release on pypi. I'm sure pypi has an API for this.

  2. Frazer McLean reporter

    I think PyPI (or perhaps Twine, which is what I use) automatically associates the files with the correct version.

    I have a script that would look like the following (see here):

    $ python appveyor/getwheels.py rel_1_0
    
    Downloading wheels for latest build on branch 'rel_1_0'.
    
    Branch:          rel_1_0
    AppVeyor build:  7
    Commit ID:       abcdefg<etc>
    Commit message:  foobar
    
    Build status:    success
    
    Downloading dist/SQLAlchemy-1.0.14-cp26-cp26m-win32.whl
    Downloading dist/SQLAlchemy-1.0.14-cp27-cp27m-win32.whl
    Downloading dist/SQLAlchemy-1.0.14-cp33-cp33m-win32.whl
    Downloading dist/SQLAlchemy-1.0.14-cp34-cp34m-win32.whl
    Downloading dist/SQLAlchemy-1.0.14-cp35-cp35m-win32.whl
    Downloading dist/SQLAlchemy-1.0.14-cp26-cp26m-win_amd64.whl
    Downloading dist/SQLAlchemy-1.0.14-cp27-cp27m-win_amd64.whl
    Downloading dist/SQLAlchemy-1.0.14-cp33-cp33m-win_amd64.whl
    Downloading dist/SQLAlchemy-1.0.14-cp34-cp34m-win_amd64.whl
    Downloading dist/SQLAlchemy-1.0.14-cp35-cp35m-win_amd64.whl
    

    It aborts if the build failed or is still running. I could add a subprocess call to twine upload each one.

  3. Mike Bayer repo owner

    Basically what I'm going to do is:

    1. assuming appveyor is a free thing I can host the appveyor account, unless you wanted me to use yours

    2. put your script somewhere

    3. run it as an hourly cron, that just checks for new releases and does the right thing

  4. Frazer McLean reporter

    1 - Yeah, my intention was for you to use your own AppVeyor account.

    3 - Sounds fine, assuming the branches you're checking are always releasable. You'd need to add some logic to prevent the script re-downloading every single hour.

    There's also a way to have different behaviour if the git commit is tagged, e.g. with Logbook we have AppVeyor uploading to PyPI directly if the commit is tagged (which requires storing PyPI credentials as "secure" environment variables)

    %BUILD% python setup.py bdist_wheel
    IF "%APPVEYOR_REPO_TAG%"=="true" (
        twine upload -u %PYPI_USERNAME% -p %PYPI_PASSWORD% dist\*.whl
    )
    

    I think the API also returns whether the commit was tagged, and so the cron script can have the same check.

  5. Mike Bayer repo owner

    I can do that. I set a tag on releases. Can we have it that if a rel_x_y_z tag is pushed, appveyor does the builds and pushes to pypi and that's it ?

  6. Frazer McLean reporter

    Yep! I can set this up. I'll open a pull request when I'm done; then you'll just need to encrypt the PyPI credentials with your AppVeyor account and put them in the appveyor.yml file.

  7. Mike Bayer repo owner

    wow that sounds great. as long as appveyor stays around w/ free windows servers + OS + Visual Studio tools that's a huge help!

  8. Log in to comment