Apply the Debian patch for reproducible wheel files, but instead of hardcoding

#52 Merged at 1740e84
Repository
warsaw
Branch
issue143
Repository
pypa
Branch
default
Author
  1. Barry Warsaw
Reviewers
Description

I applied the non-timestamp changes from the Debian patch directly. For the timestamp issue, I added support for WHEEL_FORCE_TIMESTAMP envar. I actually added a test for this latter. I may be violating project coding styles all of the place, so please be gentle. :) Hopefully the change makes sense.

Comments (14)

  1. Barry Warsaw author

    "directly" - well, other than having to write a sort_key function to sort the items in both Python 2 and 3.

    Note too that I only have Python 2.7 and 3.4 to test with atm.

  2. Daniel Holth

    There is also a per-file header on each member of the zip file (not at the end, and written during ZipFile.write()) that includes the timestamp. So unfortunately modifying all the ZipInfo is not enough to create bit-identical zipfiles time and time again.

  3. Barry Warsaw author

    Ah, thanks. I thought I could cheat :). Let me see if I can come up with something better. Too bad the zipfile module doesn't expose the right API.

  4. Barry Warsaw author

    Thanks. Let me know if there's anything else you need. If/when you commit this, I'll release a new wheel package into Debian with this patch cherry picked (at least until the next pypi release)

  5. Barry Warsaw author

    FWIW, I applied this patch to wheel_0.24.0-2 in Debian, which I've just uploaded. I added some DEP-8 tests to prove that the whl files are reproducible when the environment variable is set, even between wheels built 5 seconds apart. The tests pass so I think this PR is good!

    If you do end up modifying it before merging, let me know (or I guess bb will notify me), and I'll update the Debian patch to match. Ideally of course, we'd get a new PyPI release with the fix and I could just drop the Debian delta.

    Thanks for the help!

    1. Nate Coraor

      Directories are not explicitly added so this should not be an issue. I don't believe it's possible to add an empty directory to an sdist or a bdist using MANIFEST.in or other methods.

      Attributes are not preserved, but this matches the existing behavior of all other built distribution formats from wheel (before this change) to egg to a plain bdist. Currently, only sdist preserves permissions.

  6. Nate Coraor

    Spoke too soon, I must've checked the statement about permissions with the wrong version of wheel. Indeed, the wheel's contents are set with empty attributes, which results in files with no permissions set upon extraction. I'll have a fix for this shortly.