Calculation of the start of the zipfile (location of shebang) is incorrect

The code that calculates the position of the start of the zipfile in order to identify the shebang is incorrect. Specifically, it fails to work with zipfiles containing a shebang that have been created by the stdlib zipapp module.

The issue is that the "Offset of the central directory" field in the end of central directory record, is the offset relative to the start of the zipfile. It is legitimate for zipfiles to contain data before the actual zip data (such as the shebang, in the case of zipapp), and the offset takes this into account. The code ignores this possibility and assumes that the offset is relative to the first "local file header" (i.e., the end of the shebang). More accurately, the code assumes that the zipfile was created with no prepended data, and then data was prepended by simple copy, without updating the offset (as this is what tools normally do). The stdlib zipapp module does, however, set the offset taking the shebang into account.

This has not been an issue previously, because zipfiles can (and many do) also contain data prepended that is not reflected in the offset (for example, the launcher itself, typically). Zip utilities cope with this - Info-Zip reports it as "warning [xxx.zip]: 727 extra bytes at beginning or within zipfile", for example.

The correct means of locating the shebang needs to allow for both prepended data reflected in the offset, and prepended data not included in the offset. Doing this in full generality may require a scan for one of the signatures embedded in the zipfile data (I haven't looked at the source of any of the zip utilities to see how they handle this).

I think that the simplest solution here would be to use the existing code to locate the "presumed start" of the zipfile, and then try two approaches:

Check if the bytes immediately following the "presumed start" are #! - if so, the central directory offset takes account of the prepended shebang, and so we have found it.
Use the current approach, assuming that the shebang immediately precedes the "presumed start" - this handles the case where the shebang was prepended without modifying the offset.

In theory, it's possible that some other scenario could have happened, with some prepended data included in the offset and other data not. But it's unlikely that these situations will occur in practice.

I'll try to put together a PR to fix this issue, as it's directly affecting me (I can't use this launcher with zipapps).

Comments (8)