> This has been done in response to the discovery that the popular installer uv has a different extraction behavior to many Python-based installers that use the ZIP parser implementation provided by the zipfile standard library module.
> For maintainers of installer projects: Ensure that your ZIP implementation follows the ZIP standard and checks the Central Directory before proceeding with decompression. See the CPython zipfile module for a ZIP implementation that implements this logic. Begin checking the RECORD file against ZIP contents and erroring or warning the user that the wheel is incorrectly formatted.
Good to know that I won't need to work around any issues with `zipfile` — and it would be rather absurd for any Python-based installer to use anything else to do the decompression. (Checking RECORD for consistency is straightforward, although of course it takes time.)
... but surely uv got its zip-decompression logic from a crate rather than hand-rolling it? How many other Rust projects out there might have questionable handling of zip files?
> PyPI already implements ZIP and tarball compression-bomb detection as a part of upload processing.
... The implication is that `zipfile` doesn't handle this. But perhaps it can't really? Are there valid uses for zips that work that way? (Or maybe there isn't a clear rule for what counts as a "bomb", and PyPI has to choose a threshold value?)
woodruffw · 30m ago
> and it would be rather absurd for any Python-based installer to use anything else to do the decompression.
You'd reasonably think, but it's difficult to assert this: a lot of people use third-party tooling (uv, but also a lot of hand-rolled stuff), and Python packages aren't always processed in a straight-line-from-the-index manner.
(I think a good reference example of this is security scanners: a scanner might fetch a wheel ZIP and analyze it, and use whatever ZIP implementation it pleases.)
It's also worth noting that one of the differentials here concerns the Central Directory, but the other one is more pernicious: the ZIP APPNOTE[1] isn't really clear about how implementations should key from to EOCDR back to the local file entries, and implementations have (reasonably, IMO) interpreted the language differently. Python's zipfile chooses to do it in one way that I think is justifiable, but it's a "true" differential in the sense that there's no golden answer.
> (Or maybe there isn't a clear rule for what counts as a "bomb", and PyPI has to choose a threshold value?)
Yes, it's this. There are legitimate uses for high-ratio archives (e.g. compressed OS images), but Python package distributions are (generally) not one of them. PyPI has its own compression ratio that's intended to be a sweet spot between "that was compressed really well" and "someone is trying to ZIP-bomb the index."
Interesting. (I have neither the familarity with Rust, nor the willingness to spend time on it, to decide how much of this is the fault of the original vs the fork.)
captn3m0 · 1h ago
Now I am curious at whether these ZIP confusion attacks are mitigated at other registries that use ZIPs? Are there any such?
> For maintainers of installer projects: Ensure that your ZIP implementation follows the ZIP standard and checks the Central Directory before proceeding with decompression. See the CPython zipfile module for a ZIP implementation that implements this logic. Begin checking the RECORD file against ZIP contents and erroring or warning the user that the wheel is incorrectly formatted.
Good to know that I won't need to work around any issues with `zipfile` — and it would be rather absurd for any Python-based installer to use anything else to do the decompression. (Checking RECORD for consistency is straightforward, although of course it takes time.)
... but surely uv got its zip-decompression logic from a crate rather than hand-rolling it? How many other Rust projects out there might have questionable handling of zip files?
> PyPI already implements ZIP and tarball compression-bomb detection as a part of upload processing.
... The implication is that `zipfile` doesn't handle this. But perhaps it can't really? Are there valid uses for zips that work that way? (Or maybe there isn't a clear rule for what counts as a "bomb", and PyPI has to choose a threshold value?)
You'd reasonably think, but it's difficult to assert this: a lot of people use third-party tooling (uv, but also a lot of hand-rolled stuff), and Python packages aren't always processed in a straight-line-from-the-index manner.
(I think a good reference example of this is security scanners: a scanner might fetch a wheel ZIP and analyze it, and use whatever ZIP implementation it pleases.)
It's also worth noting that one of the differentials here concerns the Central Directory, but the other one is more pernicious: the ZIP APPNOTE[1] isn't really clear about how implementations should key from to EOCDR back to the local file entries, and implementations have (reasonably, IMO) interpreted the language differently. Python's zipfile chooses to do it in one way that I think is justifiable, but it's a "true" differential in the sense that there's no golden answer.
> (Or maybe there isn't a clear rule for what counts as a "bomb", and PyPI has to choose a threshold value?)
Yes, it's this. There are legitimate uses for high-ratio archives (e.g. compressed OS images), but Python package distributions are (generally) not one of them. PyPI has its own compression ratio that's intended to be a sweet spot between "that was compressed really well" and "someone is trying to ZIP-bomb the index."
[1]: https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT
well... https://github.com/astral-sh/rs-async-zip