Am seeking FESCOs opinion on bringing zlib back to Fedora. zlib-ng-compat does not give bitwise identical results to zlib which are needed in some applications. The particular use case of interest to me is F-Droid server:
https://gitlab.com/fdroid/fdroidserver/-/merge_requests/1641
Would using modules be a reasonable way to allow choosing between zlib and zlib-ng-compat? https://docs.fedoraproject.org/en-US/packaging-guidelines/EnvironmentModules/
One could also just indicate that they conflict with each other: https://docs.fedoraproject.org/en-US/packaging-guidelines/Conflicts/#_compat_package_conflicts
Alternatives could be used, but it maybe useful for the end user to be able to choose which zlib implementation they are using: https://docs.fedoraproject.org/en-US/packaging-guidelines/Alternatives/
Related bugzilla tickets:
https://bugzilla.redhat.com/show_bug.cgi?id=2445516
https://bugzilla.redhat.com/show_bug.cgi?id=2318255
It is possible that both issues are bugs that could be easily fixed. Having two libraries and switching between them would be very painful, so let's first figure out if this is really needed.
bitwise identical results to zlib which are needed in some applications
In general, this sounds more like an anti-feature to me. Is this in test code that expects equivalence between "file compressed with zlib in ancient times" and "file compressed with libz.so on the current system"? Or is it actually somehow requiring bit-identical compression output for a valid reason (I can't think of one)?
Yeah, this is not a good enough reason to bring it back and significantly complicate things.
Zlib in chromium is faster as it uses SIMD instructions, but can allow for bitwise identical results: - https://chromium.googlesource.com/chromium/src/third_party/zlib/
The current motivation is to verify png images distributed with Android applications. As png images allow for metadata with text fields that are not limited to a small set of flags verifying only the image after decompression can be problematic.
Fedora also has a reproducible builds initiative: https://docs.fedoraproject.org/en-US/reproducible-builds/
While people are moving to other compression algorithms such as zstd and image formats such as webp, there is likely to be still significant use of zlib, png and related software. The chromium version of zlib does not have releases though, but will likely retain backward compatibility.
Some of the flags in ISA-L would also allow for reproducibility, butt some adjustments are needed to use it as one uses zlib: https://github.com/intel/isa-l
zlib-ng-compat does not guarantee bit-wise reproducibility: https://github.com/zlib-ng/zlib-ng/blob/develop/PORTING.md#zlib-compat-mode
@fed500 See @decathorpe's comment above. It sounds like you expect bit-for-bit-identical-result with old library versions. This has very little to do with "reproducible builds initiative", which is about reproducibility with a specific version of the software. Please clarify.
It's not a cryptographic hash, it's a compression system. A user should always expect that algorithmic tweaks are a possibility between versions (adjustments to a look-back window size, for example). The fact that zlib-ng isn't bug-for-bug compatible isn't a problem as far as I see it. The file formats are compatible and zlib does not appear to offer any additional functionality that zlib-ng is missing that would warrant carrying both of them.
-1
-1 also
Metadata Update from @zbyszek: - Issue tagged with: meeting
This will be discussed during the FESCo meeting today at 18:00 London time.
-1 from me for the same reasons.
(also, bitwise reproducibility across versions of any implementation of compression algorithms have never been guaranteed... if you want reproducibility here, decompress it and recompress it again and verify that instead)
From Debian's ML-discussion, the issue seems to be whether Fedora can be used as a platform for Android APK development. To my limited understanding, this requires producing zlib-compressed APKs from source which are bitwise identical (give the same hash when signed) to what Google produces from source. That sounds terribly brittle even though it appears to have worked so far (with zlib). In order to support that use-case, do we need to bring back the lib or just the binary? In any case, I'd bake such an environment into a container to preserve it unchanged ...
What happens when Google updates to zlib-ng in the Android toolchain? When Debian switches to zlib-ng, I suspect Google will too, because their toolchains are based on Debian testing.
Bringing back just the binary would be fine. But not without any integration in alternatives and disruption to other packages.
Which means it would need to be patched to alternate name, the package name needs to not be zlib and things that are expected to use it would need to be patched accordingly.
zlib
Yeah, if it's only for F-Droid server we can have this either installing with a different name or a different path - and the package name definitely should not be zlib.
Can we have this bundled in F-Droid server, actually, if that's going to be the only user?
We discussed this during the meeting today: APPROVED (+7, 0, -2): FESCo thinks that bringing back zlib as a general purpose library is not advised. The project is encouraged to move to zlib-ng. It may bundle or otherwise make available the old zlib library in a special-purpose package, with a name and description that make it clear that is not to be used as the default implementation and that does not conflict with the default implementation.
(I'll close this as "rejected" as the closest possibility, but this just advice.)
Metadata Update from @zbyszek: - Issue close_status updated to: Rejected - Issue status updated to: Closed (was: Open)
Metadata Update from @zbyszek: - Issue untagged with: meeting