@obfusk@tech.lgbtI did some testing with zlib-ng, which Fedora and Arch are already using as a replacement for the original zlib and Debian plans to as well, and I'm not happy with what I found.
[...] With the original zlib, you will always get an identical output stream given the same input stream and compressor parameters [...] I expected that zlib-ng would often produce a different output steam than the original, but what I found was a lot more non-deterministic than just that.
With zlib-ng, feeding the data into the compressor in e.g. 1024-byte chunks always gave me a different output stream than using 4096-byte chunks [...] In fact, every chunk size I tried gave a different output. And that's with fixed size chunks, which is not a given if you're handling e.g. a stream of input.
Even using the same buffer size, I cannot get an identical compressed output stream with Python and Java any more [...]
@obfuskI did some more testing and the differences between Python and Java are a result of how they handle output buffers when compressing data with zlib: Java's DeflaterOutputStream uses a fixed size output buffer, whereas Python uses a growing buffer consisting of multiple blocks of increasing sizes and AFAIK does not make any guarantees about this implementation detail.
Thus, matching Python's output from Java is tricky but at least possible, but relies on an implementation detail that may change. And the reverse seems generally not possible. Nor does any of this change that how the input data is fed into the compressor affects the output stream.
https://github.com/python/cpython/blob/v3.13.1/Include/internal/pycore_blocks_output_buffer.h#L64