Loading... (original) (raw)

It appears the j.u.Deflater compression functionality is broken when

(1) level/strategy is changed (to new value) + (2) reset,()

which is a normal combination/use scenario when the deflater is being cached and reused. The attached test case fails with 1.2.11 (with data size > the DeflaterOutputStream's internal buffer size).

My reading of the 1.2.11 changes suggests the direct trigger is that the internal state "deflate_state.high_water" is not being reset correctly/appropriately in deflateReset/deflateResetKeep/lm_init().

It triggers a regression from 1.2.8 to 1.2.11 in the scenario that the "strategy/levle is changed" AND a followup "reset" is called (withe the assumption that everything should be reset back to initial state and we start a new compression circle, with the new level and strategy. This is how it worked in 1.2.8). Because one of the condition check is changed from "strm->total_in!=0" to "s->high_water" (for the flush last buffer" branch) in deflate.c/deflateParams().

from (1.2.8's)

if ((strategy != s->strategy || func != configuration_table[level].func) &&
strm->total_in != 0) {
/* Flush the last buffer: */
err = deflate(strm, Z_BLOCK);
if (err == Z_BUF_ERROR && s->pending == 0)
err = Z_OK;
}

to (1.2.11's)

if ((strategy != s->strategy || func != configuration_table[level].func) &&
s->high_water) {
/* Flush the last buffer: */
int err = deflate(strm, Z_BLOCK);
if (err == Z_STREAM_ERROR)
return err;
if (strm->avail_out == 0)
return Z_BUF_ERROR;
}

Given the nature of "reset" it seems reasonable to assume the "high_water" should be reset to 0, its initial value as well, inside either deflateResetKeep() or lm_init().

I have logged an issue at

https://github.com/madler/zlib/issues/275

Given jdk9 has been changed to uses the system zlib for all platforms except the windows, this one currently only fails on windows (but I would assume it will fails if the underlying system zlib is 1.2.11)

-------------------------------- test case ----------------------------------------------------
public static void main(final String[] args) throws IOException {
Path p = Paths.get("jms.compression.data.large");
int[] levels = new int[] {
Deflater.DEFAULT_COMPRESSION,
0,
Deflater.BEST_SPEED,
// 2, 3, 4, 5, 6, 7, 8,
Deflater.BEST_COMPRESSION,
};
try {
byte[] src = Files.readAllBytes(p);
if (src.length == 0) return;

Deflater def = new Deflater(Deflater.DEFAULT_COMPRESSION); //, true);
for (int level : levels) {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
System.out.println(" -------------- level:" + level + " ------------------");

def.setLevel(level);
def.reset();

try (DeflaterOutputStream defos = new DeflaterOutputStream(baos, def)) {
defos.write(src);
}
byte[] compressed = baos.toByteArray();

ByteArrayInputStream bais = new ByteArrayInputStream(compressed);
Inflater inf = new Inflater();
try (InflaterInputStream infis = new InflaterInputStream(bais, inf)) {
byte[] bytes = infis.readAllBytes();
if (!Arrays.equals(src, bytes)) {
System.out.println(" ==> NG");
new Exception().printStackTrace();
} else {
System.out.println(" ==> OK");
}
} catch (Exception x) {
System.out.println(" ==> NG");
x.printStackTrace();
}
}
} catch (IOException x) {
x.printStackTrace();
}
}