broker: multiple members for gzip compression

[williamhbaker]

Each active gzip writer introduces a small but significant amount of memory overhead, on the order of several hundred KB. When there are many active journals being written, this can add up to a large amount of memory usage.

This adds a threshold where incremental compression will occur only if there is at least 1 MB of data to compress, and creates a new gzip writing mechanism that allows closing & creating new gzip members, concatenated into the same output file. The spool logic uses this mechanism to create a new gzip member for every batch of incremental compression, eliminating the need to hold a gzip writer in memory for the entire lifetime of the fragment file.

This change only applies to standard gzip compression with client-side decompression. If decompression offloading is used, gzip files will continue to be written in a single stream, as there are issues with some object stores truncating multi-member gzip content after the first member.

Manual Testing:

• For 1000 journals actively being written observed a ~1 GiB drop in RSS memory usage post-change, confirmed by profiling to be from a reduction in flate memory overhead.
• Basic writes and reads from journals covering no compression, gzip, and snappy from file / S3 / GCS / Azure stores
• E2E testing with a Flow local stack using AWS, GCS, and Azure storage mappings
• Quick throughput testing: There wasn't any difference in attainable throughput when writing to a modest number of journals as fast as possible running things on my laptop, and CPU usage looked about the same. Theoretically I'd expect some amount of increased CPU for re-initializing new GZIP writers, but at least from this crude test I didn't see anything major.

[jgraettinger]

LGTM