Feature Request: Add different line items for different compression levels, and show compression levels explicitly

[DeflateAwning]

Here's an example of how it currently shows:

Processed 1 file, 1629908 regular extents (1629908 refs), 0 inline.
Type       Perc     Disk Usage   Uncompressed Referenced  
TOTAL       95%      467G         489G         489G       
none       100%      388G         388G         388G       
zstd        78%       78G         100G         100G      

Now, consider a scenario where someone's recently re-mounted with a different compression level, and has rewritten parts of the file (i.e., only some extents).

It should show like this (don't think too hard about the numbers - I didn't update them to be internally consistent):

Processed 1 file, 1629908 regular extents (1629908 refs), 0 inline.
Type       Perc     Disk Usage   Uncompressed Referenced  
TOTAL       95%      467G         489G         489G       
none       100%      388G         388G         388G       
zstd:3      78%       78G         100G         100G       # Original mount with default zstd:3 level.
zstd:15     71%       78G         100G         100G       # New re-mount with zstd:15 level (partial rewrite of some extents).

It is my understanding that all this data is 100% available in the filesystem, and showing it like this is just a matter of implementation in this tool.

[Zygo]

It is my understanding that all this data is 100% available in the filesystem, and showing it like this is just a matter of implementation in this tool.

That understanding is not correct. The compression level is not stored anywhere, even in the compressed data. The level changes a few parameters in kernel memory that are not needed after the compressed stream is produced.

Only the type (zstd/zlib/lzo), uncompressed and compressed sizes are stored, and the compression level cannot be inferred from this information alone.

Now, consider a scenario where someone's recently re-mounted with a different compression level, and has rewritten parts of the file (i.e., only some extents)...

This is possible to do in a different way. It would be relatively easy to partition the filesystem by data age using the generation number field on each extent. So if you record the generation number when you changed the compress level mount parameter, then you could compare data sizes written before that generation number with those after.

That could look like this:

# Data from before the change:
# compsize foo --before-generation 1234
Processed 1 file, 1629908 regular extents (1629908 refs), 0 inline.
Type       Perc     Disk Usage   Uncompressed Referenced  
TOTAL       95%      467G         489G         489G       
none       100%      388G         388G         388G       
zstd        71%       71G         100G         100G      

# compsize foo --after-generation 1234
Processed 1 file, 1121901 regular extents (1121901 refs), 0 inline.
Type       Perc     Disk Usage   Uncompressed Referenced  
TOTAL       95%      95G         100G         100G       
none       100%     188G         188G         188G       
zstd        78%      78G         100G         100G      

# Or combine the flags to take a slice in the middle:
# compsize foo --after-generation 1234 --before-generation 5678
Processed 1 file, 29908 regular extents (29908 refs), 0 inline.
Type       Perc     Disk Usage   Uncompressed Referenced  
TOTAL       95%     180G         200G         200G       
none       100%      88G          88G          88G       
zstd        75%      75G         100G         100G      

The implementation would compare the generation number of the extent ref with the --before and --after limits, and ignore everything out of range. That's the simplest way to get the numbers required to analyze compression performance.

For extra credit, these could be combined into a single table with a row for every generation partition, e.g. if the mount option was changed at generation 1234 and the current generation is 5678:

# compsize --partition-generation 1234
Processed 1 file, 1629908 regular extents (1629908 refs), 0 inline.
Type       Perc     Disk Usage   Uncompressed Referenced GenRange  
TOTAL       95%      432G         439G         439G      0-1234
TOTAL       71%       35G          50G          50G      1235-5678
none       100%      388G         388G         388G      0-1234
zstd        78%       39G          50G          50G      0-1234
zstd        71%       35G          50G          50G      1235-5678

The zstd lines correspond to data written before and after generation 1234. This example tells us all uncompressed data was written before any zstd data.

Multiple --partition-generation options could create multiple partitions if you really want to run with this idea.

[DeflateAwning]

Accidentally went and tried Cunningham’s Law here, not entirely on purpose. Thanks for explaining.

Thank you for this info! If nothing else, I learned more about btrfs.

I wonder if it's a reasonable upstream request (to btrfs) to store the compression levels alongside the generation information? If nothing else, it'd empower tools like this.

[Zygo]

I wonder if it's a reasonable upstream request (to btrfs) to store the compression levels alongside the generation information? If nothing else, it'd empower tools like this.

The missing piece of information here is, "what are tools like this for"?

Such a request should include a solid description of the use case and an argument for spending some of the very small number of bytes available in each extent (two, which must be shared by all future btrfs features...unless you want to consume 4 of the available 62 bits of extent item flags) in the metadata on this feature. Include some data about exactly what gains can be achieved. Also examine alternatives, like tracking the transid numbers and compression level changes, and explain why it's necessary to record new information in metadata instead of using what's already there.

There's a small subset of cases where we can get a benefit from this. It is a situation where:

1. We have a slow CPU. If we had a fast CPU, we'd just use zstd:15 all the time, and have no need for tracking level.
2. Our workload can tolerate compression at all. Compression at any level changes the structure of metadata in ways that hurt performance more than the actual compression of the data does (this is why compress-force is a bad idea--for reasons that have nothing to do with compression). In typical deferred-compression schemes, the data is written uncompressed for speed, and then a daemon comes along to rewrite new uncompressed data with compression if possible (or keep the uncompressed data if not). That architecture doesn't match this proposal, because it only requires transid tracking, and doesn't even look at already compressed data.
3. The total amount of compressed data on the filesystem is large enough for a meaningful reduction in size compared to zstd:1. If very little of the filesystem is compressed, it limits how much gain we can possibly get. If zstd:1 gets most of the available compression, then the gain we can achieve is limited to the difference in size between zstd:1 and zstd:15. If that difference is less than 6% on average, then the length of an average compressed extent is not reduced (remember, they have to be at least 4K smaller, or they take up the same number of disk blocks even if the number of bytes is smaller). If there's no actual gain at the end of this journey, then there's no need to do any of this work to get there.
4. We cannot infer the compression level with existing metadata (e.g. by using find-new and tracking compress option changes against transid, or by looking for large compressed extents). In cases where we can do that, do that instead of changing metadata. For a deferred-compression scheme to work at scale, you'll need the infrastructure to track extents by transid anyway.
5. This combination of constraints exists in some real-world production use case. If this is true, we'd be able to find someone to fund the feature development. If it's not true, then it's better to spend time on other priorities.

TL;DR I don't think it's a good idea. On a big filesystem it's hard to find a use case that can satisfy constraint 3. If your data is growing at 10% per year, you're buying more/bigger drives, not chasing 1% compression gains to put a one-month gap in your purchasing cycle. On a small filesystem, we have the opposite problem: compression is so important that we can't tolerate running compression at less than the maximum level, all the time, so we can't do a deferred compression scheme and there's no need for extra metadata to support one. In use cases where we simply want to know how well the different levels perform, we can just write the data twice, once with each level, and use compsize to get a score--but this isn't a production workload, it's a development-time experiment.

[DeflateAwning]

TL;DR I don't think it's a good idea.

I appreciate you taking the time to explain all that. I now agree!