Refine auto codec selection to be payload-first on full data

[cjrolo]

Compressor selection improvements:

• Added a new compressor_tiebreak_rank method to provide a deterministic tiebreaker when compressors tie on payload size and error, favoring FFT first, then Polynomial, and others last.
• Changed the compressor selection process so that, after an initial sample-based shortlist, all shortlisted compressors are evaluated on the full dataset, and the winner is chosen by smallest payload size, then error, then the new tiebreak rank. [1] [2]
• Improved the logic to ensure that there is always at least one eligible compressor result, even if all exceed the maximum error threshold.

Code quality and maintainability:

• Added std::cmp::Ordering import to support the new comparison logic in compressor selection.

[Copilot]

Pull request overview

This PR refines automatic compressor selection by making the final decision based on full-dataset payload size first, with deterministic tie-breaking when compressors produce equal payload sizes and errors.

Changes:

• Added a deterministic compressor_tiebreak_rank to break ties (favoring FFT, then Polynomial, then others).
• Reworked compress_best to use sampling only to shortlist candidates, but always evaluate shortlisted compressors on the full dataset and choose by payload → error → tiebreak rank.
• Adjusted eligibility handling to ensure at least one compressor result is always selectable, even when none meet the max error constraint.

Comments suppressed due to low confidence (1)

atsc/src/frame/mod.rs:159

• partial_cmp(...).unwrap_or(Ordering::Equal) treats NaN errors as equal to numeric errors. When payload sizes tie (and especially in the fallback path where no result meets max_error), this can cause a compressor with NaN error to be preferred over one with a finite error, so the “then error” part of the ordering is not reliably enforced. Consider using a total ordering for f64 (e.g., f64::total_cmp) or explicitly ranking NaN as worst (always greater than any finite error) before applying the tiebreak rank.

                    let by_error = a.0.error.partial_cmp(&b.0.error).unwrap_or(Ordering::Equal);
                    if by_error != Ordering::Equal {
                        return by_error;
                    }

                    Self::compressor_tiebreak_rank(a.1).cmp(&Self::compressor_tiebreak_rank(b.1))

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.