Suggestions for Limiting Sizes of Nested Enum Error Types

[bguo068]

I used to work on research projects (using Rust) without careful error handling, relying heavily on unwrap. Recently, I've been trying to adopt SNAFU and its philosophy. I enjoy the convenience of adding context and having an error enum per module. Now that I've integrated SNAFU throughout my Rust project, I've been pondering some questions, but haven't found the answers yet.

• With an error enum type per module, having deeply nested modules can lead to increasingly large error enums. The size issue is exacerbated by context fields like String, PathBuf, and carrying source and backtrace. I've frequently used boxed types, such as Box<String>, Box<PathBuf>, Box<InnerError>, and Box<Option<Backtrace>>. I've managed to restrict the size of Error types per module, but it does make the definitions cluttered. I'm currently using large-error-threshold = 25 in clippy.toml to warn me of large error enums. While a 24-byte Error enum with indirection (heap allocation for error/context) is great for the happy path, I've noticed that the Whatever type is fairly large (88 bytes on M1 Macs), and the Error type of rust-htslib is also not small (48 bytes). Is there a practical threshold for enum Error sizes? Above which should we consider boxing them to reduce size? Should we limit the depth of modules/error enums to control Error size growth?

• Currently, I've multiple Error types returned by public functions at different levels. Should we convert Errors from all APIs across different modules to a single Error type (i.e., the root enum Error)?

[Enet4]

• With an error enum type per module, having deeply nested modules can lead to increasingly large error enums. The size issue is exacerbated by context fields like String, PathBuf, and carrying source and backtrace. I've frequently used boxed types, such as Box<String>, Box<PathBuf>, Box<InnerError>, and Box<Option<Backtrace>>. I've managed to restrict the size of Error types per module, but it does make the definitions cluttered. I'm currently using large-error-threshold = 25 in clippy.toml to warn me of large error enums. While a 24-byte Error enum with indirection (heap allocation for error/context) is great for the happy path, I've noticed that the Whatever type is fairly large (88 bytes on M1 Macs), and the Error type of rust-htslib is also not small (48 bytes). Is there a practical threshold for enum Error sizes? Above which should we consider boxing them to reduce size? Should we limit the depth of modules/error enums to control Error size growth?

This is a good question, albeit tricky to answer. As the size of Backtrace depends on the target platform and/or on which backtrace implementation is used, it is hard to expect a universal maximum size. In my projects I have have taken a similar path of boxing error sources to comply with the default maximum size recommended by Clippy.

• Currently, I've multiple Error types returned by public functions at different levels. Should we convert Errors from all APIs across different modules to a single Error type (i.e., the root enum Error)?

Having a root error is not a mandatory pattern in SNAFU. An interpretation of the Many Error Types design philosophy is that if a separation of error cases is possible, that would be preferred over centralization.

[bguo068]

Thank you for your answer. I'm glad to know I'm not alone in this situation and not just overthinking it. Boxing seems to be the solution when the error enum is too large, but I should probably adjust the size threshold (using the Clippy default).

[shepmaster]

Should we convert Errors from all APIs across different modules to a single Error type (i.e., the root enum Error)?

With an error enum type per module

I've been recently going the opposite direction and making extremely fine-grained errors, down to the level of one error type per function. This allows the caller of the function to only have to deal with the exact set of errors possible. There's no need for a match arm like _ => unreachable!("This error can't occur").

having deeply nested modules can lead to increasingly large error enums.

And this is the (current?) tradeoff. As I see it, the ultimate problem is that Rust allows taking a reference to every type, which means that every value needs to be able to stand on its own.

My gut tells me that this will ultimately require a language level solution. Off the top of my head, there's been various discussions that would help:

• making functions which return Result have a special ABI
• some kind of cooperative decision of enum discriminants
• defining types that are subsets of a larger type

I think if you had to fit into a tiny size envelope today, your best bet would be to have one giant error enum. However, I wouldn't go down that path until I truly had to.

Is there a practical threshold for enum Error sizes?

Above which [size] should we consider boxing them [...]?

Should we limit the depth of modules/error enums to control Error size growth?

I think in general this is going to be application-dependent, but in the absence of benchmarks / profiling data, I'd probably stick to whatever Clippy warns about by default.

The size issue is exacerbated by context fields like String, PathBuf, and carrying source and backtrace.

I'm surprised to hear that context field is causing the problem. Were I faced with this issue, I'd probably limit myself to boxing entire error values and not worry about specific fields. Depending on what details you want to expose, you could potentially make use of [opaque errors][opaque] to (maybe automatically?) perform the boxing at specific boundaries.

carrying [...] backtrace

I basically never use a Backtrace. Instead I create more error types, each of which is only used in one place. This makes a "semantic backtrace" when you follow the error cause chain. You don't even need to attach file/line information because a basic search should result in the one and only location the error can occur.

I've noticed that the Whatever type is fairly large (88 bytes on M1 Macs), and the Error type of rust-htslib is also not small (48 bytes).

I've spent zero time trying to optimize Whatever. To a large extent, it's only there as a first stepping stone to better error handling, and isn't really intended as a final solution for people who are deeply caring about their errors.

[bguo068]

Thank you sharing your thoughts and the patterns you used for error handling.

I've recently shifted to making extremely fine-grained errors, assigning one error type per function.

I would love to see this pattern in practice. Do you happen to have some open-source project where you used this pattern?

I'm surprised to hear that context field is causing the problem. Were I faced with this issue, I'd probably limit myself to boxing entire error values and not worry about specific fields.

I may have set the size threshold for the error enum too low at 24 bytes, leading to unnecessary boxing of fields. Using a higher threshold, such as 54 bytes, would eliminate the need for boxing those fields.

I basically never use a Backtrace. Instead I create more error types, each of which is only used in one place. This makes a "semantic backtrace" when you follow the error cause chain. You don't even need to attach file/line information because a basic search should result in the one and only location the error can occur.

I really like the idea of a “semantic backtrace.” I’ve even considered pushing this pattern further by explicitly separating semantic error classification from diagnostic detail.

For example:

// `kind` represents the semantic error tree:
// a nested enum where all leaf variants are pure tags,
// and all non-leaf variants only wrap their child enum.
//
// `source` carries diagnostic context and can accumulate
// arbitrary details (including a real backtrace).

struct MyLibError {
    kind: ErrorKind,
    source: Option<anyhow::Error>, // or using String or SmartString
}

This makes the semantic structure of failures explicit and stable, while allowing diagnostic information to grow independently and without inflating the enum hierarchy.

std::io::Error already follows a similar principle, although its ErrorKind is a flat enum of tags rather than a nested semantic tree (see std::io::ErrorKind: https://doc.rust-lang.org/std/io/enum.ErrorKind.html). In both cases, the key idea is that classification and diagnostics are decoupled.

I think this approach could help mitigate the “ever-growing error enum” problem in larger codebases. That said, I’m not sure whether and how widely this pattern is used in real-world projects, or whether it holds up ergonomically outside of foundational libraries like std. Would something like this be of interest to the SNAFU or the broader community?

[bguo068]

To myself:

Examples of Using Snafu

GreptimeDB as an

• Blog Post: GreptimeDB: Error Handling in Rust
• Repository: GreptimeDB GitHub

Patterns Utilized:

• Utilizes snafu::Location to track error origins instead of backtrace; a custom macro is derived for error stack pretty-printing: stack_trace_debug.
• Implements a flat error enum (derived with Snafu) for each sub-crate (internal error type).
• Groups error enum variants into public status codes.
• The error enum size is big (~100-200 bytes).

[To be updated as I read more]