Inline handling of "order susceptible" fields

[nachog00]

This line reminds me of the problem we have with txids being reversed in some places... we probably want either our won Hash type or the BlockInfo type to implement dedicated methods to construct from "order susceptible" sources, so that we don't depend on a line on an rpc endpoint for this critical bit.

I wouldn't block on this for this PR, but maybe leave a Todo! mentioning the problem?

Originally posted by @nachog00 in #632 (comment)

[nachog00]

#634 is the intended fix to the "reversed txids" problem

[nachog00]

Trying to recapitulate. The general case of this problem seems to be:

Type Foo is some hash/string and it has a canonical orientation. But it also has a common reversed representation which is used in some contexts, typically associated with a "display order" idea.

The problem seems to be that conversions between the two orientations are common in interfaces, and it's hard to be sure of which is the right one in any given context, as well as telling which order IS being assumed in a given context.

My hypothesis is that this problem is exacerbated by the fact that the decision of flipping the order or not is being made ad-hoc on the interfaces, with little formal definition of ORDER(layerA) vs ORDER(layerB).

The typical case, AFAIK looks something like this:

pub struct Foo(pub [u8; 32]);

impl Foo {
    pub fn bytes_in_display_order(&self) -> [u8; 32] { ... }

    pub fn from_bytes_in_display_order(bytes_in_display_order: &[u8; 32]) -> Foo { ... }
}

Then when we get Foo data from sources A and B, and maybe send Foo data to sources C and D, we do:

let foo_from_A: Foo = Foo::new(response_from_source_A);
let foo_from_B: Foo = Foo::from_bytes_in_display_order(&response_from_source_B);

let foo_from_db: Foo = read_foo(...);
let foo_to_C: [u8; 32] = foo_from_db.0;
let foo_to_D: [u8; 32] = foo_from_db.bytes_in_display_order();

This implicitly encodes that source A deals with Foo in the "normal" order and source B does so in the "reversed" order. Likewise for C and D.

Why This Approach Is Problematic

This seems fine and sufficient, but once we need to perform these transformations coherently on multiple interfacing points, then it's really easy to end up with ad-hoc encoding which is:

• Really hard to track: because one must parse which call of from_bytes_in_display_order is meant for source B and which might be for source E, for example. The symbol remains from_bytes_in_display_order so there's the cognitive overload of discriminating them.

• Hard to maintain: if sources B and E both deal with the "normal" order but then, let's say, E changes upstream and now works with "reversed" order, then we would need to find all instances of from_bytes_in_display_order, discern if they are meant for B or E (and whichever many more similar sources there might be) and change them in every place. It's even harder for an analogous scenario with sources A and (e.g.) F, since there's not even a clear from_bytes_in_display_order but a more ambiguous new which might appear even beyond the scope of interfaces.

This whole situation becomes even worse when we notice that the internal value being public, there's nothing stopping devs from instantiating Foo literally, and ad-hoc manipulating raw bytes:

let mut bytes = response_from_source_B;
bytes.reverse();
let foo_from_B = Foo(bytes);

And then inadvertently going:

db.write(foo_from_B.bytes_in_display_order());

Or simply further obscuring the set of occasions where a foo_from_B was instantiated because now it's both Foo::new and Foo(...) that we need to look after.

Proposed Solution

• We make the internal value private
• We make the order inherent methods also private (no public from_bytes_in_display_order or bytes_in_display_order)
• We implement dedicated public <from|to>_source_X methods

This way, the only way to construct or access a value from Foo is through a set of curated methods, one for each interface.

On the A, B, C, D sources example from above, this would look like:

pub struct Foo([u8; 32]);

impl Foo {
    // Private order-specific methods - these make the intent explicit
    fn canonical_order(&self) -> [u8; 32] {
        self.0
    }

    fn bytes_in_display_order(&self) -> [u8; 32] {
        let mut bytes = self.0;
        bytes.reverse();
        bytes
    }

    // Private constructors
    fn from_canonical_order(bytes: [u8; 32]) -> Self {
        Self(bytes)
    }

    fn from_bytes_in_display_order(bytes_in_display_order: [u8; 32]) -> Self {
        let mut bytes = bytes_in_display_order;
        bytes.reverse();
        Self(bytes)
    }

    // Public source-curated methods
    pub fn from_source_a(value: [u8; 32]) -> Self {
        Self::from_canonical_order(value)
    }

    pub fn from_source_b(value: [u8; 32]) -> Self {
        Self::from_bytes_in_display_order(value)
    }

    pub fn to_source_c(&self) -> [u8; 32] {
        self.canonical_order()
    }

    pub fn to_source_d(&self) -> [u8; 32] {
        self.bytes_in_display_order()
    }
}

Benefits

With this pattern:

• It's trivial to find all places where we ingest/pass values to/from any given source
• It's trivial to modify the assumed order for any given source
• It's impossible for devs to mistakenly ingest/pass values in the wrong order, unless they mistake the source itself

[nachog00]

To further standardize this pattern across the codebase, we could introduce a trait that captures the to/from + order axis generically with minimal required implementation.

Trait Definition with Default Implementations

/// Trait for types that have both canonical and reversed byte orders.
/// Only requires implementing three methods; the rest have default implementations.
trait ReversibleField: Sized {
    type Value;

    /// Get bytes in canonical order (required)
    fn to_canonical_order(&self) -> Self::Value;

    /// Construct from canonical order bytes (required)
    fn from_canonical_order(value: Self::Value) -> Self;

    /// Reverse the byte order of a value (required)
    fn reverse(value: Self::Value) -> Self::Value;

    /// Get bytes in reversed order (default implementation provided)
    fn to_reversed_order(&self) -> Self::Value {
        Self::reverse(self.to_canonical_order())
    }

    /// Construct from reversed order bytes (default implementation provided)
    fn from_reversed_order(value: Self::Value) -> Self {
        Self::from_canonical_order(Self::reverse(value))
    }
}

Complete Implementation Example

Implementations only need to provide three methods:

pub struct Foo([u8; 32]);

impl ReversibleField for Foo {
    type Value = [u8; 32];

    fn to_canonical_order(&self) -> Self::Value {
        self.0
    }

    fn from_canonical_order(bytes: Self::Value) -> Self {
        Self(bytes)
    }

    fn reverse(mut bytes: Self::Value) -> Self::Value {
        bytes.reverse();
        bytes
    }
}

// The to_reversed_order() and from_reversed_order() methods are automatically available!

impl Foo {
    // Public source-curated methods built on top of the trait
    pub fn from_source_a(value: [u8; 32]) -> Self {
        Self::from_canonical_order(value)
    }

    pub fn from_source_b(value: [u8; 32]) -> Self {
        Self::from_reversed_order(value)  // Uses default impl
    }

    pub fn to_source_c(&self) -> [u8; 32] {
        self.to_canonical_order()
    }

    pub fn to_source_d(&self) -> [u8; 32] {
        self.to_reversed_order()  // Uses default impl
    }
}

Benefits

With this trait-based approach:

• Minimal implementation: Only 3 methods required (to_canonical_order, from_canonical_order, reverse)
• Consistent naming: All methods follow the to_*_order / from_*_order pattern
• Type agnostic: Works with any internal representation (tuple struct, named fields, enums)
• Testable: Can write generic tests that work for any ReversibleField implementor
• Clear semantics: The operations map directly to the 2×2 matrix of (from/to) × (canonical/reversed)

Critical: Trait Must Be Module-Private

The trait should NOT be pub. In Rust, trait methods are only callable when the trait is in scope, so keeping it private ensures that external code cannot call from_canonical_order(), to_reversed_order(), etc. directly. They can only use the explicitly public from_source_X/to_source_X methods. This is essential for maintaining the constrained API.

Further Refinement: NewtypeWrapper Pattern

For codebases with many single-field newtypes, this could be further streamlined by extracting the common "access and construct" operations into a separate NewtypeWrapper trait. This would reduce the required implementation to just reverse(), with to_canonical_order() and from_canonical_order() delegating to the wrapper trait.

Additionally, such a NewtypeWrapper trait could be useful beyond just reversible fields - it could be made pub and re-exported for newtypes that DO want to expose getters/setters, while remaining private for the reversible field pattern to maintain API constraints.

[nachog00]

For maximum ergonomics, we can provide blanket implementations of ReversibleField for common inner types through simple marker traits:

trait ByteArrayNewtype<const N: usize>: Sized {
    fn as_bytes(&self) -> [u8; N];
    fn from_bytes(bytes: [u8; N]) -> Self;
}

impl<T: ByteArrayNewtype<N>> ReversibleField for T { ... }

With this in place, implementing a new reversible type becomes a two-method implementation that automatically provides all ReversibleField functionality. Similar blanket implementations could be provided for String, Vec<u8>, and other common inner types.

This gives you:

• Minimal boilerplate - just implement the marker trait
• Consistent behavior across all types with the same inner representation
• Custom implementations still possible when needed

[dorianvp]

Leaving a note here: When exposing reversed bytes (display order), we should make sure that they're not represented as the raw bytes, but instead always hex-encoded. Raw bytes are not meant for displaying, and so this should be noticed (and limited) at the API level.

[zancas]

This all sounds nice @nachog00 do you have an execution plan/issues/a schedule? When will this turn into code in zaino?