xdr: add views API document for team review

[tamirms]

Summary

• Preliminary document describing the XDR zero-copy views API for team feedback
• Covers usage patterns, navigation, validation, error handling
• Performance benchmarks across 1,000 real pubnet ledgers
• Security properties: guarantees, failure modes, caller responsibilities

This is just the API document for early feedback — the full implementation will follow in a separate PR.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

Adds an initial design/usage document for a proposed “XDR zero-copy views” API, intended to enable lazy navigation of XDR-encoded ledger data without full decoding.

Changes:

• Introduces xdr/views_api.md describing the views concept, navigation patterns, validation, errors, performance benchmarks, and security properties.
• Documents proposed API surface (view types, accessors, Raw(), Copy(), Valid(), iterators, and error kinds).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 4 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 3 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 3 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[leighmcculloch]

This looks really good to me. I have a couple thoughts inline.

[leighmcculloch]

"fat slices": they extend to the end of the parent buffer

Is this optimisation a significant win? Reason being that if an application does mutate one view it could write into the backing array range of other views. It would be safer when slicing to also cap the new slice at its length so that corruption across views is not possible.

[tamirms]

I think this concern goes away once I correct the design to make it clear that mutation of the byte slice is not safe.

[karthikiyer56]

this is where i really miss Scala's val if these were immutable byte arrays by type, the compilre would enforce the "dont mutate" contract for us and this woudl be a non-issue.
Go doesnt give us that, so the immutability here is a documentation contract rather than a type-level gaurantee(right?)

which brings me back to @leighmcculloch's point here..
I want to dissect this a bit further (for my own edification, more than anything else)

Since views are byte[] named arrays, every byte[] operation works on them - including append().
With fat slices, a sub-view's capacity extends to the end of the 1.5MB buffer.
Meaning... if someone accidentally does this:

subView = append(subView, extraBytes...)

the, Go sees plenty of capacity, reueses the OG backing array, and siltenly overwrites everything after that field
no allocation, no error, no panic - just silent corruption.

Now, with capped slices (i.e the 3-tuple variety - buf[start:end:end], the same append() would exceed the capacity, forcing a new alloc, and thus leave the OF buffer untouched (as in, mutation safe)

I get that the doc says "all mutation is unsafe" and that capping requiers computing field sizes (defeating lazy navigation for variable-size types).
this is probably fine in practice — nobody should be appending to a view. but since Go cant enforce that at the type level, a couple things would help:

1/
a brief note on WHY fat slices were chosen over capped slices. It's a slightly non-obvious design choice, so perhaps it is worth documenting.
Something like:

Why fat slices? The alternative — capped slices via Go's three-index syntax buf[start:end:end] — would prevent accidental writes from corrupting sibling fields.
However, capping requires knowing the field's byte size at navigation time.
For fixed-size types (int32, Hash, etc) the size is a compile-time constant so capping is free.
For variable-size types however(strings, variable-length arrays), computing the size requires scanning the fields contents — same work as full decoding, defeating lazy navigation.
Fat slices avoid this cost by deferring size computation to Raw(), which is only called when the caller explicitly needs exact bytes.

2/
maybe an explicit callout that append() on views is particualrly dangerous — its the most natural way a Go dev would accidentally violate the immutability contract.
unlike view[i] = x (which at least looks like mutation), append dosnt feel like its mutating someone elses data 🤷

Dont get me wrong. I am not contesting the design.
the design is sound, just thinking about what would make the "here be dragons" section more concrete for go devs who wont have a Scala compiler slapping thier hands.

[tamirms]

Updated the doc to address review feedback:

• @leighmcculloch's mutation concern: Corrected to state that ALL mutation (serial or concurrent) of the underlying buffer is unsafe, not just concurrent mutation. Views assume the bytes are immutable for the view's lifetime. Also, renamed Validate() to ValidateFull() to properly communicate that validation is already done via navigation.

• @dmkozh's depth limit recommendation: Dropped the default depth limit to 1500 to match stellar-core's xdr::marshaling_stack_limit. Replaced the two independent configurable limits (Valid(WithMaxDepth(n)) + SetMaxViewDepth()) with a single fixed constant — simpler, no configuration footgun.

• @Shaptic's ergonomics concern about verbosity/chaining: Added documentation for Must methods (MustField(), MustValue()) and Try/TryVoid blocks that recover Must panics as errors. This enables clean chaining without per-field error checks.

[karthikiyer56]

nit:
For the untrusted-input case where callers should call ValidateFull() first: it would be useful to note the combined cost. E.g., ValidateFull (489us) + extract all transactions (657us) = ~1,146us total, still 6.5x faster than full decode (7.5ms).

This helps readers evaluate whether the "validate first, then navigate" pattern is still a significant win vs. just doing a full decode.

[karthikiyer56]

this cud be just me, so take it with a pinch of salt.

after reading this doc a few times, i find that Raw() gets a lot more mention than Copy() the impresson i come away with is that Raw() is the defualt choice and Copy() is a rare edge case, when, in truth, it depends on your uscase.

might just be the way the sections are sized — not sure if thats intentional