-
-
Notifications
You must be signed in to change notification settings - Fork 280
Description
TL;DR
There's no safe way to attempt deserialization in streaming scenarios without exceptions or losing byte position.
Proposing TryDeserialize / TryReadPackable APIs that return bool + bytesConsumed and only advance on success.
This would make streaming protocol parsing safer and more efficient.
Problem
When implementing streaming protocols with tagged messages, there's no safe way to attempt deserialization without risking exceptions or losing track of byte consumption.
This becomes problematic when using MemoryPackReader on buffers that may contain incomplete data (e.g., TCP streams).
Use Case
Consider a streaming protocol with packets structured as [PacketType][Message]:
- Read
PacketTypeenum to determine message type. - Attempt to deserialize the corresponding message type.
- Handle cases where the buffer contains only part of the message.
- Track the reader position precisely so the buffer can be managed correctly.
Desired usage:
if (reader.TryReadPackable<Message>(out var packet))
{
// Process packet safely
}
// Reader advances only on successCurrent Limitations
ReadPackable<T>()throws on incomplete data.- No way to safely probe for full messages in a streaming buffer.
- Exception handling adds overhead and complexity in high-throughput scenarios.
Proposed API
Serializer-level:
public static bool TryDeserialize<T>(
ReadOnlySpan<byte> buffer,
out T value,
out int bytesConsumed);
public static bool TryDeserialize<T>(
ReadOnlySpan<byte> buffer,
out T value,
out int bytesConsumed,
MemoryPackSerializerOptions options);Reader-level:
public bool TryReadPackable<T>(out T value)
where T : IMemoryPackable<T>;Behavior
- Return
trueif a complete object is read successfully. - Return
falseif insufficient data is available. - Advance the reader position only on success.
- Pattern matches existing
ReadPackable<T>(), but safely.
Benefits
- Enables efficient, exception-free streaming protocol parsing.
- Provides precise buffer management.
- Aligns with common
Try\*patterns in .NET (TryParse,TryRead).
Why not length-prefixing?
In many protocols, prefixing each packet with its length would solve incomplete data issues.
However, with MemoryPack this is not straightforward because:
- A message must be fully serialized before its length is known.
- Backpatching the length isn't possible without extra buffering or workarounds.
- This prevents true streaming serialization where messages can be written progressively.