Skip to content

Feature Request: Add TryDeserialize / TryReadPackable APIs for streaming scenarios #419

@xuri-ajiva

Description

@xuri-ajiva

TL;DR

There's no safe way to attempt deserialization in streaming scenarios without exceptions or losing byte position.
Proposing TryDeserialize / TryReadPackable APIs that return bool + bytesConsumed and only advance on success.
This would make streaming protocol parsing safer and more efficient.

Problem

When implementing streaming protocols with tagged messages, there's no safe way to attempt deserialization without risking exceptions or losing track of byte consumption.

This becomes problematic when using MemoryPackReader on buffers that may contain incomplete data (e.g., TCP streams).

Use Case

Consider a streaming protocol with packets structured as [PacketType][Message]:

  1. Read PacketType enum to determine message type.
  2. Attempt to deserialize the corresponding message type.
  3. Handle cases where the buffer contains only part of the message.
  4. Track the reader position precisely so the buffer can be managed correctly.

Desired usage:

if (reader.TryReadPackable<Message>(out var packet))
{
    // Process packet safely
}
// Reader advances only on success

Current Limitations

  • ReadPackable<T>() throws on incomplete data.
  • No way to safely probe for full messages in a streaming buffer.
  • Exception handling adds overhead and complexity in high-throughput scenarios.

Proposed API

Serializer-level:

public static bool TryDeserialize<T>(
    ReadOnlySpan<byte> buffer,
    out T value,
    out int bytesConsumed);

public static bool TryDeserialize<T>(
    ReadOnlySpan<byte> buffer,
    out T value,
    out int bytesConsumed,
    MemoryPackSerializerOptions options);

Reader-level:

public bool TryReadPackable<T>(out T value)
    where T : IMemoryPackable<T>;

Behavior

  • Return true if a complete object is read successfully.
  • Return false if insufficient data is available.
  • Advance the reader position only on success.
  • Pattern matches existing ReadPackable<T>(), but safely.

Benefits

  • Enables efficient, exception-free streaming protocol parsing.
  • Provides precise buffer management.
  • Aligns with common Try\* patterns in .NET (TryParse, TryRead).

Why not length-prefixing?

In many protocols, prefixing each packet with its length would solve incomplete data issues.
However, with MemoryPack this is not straightforward because:

  • A message must be fully serialized before its length is known.
  • Backpatching the length isn't possible without extra buffering or workarounds.
  • This prevents true streaming serialization where messages can be written progressively.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions