Skip to content

Conversation

@vnvo
Copy link
Owner

@vnvo vnvo commented Jan 22, 2026

feat: Configurable envelope formats and wire encoding for sinks
Adds support for multiple output envelope formats across all sinks, enabling Debezium compatibility and CloudEvents support without changing the internal event model.
Envelope Formats

Native - Direct Event serialization (default, lowest overhead)
Debezium - Wire-compatible with Debezium's JsonConverter schemaless mode ({"schema":null,"payload":{...}})
CloudEvents - CNCF CloudEvents 1.0 specification

Configuration
yamlsinks:

  • type: kafka
    config:
    id: kafka-sink
    brokers: localhost:9092
    topic: events
    envelope:
    type: debezium # native | debezium | cloudevents
    encoding: json # json (default), avro planned
    Changes
    Core:

New envelope module with Native, Debezium, and CloudEvents implementations
New encoding module with Json encoder (extensible for Avro/Protobuf)
Event structure aligned with Debezium payload format internally

Sources:

MySQL and PostgreSQL sources refactored to adopt envelope/encode model
Fixed MySQL DDL handling and schema reload

Sinks:

All sinks (Kafka, Redis, NATS) updated to use configurable envelope and encoding
Per-sink envelope configuration via YAML

Tests:

All sink integration tests updated for envelope/encoding support
Turso integration tests temporarily disabled

Docs:

New envelopes.md documentation with wire format examples
Updated README with quick start and envelope configuration
Updated example configs

Migration
Existing pipelines continue to work unchanged (defaults to native envelope, json encoding). Users migrating from Debezium can set envelope: { type: debezium } for drop-in compatibility.

closes #36 and #37

@vnvo vnvo linked an issue Jan 22, 2026 that may be closed by this pull request
@vnvo vnvo merged commit af0710d into main Jan 22, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

include-schema for debezium envelope Debezium Envelop

2 participants