Skip to content

Release/consumoor #792

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
samcm wants to merge 20 commits into
base: master
Choose a base branch
from
+137,137 −1,575
Open

Release/consumoor #792

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
d3eed64
feat(consumoor): bump default ClickHouse maxConns from 8 to 32 (#790)
samcm Feb 25, 2026
49cf6c8
feat(consumoor): add per-table adaptive concurrency limiting for Clic…
samcm Feb 25, 2026
36a903d
feat(consumoor): add configurable TCP connect timeout for kafka broke…
samcm Feb 25, 2026
27fefae
fix(consumoor): query lag for per-topic consumer groups instead of ba…
samcm Feb 25, 2026
13bdc19
feat(consumoor): add Benthos output batching to fix 1-row INSERT bott…
samcm Feb 25, 2026
308d033
feat(consumoor): remove per-table serialization, enable concurrent IN…
samcm Feb 25, 2026
7a0b53d
feat(consumoor): add per-topic batch config overrides
samcm Feb 25, 2026
e974d8c
perf(consumoor): reduce GC pressure via allocation optimizations
samcm Feb 25, 2026
8fa34c2
perf(consumoor): use vtprotobuf for reflection-free proto deserializa…
samcm Feb 25, 2026
67c9da4
feat(consumoor): use per-topic consumer groups for lag monitoring
samcm Feb 25, 2026
b5132b9
feat(consumoor): add configurable rebalance timeout for faster partit…
samcm Feb 25, 2026
1e404a8
perf(consumoor): use sync.Pool for DecoratedEvent via vtprotobuf pool…
samcm Feb 25, 2026
f317b9a
feat(consumoor): add event_lag_seconds histogram metric
samcm Feb 25, 2026
9b14199
feat(consumoor): increase default outputBatchPeriod to 5s
samcm Feb 26, 2026
43d4948
feat(consumoor): add missing observability metrics and dashboard panels
samcm Feb 26, 2026
0a50359
fix(consumoor): align data transformations with Vector pipeline
samcm Feb 27, 2026
643cc79
consumoor: add rate-limited logging, unknown event warnings, and inva…
samcm Mar 2, 2026
794abd9
consumoor: log bad events in JSON form for invalid events and flatten…
samcm Mar 2, 2026
6cc72ce
consumoor: fix error handling gaps — DLQ invalid events, permanent fl…
samcm Mar 2, 2026
d966f0e
consumoor: drop intentionally unsupported events; NAK unknown routes
samcm Mar 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions buf.gen.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,15 @@ plugins:
- remote: buf.build/grpc/go:v1.5.1
out: .
opt: paths=source_relative
- remote: buf.build/community/planetscale-vtprotobuf
out: .
opt:
- paths=source_relative
- features=marshal+unmarshal+size+pool
- pool=github.com/ethpandaops/xatu/pkg/proto/xatu.*
- pool=github.com/ethpandaops/xatu/pkg/proto/eth/v1.*
- pool=github.com/ethpandaops/xatu/pkg/proto/eth/v2.*
- pool=github.com/ethpandaops/xatu/pkg/proto/libp2p.*
- pool=github.com/ethpandaops/xatu/pkg/proto/libp2p/gossipsub.*
- pool=github.com/ethpandaops/xatu/pkg/proto/mevrelay.*
- pool=github.com/ethpandaops/xatu/pkg/proto/noderecord.*
Loading
Loading