Skip to content

Optimizations (inspired by LP)#73

Merged
rkallos merged 23 commits intomainfrom
perf/telemetry-persist
Apr 1, 2026
Merged

Optimizations (inspired by LP)#73
rkallos merged 23 commits intomainfrom
perf/telemetry-persist

Conversation

@rkallos
Copy link
Copy Markdown
Owner

@rkallos rkallos commented Mar 27, 2026

Optimizations

  • Eliminate persistent_term lookup per event by upgrading to :telemetry ~> 1.4 and using :telemetry.persist/0.
  • Use per-metric insert closures that capture the storage module, metric id, and metric struct. This avoids pattern matching the metric type and doing a map lookup.
  • Deduplicate tag computation across metrics. Unique tag functions are evaluated once per event and stored in a tuple for fast access.
  • Add Storage.resolve/1 callback. Called once per event (instead of once per metric) to resolve the per-scheduler ETS table in Peep.Storage.Striped. Eliminates repeated :erlang.system_info(:scheduler_id) + elem/2 calls when multiple metrics share an event. This may introduce more contention in ETS tables when events insert many metrics, but microbenchmarks and running this branch in production still show improved throughput.
  • Branch in store_metrics to avoid calling functions to fetch measurements for counters, and avoid looking up a :keep fun.
  • Make Peep.Atomics.insert/2 a macro, inlining bucket calculation and :atomics.add calls.
  • Switch ids_to_metrics from map to tuple

Bug fixes

  • Fix StatsD distribution reporting: Statsd.Cache was filtering on "+Inf" (string) but buckets use the :infinity atom. Added regression test.
  • Fix Peep.Persistent.erase/1 calling :persistent_term.erase(name) instead of :persistent_term.erase(key(name)), Added regression test.

Cleanup

  • Remove Peep.Persistent.fast_fetch/1 macro (no longer needed; config is passed via handler arg)
  • Move Peep.insert_metric/4 to test helpers (was only used in tests)
  • Remove metrics_to_ids from Peep.Persistent (was only used by the now-removed Peep.insert_metric/4)
  • Move Peep.assign_metric_ids/1 into Peep.Persistent as a private function
  • Simplify StatsD option checking in Peep.init/1
  • Fix erroneous type specs on metric_id, prune_tags callback, and ids_to_metrics
  • Unify Peep startup across test files into shared TestHelpers.start_peep/2
  • Fix test race conditions

@rkallos rkallos force-pushed the perf/telemetry-persist branch from 6968f34 to f6c8663 Compare March 31, 2026 15:21
@rkallos rkallos changed the title WIP: Optimizations (inspired by LP) Optimizations (inspired by LP) Apr 1, 2026
@rkallos rkallos merged commit 462868b into main Apr 1, 2026
1 check passed
@rkallos rkallos deleted the perf/telemetry-persist branch April 1, 2026 13:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant