Support for exporting metrics matching given label(s?)

[hauleth]

In our case we provide a way for customers to fetch the metrics of their own tenant. This however requires gathering metrics where the given metric value is set to specific value. It would be handy to have function like:

Peep.get_metrics_for(tags: %{foo: "bar"})

Or similar that would provide a way to gather metrics only for specific tag(s?).

[hauleth]

Quick and dirty implementation that uses ETS store:

def fetch_metrics_for(name, tags) do
  match =
    for {name, value} <- tags do
      {:"=:=", {:map_get, {:const, name}, :"$1"}, {:const, value}}
    end

  {_, store} = Peep.Persistent.storage(name)

  store
  |> List.wrap()
  |> Enum.flat_map(fn tid ->
    :ets.select(tid, [{{{:_, :"$1", :_}, :_}, match, [:"$_"]}])
  end)
  |> group_metrics(%{})
end

defp group_metrics([], acc) do
  acc
end

defp group_metrics([metric | rest], acc) do
  acc2 = group_metric(metric, acc)
  group_metrics(rest, acc2)
end

defp group_metric({{%Metrics.Counter{} = metric, tags, _}, value}, acc) do
  update_in(acc, [Access.key(metric, %{}), Access.key(tags, 0)], &(&1 + value))
end

defp group_metric({{%Metrics.Sum{} = metric, tags, _}, value}, acc) do
  update_in(acc, [Access.key(metric, %{}), Access.key(tags, 0)], &(&1 + value))
end

defp group_metric({{%Metrics.LastValue{} = metric, tags}, value}, acc) do
  put_in(acc, [Access.key(metric, %{}), Access.key(tags)], value)
end

defp group_metric({{%Metrics.Distribution{} = metric, tags}, atomics}, acc) do
  put_in(acc, [Access.key(metric, %{}), Access.key(tags)], Peep.Storage.Atomics.values(atomics))
end

[rkallos]

Interesting use-case once again!

From a cursory look, in your quick-and-dirty implementation, I suspect each call to :ets.select/2 would fully scan each ETS table, which might get expensive if you have M tenants and each call is only ever interested in (1/M) of the data. Depending on how often Peep.get_metrics_for/* is called, I wonder whether a different architecture might be warranted.

I see you have something very similar to the above implemented in Supavisor. Is it running in production? Is the performance acceptable?