Conversation
Use of prometheus metrics is cleaner.
gammazero
force-pushed
the
prometheus-metrics
branch
from
805997d to
9d0fcf7
Compare
gammazero
requested review from
byo and
Copilot
and removed request for
Copilot
| Aggregation: view.Sum(), | ||
| } | ||
| CacheEvictions = prometheus.NewGauge( | ||
| prometheus.GaugeOpts{ |
There was a problem hiding this comment.
I wonder if this shouldn't be a counter, may be a bit tricky as from what I can see it may require creating a custom collector, but the type is used by tools such as grafana explorer to suggest functions that work well with counters.
There was a problem hiding this comment.
It would make sense as a counter, but cannot do that because I need to get this as a value from the cache implementation.
| var ( | ||
| cacheTag, _ = tag.NewKey("cache") | ||
| flushCount = prometheus.NewGauge( | ||
| prometheus.GaugeOpts{ |
There was a problem hiding this comment.
I cannot use a counter for this, because it is a value that is retrieved from pebble.
| CacheMultihashes = stats.Int64("core/cache/multihashes", "Number of cached multihashes", stats.UnitDimensionless) | ||
| CacheValues = stats.Int64("core/cache/values", "Number of cached values", stats.UnitDimensionless) | ||
| CacheEvictions = stats.Int64("core/cache/evictions", "Number of indexes evicted from cache", stats.UnitDimensionless) | ||
| CacheMisuse = stats.Int64("core/cache/misuse", "Cache clears due to high value to multihash ratio (indexer misuse)", stats.UnitDimensionless) |
There was a problem hiding this comment.
This seems to be removed - what is the reason?
There was a problem hiding this comment.
They are not removed. Just represented as prometheus counters or gauges. For example, CacheHits is defined just below this comment.
| Measure: StoreSize, | ||
| Aggregation: view.LastValue(), | ||
| } | ||
| GetIndexLatency = prometheus.NewGauge( |
There was a problem hiding this comment.
Can we create the histogram in grafana instead, and just track the raw value here? If we can do that, then the viewer can configure whatever bucket sizes work best for them to see interesting data.
| GetIndexLatency = stats.Float64("core/get_index_latency", "Internal lookup time for a single index", stats.UnitMilliseconds) | ||
| IngestMultihashes = stats.Int64("core/ingest_multihashes", "Number of multihashes put into the indexer", stats.UnitDimensionless) | ||
| RemovedProviders = stats.Int64("core/removed_providers", "Number of providers removed from indexer", stats.UnitDimensionless) | ||
| StoreSize = stats.Int64("core/storage_size", "Bytes of storage used to store the indexed content", stats.UnitBytes) |
There was a problem hiding this comment.
That one (StoreSize) and two below (DHMultihashLatency and DHMetadataLatency) are missing in the new metrics set? What was the reason?
There was a problem hiding this comment.
StoreSize was removed because it is not used. Since some stores do not provide any indication of how much data they store, we look at disk usage instead.
DHMultihashLatency and DHMetadataLatency are not used anywhere AFAIK. storetheindex keeps metrics on the multihash/value write latency for all backends, whether using dhstore, pebble, or something else.
2 participants
Use of prometheus metrics is cleaner.