feat(telemetry): add distributed heartbeat metrics and centralize attribute keys

[hyp3rd]

• Add DistHeartbeatMetrics() method to expose heartbeat success/failure, nodes removed, and read primary promote metrics
• Create internal/telemetry/attrs package to centralize OpenTelemetry attribute key constants
• Add /cluster/heartbeat endpoint to management HTTP server for heartbeat metrics access
• Update cache Item struct with LastUpdated field for distributed usage tracking
• Set LastUpdated timestamp for replicated writes and version assignments
• Refactor OTel middleware to use centralized attribute constants from attrs package

This change improves observability for distributed cache operations and reduces code duplication by centralizing telemetry attribute definitions.

[trunk-io]

Running Code Quality on PRs by uploading data to Trunk will soon be removed. You can still run checks on your PRs using trunk-action - see the migration guide for more information.

[Copilot]

Pull Request Overview

This PR adds distributed heartbeat metrics and centralizes OpenTelemetry attribute keys for improved observability and maintainability. It introduces a new HTTP endpoint for heartbeat metrics access and adds tracking capabilities for distributed cache operations.

• Centralizes OpenTelemetry attribute key constants in a new attrs package to reduce duplication
• Adds distributed heartbeat metrics collection and HTTP endpoint access
• Enhances cache Item struct with LastUpdated timestamp for distributed usage tracking

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
pkg/middleware/otel_tracing.go	Refactored to use centralized attribute constants from attrs package
pkg/middleware/otel_metrics.go	Updated to use centralized attribute constants and improved parameter naming
pkg/cache/v2/item.go	Added LastUpdated field to track write/version assignment times
pkg/backend/dist_memory.go	Set LastUpdated timestamp when assigning versions on primary path
pkg/backend/dist_http_transport.go	Set LastUpdated timestamp for decoded items
pkg/backend/dist_http_server.go	Set LastUpdated timestamp for replicated writes
management_http.go	Added /cluster/heartbeat endpoint for heartbeat metrics access
internal/telemetry/attrs/keys.go	New package centralizing OpenTelemetry attribute key constants
hypercache.go	Added DistHeartbeatMetrics method to expose heartbeat metrics

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}