Tail Sampling Processor for OpenTelemetry Go SDK

Tail sampling implementation for Go applications. Make sampling decisions based on complete trace data instead of at span creation time.

Adapted from OpenTelemetry Collector Contrib.

Features

• Latency-based sampling
• Error-based sampling
• Probabilistic sampling
• Rate limiting
• Memory-bounded with automatic eviction

Installation

go get github.com/lukashes/tailsampling-go

Quick Start

package main

import (
    "context"
    "time"

    tailsampling "github.com/lukashes/tailsampling-go"
    "github.com/lukashes/tailsampling-go/policies"
    "go.opentelemetry.io/otel"
    sdktrace "go.opentelemetry.io/otel/sdk/trace"
)

func main() {
    exporter := // your exporter

    statusPolicy, _ := policies.NewStatusCode("errors", []string{"ERROR"})
    tailSampler, _ := tailsampling.NewTailSamplingProcessor(
        tailsampling.WithDecisionWait(5*time.Second),
        tailsampling.WithNumTraces(10000),
        tailsampling.WithPolicies(
            policies.NewLatency("latency", 100, 0),
            statusPolicy,
            policies.NewProbabilistic("baseline", 1.0, ""),
        ),
        tailsampling.WithExporter(exporter),
    )

    tp := sdktrace.NewTracerProvider(
        sdktrace.WithSpanProcessor(tailSampler),
    )
    otel.SetTracerProvider(tp)
}

Configuration Options

Processor Configuration

Option	Default	Description
WithDecisionWait	30s	Time to wait before making sampling decision (max 24h)
WithNumTraces	50000	Maximum traces in memory (max 10M)
WithSampleOnFirstMatch	false	Stop evaluation after first match
WithExporter	-	Span exporter (required)

Sampling Policies

Latency Policy - Sample traces exceeding latency threshold:

policies.NewLatency("slow", 100, 0)  // > 100ms
policies.NewLatency("medium", 100, 1000)  // 100-1000ms

Status Code Policy - Sample traces with specific status codes:

policies.NewStatusCode("errors", []string{"ERROR"})

Probabilistic Policy - Sample a percentage of traces:

policies.NewProbabilistic("baseline", 1.0, "")  // 1%

Rate Limiting Policy - Limit spans per second:

policies.NewRateLimiting("rate-limit", 1000)

Limitations

This processor operates within a single process and only sees local spans. For distributed tracing across multiple services, use the OpenTelemetry Collector instead.

Memory is bounded by NumTraces (max concurrent traces) and MaxSpansPerTrace (100 spans per trace). Monitor metrics to tune these values for your workload.

License

Apache License 2.0

This project includes code adapted from OpenTelemetry Collector Contrib, Copyright The OpenTelemetry Authors, licensed under Apache 2.0.