Telemetry Export

OTLP Pipeline Architecture

Synapse's telemetry subsystem is built on the OpenTelemetry Protocol (OTLP), exposing a push-based export pipeline that forwards traces, metrics, and structured log streams to any compliant backend — including Datadog, Grafana Tempo, Honeycomb, and self-hosted Jaeger deployments. Export configuration is submitted via POST /v1/telemetry/export, which provisions a dedicated forwarding agent within the tenant's network segment. The agent batches spans in 5-second flush windows and compresses payloads using zstd before transmission, reducing egress costs by up to 68% compared to uncompressed JSON export.

Trace Schema & Span Attributes

Every inference request generates a root span with the synapse.inference instrumentation scope, containing child spans for tokenization, model dispatch, cache lookup, and detokenization. Standard span attributes include synapse.model.id, synapse.tokens.prompt, synapse.tokens.completion, synapse.latency.ttft_ms (time-to-first-token), and synapse.fallback.triggered. This schema is versioned independently of the core API and follows SemVer; additive attribute additions are considered non-breaking, while removals are announced with a 90-day deprecation notice in the changelog.

Filtering, Sampling & Cost Control

High-volume deployments processing millions of daily inferences can manage telemetry costs through head-based and tail-based sampling configured on the export endpoint. Head-based sampling accepts a float between 0.0 and 1.0 applied uniformly at the trace root. Tail-based sampling — available on Enterprise plans — defers the sampling decision until the full trace is assembled, allowing operators to retain 100% of error traces and sampled-down success traces without writing custom collector pipelines. Attribute-level filtering via the exclude_attributes array strips high-cardinality fields such as raw prompt text before export, supporting compliance requirements that prohibit prompt data leaving the Synapse trust boundary.