Skip to main content
A trace is one request’s journey across your services. A span is one operation within it. Traces answer where the time went and which hop failed — questions no metric and no log line can answer on their own.
Before you start. You need traces arriving — see Instrument your application or zero-code instrumentation.

​
Finding the trace you want

You rarely browse traces. You arrive at one from something else: Filter by service, operation, duration and status. Slowest first, errors only is the filter that answers most questions.

​
Reading a waterfall

Each bar is a span; nesting is causality. Three things to read, in order:
1

Total duration versus the widest child

If the root is 900ms and its widest child is 880ms, the time is in that child — follow it down. If the root is 900ms and every child is small, the time is in your service, between the calls.
2

Sequential versus parallel

Bars that step down one after another are sequential calls. Ten 50ms calls in series is 500ms that could have been 50ms. This is the single most common finding in a first trace review.
3

The failing span, not the failing trace

A trace marked failed shows the error at the root and at the actual source. The deepest failing span is the cause; everything above it is propagation.

​
Spans, attributes and events

A span carries a name, a duration, a status, and attributes describing the operation — http.method, http.route, http.status_code, db.system. Events are timestamped moments inside a span. Span fields are queryable, and any name not in the promoted set falls through to an attribute lookup — the same behaviour as log search.

​
Traces with a hole in them

A trace missing a service in the middle has one of two causes.Context propagation stopped. A service received a request without a traceparent header, so it started a new trace. Common where a call goes through a queue, a thread pool, a detached background task, or an HTTP client the SDK does not instrument.Sampling dropped it. If services sample independently rather than honouring the parent’s decision, each drops its own spans and you get partial traces rather than fewer traces. See Sampling.The two are distinguishable: a propagation break makes the downstream work appear as a separate root trace; sampling makes it disappear entirely.

​
Logs for a span

A log line written while a span was active carries its trace id and is reachable from the trace.
This requires a trace-aware logging integration. A print(), a bare file logger, or a logging framework without the OpenTelemetry bridge produces log lines that are correct, searchable, and permanently disconnected from every trace.Empty “logs for this span” on a service you know is logging means the logger, not the trace.

​
Metrics derived from traces

Request rate, error rate and duration per service are computed from spans, so instrumenting for traces gives you service-level metrics without extra work.
If you sample traces, do not count spans to get a rate. At 10% sampling, counting traces reports a tenth of reality. Take rates and error ratios from metrics, which are not sampled, and use traces to explain what the metric shows.

​
Retention

Traces are kept for 30 days, like logs. Unlike metrics, there is no long-lived rollup — a trace is either inside the window or gone. If a specific trace matters for a post-incident review, capture it before it ages out.

​
Next

Service map

The dependency graph these traces build.

Manual instrumentation

Add spans where auto-instrumentation cannot see.

Searching logs

The other half of an investigation.