> ## Documentation Index
> Fetch the complete documentation index at: https://docs.aiaxoniq.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Collector configuration

> The OpenTelemetry Collector pipeline explained: receivers, processors and exporters, in the order they must appear, with the processors worth adding.

The Collector is one binary with a pipeline: **receivers** take telemetry in,
**processors** transform it, **exporters** send it on. This page is the
reference for building that pipeline. The platform pages
([Docker](/send-data/platforms/docker),
[Linux](/send-data/platforms/linux),
[Kubernetes](/send-data/platforms/kubernetes)) give you a working one to start
from.

## The minimum that works

```yaml config.yaml theme={null}
receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
  hostmetrics:
    collection_interval: 30s
    scrapers:
      cpu:
      memory:
      disk:
      filesystem:
      network:
      load:

processors:
  # Order matters and is not alphabetical. memory_limiter can only shed load it
  # sees before anything has buffered it, so it goes first; batch should group
  # records after every other processor has finished changing them, so it goes
  # last. A pipeline that batches first and limits afterwards will still run out
  # of memory under exactly the load the limiter was added for.
  memory_limiter:
    check_interval: 1s
    limit_percentage: 80
    spike_limit_percentage: 25
  resourcedetection:
    detectors: [env, system]
    system:
      hostname_sources: [os]
  batch:
    timeout: 5s
    send_batch_size: 1000

exporters:
  otlphttp/aiaxoniq:
    endpoint: ${env:OIQ_ENDPOINT}
    headers:
      X-License-Key: ${env:OIQ_LICENSE_KEY}
    compression: gzip

service:
  telemetry:
    metrics:
      readers:
        - pull:
            exporter:
              prometheus:
                host: 0.0.0.0
                port: 8888
  pipelines:
    traces:
      receivers: [otlp]
      processors: [memory_limiter, resourcedetection, batch]
      exporters: [otlphttp/aiaxoniq]
    metrics:
      receivers: [otlp, hostmetrics]
      processors: [memory_limiter, resourcedetection, batch]
      exporters: [otlphttp/aiaxoniq]
    logs:
      receivers: [otlp]
      processors: [memory_limiter, resourcedetection, batch]
      exporters: [otlphttp/aiaxoniq]
```

Everything below is something you add to that.

## The pipeline

```yaml theme={null}
service:
  pipelines:
    traces:
      receivers:  [otlp]
      processors: [memory_limiter, batch]
      exporters:  [otlp]
```

<Warning>
  **Processor order is execution order, and it matters.**

  * `memory_limiter` must be **first**. Its job is to reject work before the
    process runs out of memory, which it cannot do from the middle of a
    pipeline.
  * `batch` should be **last**, or as close as possible. Batching before a
    filter means you built batches out of records you then threw away.
  * Anything that drops data — filtering, sampling — belongs before anything
    expensive.

  A pipeline that lists the right processors in the wrong order is valid
  configuration and does the wrong thing silently.
</Warning>

<Note>
  **A component defined but not listed in a pipeline does nothing.** This is
  the most common Collector configuration mistake: the processor is configured
  correctly, the file is valid, and it is never executed because it was not
  added to `service.pipelines`.
</Note>

## Processors worth adding

<AccordionGroup>
  <Accordion title="memory_limiter — stop the Collector from being the outage" icon="shield">
    ```yaml theme={null}
    processors:
      memory_limiter:
        check_interval: 1s
        limit_percentage: 80
        spike_limit_percentage: 25
    ```

    Without it, a traffic spike can push the Collector into the kernel's
    out-of-memory killer — and on a Kubernetes node, the thing that dies may
    not be the Collector. Always include it, first.
  </Accordion>

  <Accordion title="batch — fewer, larger requests" icon="boxes-stacked">
    ```yaml theme={null}
    processors:
      batch:
        timeout: 5s
        send_batch_size: 8192
        send_batch_max_size: 10000
    ```

    The receiver rate-limits *requests*, not records, so batching is what keeps
    you inside it. `send_batch_max_size` also stops a batch growing past the
    body size limit. See [Exporters](/send-data/otel/exporters#batching).
  </Accordion>

  <Accordion title="filter — drop what you will never read" icon="filter">
    Usually the single largest reduction available, and it costs you nothing.

    ```yaml theme={null}
    processors:
      filter/health:
        error_mode: ignore
        traces:
          span:
            - 'attributes["http.route"] == "/health"'
            - 'attributes["http.route"] == "/metrics"'
    ```

    Health checks and metrics scrapes are the highest-frequency, lowest-value
    traffic in most systems.
  </Accordion>

  <Accordion title="resource — set or normalise resource attributes" icon="tags">
    ```yaml theme={null}
    processors:
      resource:
        attributes:
          - key: deployment.environment
            value: production
            action: insert
    ```

    `insert` only fills a missing value; `upsert` overwrites what the
    application sent. Prefer `insert` unless you mean to override applications.
    See [Resource attributes](/send-data/otel/resource-attributes).
  </Accordion>

  <Accordion title="redaction and transform — keep sensitive values out" icon="user-secret">
    Telemetry pipelines are append-only, and there is no surgical delete once
    data has arrived. Masking in flight is the only place this is cheap.

    ```yaml theme={null}
    processors:
      attributes/scrub:
        actions:
          - key: http.request.header.authorization
            action: delete
          - key: user.email
            action: hash
    ```

    This is the mechanism behind the warning on
    [Data retention](/concepts/retention#deleting-data-early).
  </Accordion>

  <Accordion title="k8sattributes — Kubernetes metadata" icon="dharmachakra">
    ```yaml theme={null}
    processors:
      k8sattributes:
        extract:
          metadata:
            - k8s.namespace.name
            - k8s.pod.name
            - k8s.node.name
            - k8s.deployment.name
    ```

    Needs read access to pods. The [Kubernetes
    page](/send-data/platforms/kubernetes) has the working manifests.
  </Accordion>

  <Accordion title="tail_sampling — keep the traces that matter" icon="percent">
    Only on a gateway Collector, and only where every span of a trace reaches
    the same instance. See [Sampling](/send-data/otel/sampling#tail-sampling).
  </Accordion>
</AccordionGroup>

## Receivers beyond OTLP

The Collector can collect as well as receive, which is where host and
infrastructure telemetry comes from:

| Receiver       | Produces                                                    |
| :------------- | :---------------------------------------------------------- |
| `otlp`         | Whatever your applications send. The one you always have.   |
| `hostmetrics`  | CPU, memory, disk, filesystem, network for the host         |
| `docker_stats` | Per-container resource usage                                |
| `kubeletstats` | Per-pod and per-container usage on a node                   |
| `filelog`      | Log files, tailed and parsed                                |
| `prometheus`   | Scrapes Prometheus endpoints, including the Collector's own |

<Note>
  **This is why a Collector is worth running even when your applications export
  directly.** Nothing an SDK does produces host CPU, disk pressure or container
  restarts, and those are frequently the explanation for what the application
  telemetry shows.
</Note>

## Environment variables and secrets

```yaml theme={null}
exporters:
  otlp:
    endpoint: ${env:OIQ_ENDPOINT}
    headers:
      X-License-Key: ${env:OIQ_LICENSE_KEY}
```

<Warning>
  **Never commit a license key into a Collector configuration file.** Use
  environment variable substitution, and supply the value from a secret store,
  a Kubernetes Secret, or a `0600` environment file.

  A configuration file with a key in it tends to end up in a repository, in a
  container image, and in a support ticket.
</Warning>

## Validate before restarting

```bash theme={null}
otelcol validate --config=config.yaml
```

<Note>
  **A Collector with an invalid configuration does not start.** It logs the
  error and exits — it does not fall back to a previous configuration. On a
  gateway that every service exports to, that is an outage of your telemetry
  rather than a warning, so validate first.
</Note>

## Watch the Collector itself

The Collector exposes its own metrics, and two of them answer most questions:

| Metric                               | Meaning                                                                   |
| :----------------------------------- | :------------------------------------------------------------------------ |
| `otelcol_exporter_sent_spans`        | Data is leaving. If this is zero, nothing is being exported.              |
| `otelcol_exporter_send_failed_spans` | Exports are being rejected — check the log for the status code.           |
| `otelcol_processor_dropped_spans`    | Something in the pipeline is discarding data, usually the memory limiter. |

Scrape them with the `prometheus` receiver and send them to aiAxonIQ, so your
telemetry pipeline is itself monitored. A silently failing Collector otherwise
looks exactly like an application that stopped producing traffic.

## Troubleshooting

<AccordionGroup>
  <Accordion title="The collector starts, then logs 'Exporting failed. Will retry'" icon="arrows-rotate">
    Read the `error` field on that line — it names the cause exactly.

    ```text theme={null}
    Exporting failed. Will retry the request after interval.
      {"kind": "exporter", "name": "otlphttp/aiaxoniq",
       "error": "failed to make an HTTP request: Post \"…/v1/metrics\":
                 dial tcp: lookup … : no such host", "interval": "5.2s"}
    ```

    * `no such host` — the endpoint hostname does not resolve. Check
      `$OIQ_ENDPOINT` against the value on **Get Started**.
    * `connection refused` — the host resolves but nothing is listening on that
      port.
    * `401 Unauthorized` — the key is missing, malformed or revoked.
    * `404` — the endpoint already ends in `/v1/…`. It must be the **base** URL;
      the collector appends the signal path itself.

    The collector retries with backoff and does not drop data while it retries,
    so a transient failure here is not a loss.
  </Accordion>

  <Accordion title="Nothing at all in the logs after 'Everything is ready'" icon="ear-listen">
    The collector is running and receiving nothing. That is an application-side
    problem, not a collector one — your services are not exporting to it.

    Check that your application's `OTEL_EXPORTER_OTLP_ENDPOINT` points at the
    collector's OTLP port (`4318` for HTTP, `4317` for gRPC), not at aiAxonIQ.
  </Accordion>

  <Accordion title="401 with a key you know is correct" icon="key">
    Two causes that are not about the key's value:

    * **A trailing newline.** A key read from a file created by a shell heredoc,
      or a Kubernetes Secret made with `--from-file`, carries the newline as
      part of the value. Use `--from-literal`, or `printf` rather than `echo`.
    * **A validation outage.** If the receiver cannot reach the service that
      validates keys it fails closed and returns the same `401`. A sudden `401`
      across every service at once, with a key you have not changed, is far more
      likely to be this. Check `$OIQ_ENDPOINT/health` first.
  </Accordion>

  <Accordion title="Config changes appear to do nothing" icon="file-pen">
    The collector reads its configuration only at startup. Restart it after any
    edit, and confirm the file you edited is the one mounted into the process —
    a bind mount pointing at a path that does not exist silently yields the
    image's default config rather than an error.
  </Accordion>
</AccordionGroup>

## Next

<CardGroup cols={3}>
  <Card title="Exporters" icon="paper-plane" href="/send-data/otel/exporters">
    Batching, retries, queueing and multiple destinations.
  </Card>

  <Card title="Sampling" icon="percent" href="/send-data/otel/sampling">
    Head and tail sampling, and the rule that keeps traces whole.
  </Card>

  <Card title="Kubernetes" icon="dharmachakra" href="/send-data/platforms/kubernetes">
    A DaemonSet and gateway pair, with working manifests.
  </Card>
</CardGroup>