Zero-code troubleshooting

Find your symptom. Each section names the cause and the command that confirms it, so you are not changing things hopefully.

Availability. The zero-code agent is an early-access distribution. It is installable, upgradable and tested, and it is not yet an integrated part of the product: there is no dashboard page for it, no in-product installer and no self-service download. Ask your account contact for access.Its telemetry, once flowing, is ordinary OTLP and needs nothing special — it appears in Services, Traces and Metrics exactly like SDK telemetry does. That is the design rather than a shortcut.

The agent runs and collects nothing

The most common failure, and the hardest to spot, because everything looks right.

Confirm

cat /proc/sys/kernel/perf_event_paranoid        # must be <= 1 for profiles B and C
journalctl -u aiaxoniq-zerocode | grep -i "perf event"

On Kubernetes:

kubectl -n aiaxoniq logs -l app.kubernetes.io/name=opentelemetry-ebpf-instrumentation \
  | grep -i "perf event"

A line reading opening perf event: permission denied is the confirmation.

Why it happens

Above 1, the kernel denies perf-event access even when CAP_PERFMON is granted. Profiles B and C attach kprobes, so the agent acquires every capability it asked for, passes its own gate, reports healthy — and attaches nothing.

Fix

sudo sysctl -w kernel.perf_event_paranoid=1
echo 'kernel.perf_event_paranoid=1' | sudo tee /etc/sysctl.d/99-aiaxoniq.conf

On managed Kubernetes this is a node-level change. On OpenShift it is a machine-config change.

Do not reach for CAP_SYS_ADMIN instead. Upstream documents it as the workaround on AKS and EKS, and it does work — by granting something near-equivalent to root on every node in your cluster. Set the sysctl.

Preflight now treats this as blocking, so a fresh install refuses rather than misleads. An install that predates that check is the one to look at.

The agent refuses to start

The capability enforcement is working as intended: the agent will not attach partially and then report nothing. The startup message names exactly what is missing. The usual cause on Linux is setcap failing on a filesystem without extended attribute support. systemd’s ambient capabilities still grant them, so confirm the systemd unit is what is actually starting the process rather than a hand-rolled invocation.

Only some services appear

Not “nothing” but “not everything” — a different problem with three distinct causes.

The TLS library is not one the agent hooks

The agent hooks OpenSSL’s read and write functions. Statically linked TLS, BoringSSL, rustls, GnuTLS and NSS are each a potential gap.Node.js bundles its own OpenSSL, and Rust’s rustls is not OpenSSL — both are known risks rather than known failures, because the full matrix has not been measured. This presents as “that service is missing”, never as an error.There is no configuration fix. Where you need those services covered, add the OpenTelemetry SDK to them — see Zero-code and the SDK together.

The agent cannot see the process

Host PID access again — but partially. A service in a container the agent cannot see in its namespace produces nothing while its neighbours produce everything.Check --pid=host (Docker), pid: host (Compose), hostPID: true (Kubernetes).

The protocol is not decoded

The agent decodes a wide but finite set: HTTP/S, HTTP/2, gRPC, JSON-RPC, common SQL databases, Redis, MongoDB, Kafka, GraphQL, MQTT, NATS, AMQP, object storage and several GenAI APIs.A service speaking a bespoke binary protocol produces connection-level information and no request-level spans. This is a boundary of the technique, not a setting.

The agent only sees itself

One process discovered — its own. There is no shared PID namespace.

Platform	Setting
Docker	`--pid=host`
Docker Compose	`pid: host`
Kubernetes	`hostPID: true` (the chart sets it)

This is the single most common installation mistake, and it produces no error anywhere. If your service inventory did not grow after installing, check this before anything else.

401 Unauthorized in the agent log

The license key is missing, malformed or revoked.

sudo grep OTEL_EXPORTER_OTLP_HEADERS /etc/aiaxoniq-zerocode/agent.env

On Kubernetes, confirm the Secret exists and is mounted — but note the key is deliberately never in the ConfigMap, so its absence there is correct.

On Linux, the usual cause is a missing -E on sudo. Without it, OIQ_LICENSE_KEY does not survive the privilege change, the installer writes a placeholder and says so, and every export is then rejected. Re-run the installer with sudo -E and --force-config.

See Create a license key to confirm the key is still active, and Ingest endpoints for what each status code means.

429 Rate limited

A DaemonSet is N exporters spending one organization’s rate-limit budget, so a fleet reaches a limit that a single host never would. Reduce volume, in order of effect:

Confirm health and metrics endpoints are still being dropped

They are excluded by default, and a hand-edited configuration is the usual way that stops being true. They are frequently the largest share of total volume.

Add explicit route patterns

Route unification is on by default, but the heuristic is a fallback. Your own patterns are always better — see Metric cardinality below.

Narrow what is instrumented

Exclude services whose telemetry you do not use.

Ingest endpoints has the rate limits themselves.

Metric cardinality explosion

Too many metric series, and an ingest bill to match. Almost always raw URL paths becoming labels. Route unification is on by default and reduces unknown paths heuristically — but explicit patterns always beat the heuristic:

routes:
  patterns:
    - /users/:user_id/orders/:order_id

Then restart the agent — systemctl restart aiaxoniq-zerocode, or set the patterns through the chart on Kubernetes. See cardinality for why this costs what it does.

Kubernetes metadata is missing

Pods, namespaces and workloads are not attached to telemetry. The agent’s cluster access is list and watch on pods, services, nodes and replicasets — if the chart’s RBAC was modified or a policy blocks it, decoration silently stops while spans keep flowing. Check the agent log for authorization errors against the Kubernetes API.

Configuration changes have no effect

The agent reads its configuration at start. Editing a file and waiting changes nothing.

sudo systemctl restart aiaxoniq-zerocode            # Linux
docker compose -f docker/docker-compose.yaml restart # Compose

On Kubernetes, changing values means a helm upgrade and a pod restart.

The second cause is editing the wrong file — a configuration generated to one path while the service reads another. Confirm with the doctor tool, which reports the configuration actually in effect.

The kernel or platform is not supported

./bin/oiq-zerocode-preflight

It names the specific reason rather than failing generically. The common ones are a kernel below 5.8, missing kernel type information, or a platform where there is no node to install on at all. See Compatibility for the full matrix, including the platforms that will never work and what to use instead.

The agent instruments itself

It should not — self-exclusion is part of the shipped configuration. If the agent appears in your own service map, that block did not survive a configuration edit. Regenerate rather than patch.

Still stuck

./bin/oiq-zerocode-doctor           # health, connectivity, config validation
./bin/oiq-zerocode-audit            # the privilege actually held
./bin/oiq-zerocode-support-bundle   # redacted diagnostics for support

The support bundle redacts the license key and other credentials, and deletes itself rather than write a file it could not fully redact.

Verification

The four checks that separate running from working.

Known limitations

What is absent by design, so you stop looking for it.

Platform troubleshooting

When the problem is in the pipeline rather than the agent.

Introduction

Design and privilege

Install

Operate

Reference

Zero-code troubleshooting

The agent runs and collects nothing

Confirm

Why it happens

Fix

The agent refuses to start

Only some services appear

The agent only sees itself

401 Unauthorized in the agent log

429 Rate limited

Metric cardinality explosion

Kubernetes metadata is missing

Configuration changes have no effect

The kernel or platform is not supported

The agent instruments itself

Still stuck

Next

Verification

Known limitations

Platform troubleshooting

​The agent runs and collects nothing

​Confirm

​Why it happens

​Fix

​The agent refuses to start

​Only some services appear

​The agent only sees itself

​401 Unauthorized in the agent log

​429 Rate limited

​Metric cardinality explosion

​Kubernetes metadata is missing

​Configuration changes have no effect

​The kernel or platform is not supported

​The agent instruments itself

​Still stuck

​Next

Verification

Known limitations

Platform troubleshooting

The agent runs and collects nothing

Confirm

Why it happens

Fix

The agent refuses to start

Only some services appear

The agent only sees itself

401 Unauthorized in the agent log

429 Rate limited

Metric cardinality explosion

Kubernetes metadata is missing

Configuration changes have no effect

The kernel or platform is not supported

The agent instruments itself

Still stuck

Next