Configuring AI providers

The language-model features of aiAxonIQ — Investigate, Ask your data, Root cause, Suggested fixes, Correlation and the Knowledge base — call a model provider that you configure on the AI service. This page is for operators of a self-hosted deployment.

Using aiAxonIQ as a hosted service? You do not configure a provider. Which model-backed features are available depends on your plan and deployment; each page in the AI Assistant section says when its prerequisite is missing. See AI Assistant and Investigate for what each page needs.

Nothing here is needed for anomaly detection or forecasting. Both are statistical and work with no provider configured. With no provider, the AI service starts normally and the model-backed pages refuse with a stated reason rather than failing silently.

Supported providers

The provider is read in one place in the AI service, from these variables:

`LLM_PROVIDER`	API used	Needs `LLM_API_KEY`	Default model
`gemini` (the default)	Google Gemini	Yes	`gemini-2.0-flash`
`openai`	OpenAI chat completions	Yes	`gpt-4o-mini`
`openrouter`	OpenRouter, through the OpenAI-compatible API	Yes	`gpt-4o-mini`
`ollama`	A local Ollama server, through its OpenAI-compatible API	No	`llama3`

The value is case-insensitive, and may be written <provider>/<model> — for example openai/gpt-4o — which names the model in the same breath.

Correlation and the Knowledge base need LLM_PROVIDER=openai. They rank by embeddings, and the only embedding model the AI service uses is OpenAI’s. With any other provider those pages — and the runbook half of Suggested fixes — say Semantic ranking is unavailable rather than ranking by something that is not similarity.

Settings on the AI service

Variable	Default	What it does
`LLM_PROVIDER`	`gemini`	The provider, optionally `<provider>/<model>`.
`LLM_API_KEY`	empty	The provider’s API key. Not needed for `ollama`.
`LLM_MODEL`	empty (the provider’s default model)	The model. The `/<model>` suffix on `LLM_PROVIDER` wins if both are set. Set this when a vendor retires a model.
`LLM_BASE_URL`	empty (derived from the provider)	Overrides the OpenAI-compatible base URL — the way to reach a gateway or proxy. A trailing slash is removed.
`LLM_EGRESS_ALLOWLIST`	empty	Comma-separated hosts added to the hosts `LLM_BASE_URL` may point at. See below.
`OLLAMA_URL`	`http://localhost:11434`	The Ollama server, used only when `LLM_PROVIDER=ollama`.
`LLM_MAX_DECISION_TOKENS`	`1200`	The reply budget for one Investigate step.
`CH_AI_MASTER_SECRET`	none	Required for Ask your data. The per-organization database identity a generated query runs as is derived from it. At least 32 characters and not a published placeholder, or those queries are refused. Generate one with `openssl rand -hex 32`.

LLM_BASE_URL is checked before anything is sent. Its host must be one of api.openai.com, generativelanguage.googleapis.com, openrouter.ai, api.anthropic.com, localhost or 127.0.0.1, or a host you add in LLM_EGRESS_ALLOWLIST. Otherwise the investigation step is refused with a message naming the host. This guards against a typo sending telemetry to a host nobody chose; it is not a defence against someone who can edit your environment.

The check applies to Investigate only. Root cause, Suggested fixes, Ask your data and the Knowledge base call the provider without it. Set LLM_BASE_URL with the same care either way.

Settings on the API service

These tune the investigation agent, which runs in the API service.

Variable	Default	What it does
`AGENT_CONCURRENCY`	`8`	Investigations run at once, per API instance.
`AGENT_TENANT_MAX_ACTIVE`	3 × `AGENT_CONCURRENCY` (24)	Investigations one organization may have queued or running (a run waiting on a person does not count). Above it, a new one is refused with a retry hint. `0` means no limit.
`AGENT_GLOBAL_MAX_QUEUED`	`500`	Deployment-wide backstop on queued investigations. `0` means no limit.
`AGENT_MAX_PROMPT_TOKENS`	unset (64,000)	Lowers the model-input budget for investigations started automatically when an alert opens an incident. Lower it if your model accepts less. It can only lower the limit.

The per-run limits — steps, tool calls, time, tokens, estimated cost — are fixed defaults, listed in How the investigation agent is governed.

Governance settings

Variable	Default	What it does
`AGENT_EVIDENCE_MODE`	`off`	`required` records every investigation tool call in the evidence ledger — an intent before it runs, an outcome after — and stops the run if a record cannot be written. It is also what makes approvals possible. `off` records nothing, and no call that needs an approval can run. Any other value stops the API from starting.
`EVIDENCE_SIGNING_KEY`	empty	Signs evidence-ledger entries. `AGENT_EVIDENCE_MODE=required` without it stops the API from starting, naming both variables. Generate one with `openssl rand -base64 32`.

Keep a copy of EVIDENCE_SIGNING_KEY somewhere other than the host. It cannot be reissued: entries signed with it can only be verified with it. See Self-hosted hardening.

Keep the AI service private

The AI service must be reachable only by the aiAxonIQ API. The browser never calls it: every AI page goes through the API under /api/ai/* and the investigation routes, which authenticate the user and attach the organization.

Do not route to it from your reverse proxy, and do not publish its port beyond the host it runs on.
The API and the AI service authenticate each other with a shared internal key. It must be set, and strong, on both; the deployment material that ships with your installation names it.

The AI service holds the provider key and receives telemetry on your organizations’ behalf, which is why it must not be reachable by anything else. See Platform architecture.

Check the configuration

Read the startup log

When the AI service starts without a usable key it logs a warning box beginning LLM_API_KEY is not set!. With one, it logs a line of the form LLM configured: provider=…, model=…, key=***… showing only the last four characters of the key.

Ask the AI service

From a machine that can reach the AI service — normally the host it runs on, since it must not be reachable from anywhere else — request its health endpoint. Replace <ai-service-address> with the address your installation gives it:

curl -s http://<ai-service-address>/health

The answer includes the provider and whether it is configured:

{
  "status": "ok",
  "llm_provider": "gemini",
  "llm_configured": false,
  "redis_connected": true
}

llm_configured is true when LLM_API_KEY is set, or when the provider is ollama, which needs no key. The response has further fields, such as the forecast worker pool; status is degraded, with HTTP 503, when the AI service’s cache is unreachable.

Use a page

Open AI Assistant → Root cause and analyse a short log snippet. A configured provider returns an answer ending Generated by <provider>.

GET /api/ai/health answers a different question. It is available to Admin and above, and reports whether the API can reach the AI service and how quickly. Its configured field is always true — it means the API has an address for the AI service, not that a model is configured. Use llm_configured above for that.

What is sent to the provider

Only when a model-backed feature is used, and only what that feature needs:

Feature	Sent to the provider
Investigate	Your question and the conversation so far, the list of tools the run may use, and the results of the tool calls made so far — which can include log lines, span details, metric values and service names from your organization. Values the capture policy classifies as credentials are redacted first.
Ask your data	Your question and a fixed description of the data model. The query results are not sent.
Root cause	The snippet and service name you entered.
Suggested fixes	The snippet you entered and, for the runbook half, matching Knowledge base entries.
Correlation, Knowledge base	The text being searched or indexed, sent to OpenAI’s embeddings API.

Anomaly detection and forecasting send nothing to a provider. The model’s reasoning is never stored or streamed to the browser — see How the investigation agent is governed.

The provider’s own terms govern what it keeps. Whether it retains or trains on what it is sent is set by your agreement with that provider, not by aiAxonIQ. If telemetry must not leave your network, use LLM_PROVIDER=ollama with a model you host.

With gemini, the key travels in the request URL, as Google’s API requires. A proxy between the AI service and Google that logs URLs will log the key.

Troubleshooting

llm_configured is true but pages still refuse

Check which page. Correlation and the Knowledge base also need LLM_PROVIDER=openai. Ask your data also needs CH_AI_MASTER_SECRET.

Investigations fail with llm_unavailable

The AI service answered with an error or could not be reached. The same code appears whether the service is down or simply has no key — check llm_configured first, then the AI service’s log for the provider’s response.

Investigate refuses a gateway URL

LLM_BASE_URL points at a host outside the allow-list. Add the host to LLM_EGRESS_ALLOWLIST if it is deliberate.

Approvals are refused with 'Approvals need the evidence ledger'

AGENT_EVIDENCE_MODE is off. Set it to required with an EVIDENCE_SIGNING_KEY. No registered tool needs an approval today.

A setting in the environment file has no effect

Settings are read when the service starts. Restart the AI service (for LLM_* and CH_AI_MASTER_SECRET) or the API (for AGENT_* and EVIDENCE_*) after changing one.

AI Assistant and Investigate

What each page needs and shows.

How the agent is governed

Tools, approvals, limits and the audit trail.

Self-hosted hardening

Secrets, network exposure and backups.

Logs

APM, metrics and traces

Infrastructure

Real user monitoring

Visualize

Detect and respond

AI and LLM

Integrations

Account

Configuring AI providers

Supported providers

Settings on the AI service

Settings on the API service

Governance settings

Keep the AI service private

Check the configuration

What is sent to the provider

Troubleshooting

Next

AI Assistant and Investigate

How the agent is governed

Self-hosted hardening

​Supported providers

​Settings on the AI service

​Settings on the API service

​Governance settings

​Keep the AI service private

​Check the configuration

​What is sent to the provider

​Troubleshooting

​Next

AI Assistant and Investigate

How the agent is governed

Self-hosted hardening

Supported providers

Settings on the AI service

Settings on the API service

Governance settings

Keep the AI service private

Check the configuration

What is sent to the provider

Troubleshooting

Next