Setting up an AI provider

When: a customer wants the AI features on. AI is off in every install until this is done, and stays off if it is never done — no key, no AI.

Who: owner or admin. Choosing where mail content is sent for processing is a server-wide decision, not a per-domain one.

Before you start: decide whether mail content may leave the machine at all. If the answer is no, skip to Ollama — it is the only option that keeps everything local, and local-only mode enforces it.

On a brand-new server you can do all of this in the Setup Wizard instead, at its optional AI step, and skipping that step is what leaves AI off (first-boot-setup). It is the same form and the same stored configuration as the admin screen below; the curl calls here are the API underneath both. A headless install carries the same thing as the answers file’s optional ai block (headless-install).

Which provider

GET /api/v1/ai/providers returns the list the admin UI renders, with each entry’s endpoint, whether it needs a key, whether it can be used locally, and whether we have verified it.

Provider	Key from	Notes
Anthropic (Claude)	platform.claude.com	The only one with prompt caching, which makes long threads noticeably cheaper.
OpenAI	platform.openai.com	The reference chat-completions endpoint.
Google Gemini	ai.google.dev	Uses Google’s OpenAI-compatible route.
xAI (Grok)	docs.x.ai
Meta Llama API	llama.developer.meta.com	Waitlisted and regionally limited.
Groq	console.groq.com	Fastest of the hosted options for short inbox features.
vLLM	self-hosted	Your own GPU box. Works with local-only.
Ollama	self-hosted	Runs on the mail server itself. Works with local-only.
Other OpenAI-compatible	—	llama.cpp, LM Studio, OpenRouter, a corporate gateway.

“Verified” versus “compatible.” An entry marked verified has had every AI feature run against it. One marked compatible, untested speaks a protocol we implement and is expected to work, but we have not proven it. Both are supported configurations; only one carries a guarantee. Nothing is marked verified until a real run says so.

Model names come from your provider, not from us. Model ids change faster than our releases. Pick one from your provider’s documentation, save it, then use Test connection — the response lists what the endpoint actually offers and says whether the model you chose is among them.

Anthropic

Create a key at platform.claude.com → API keys.

Configure:

curl -X PUT https://mail.example.com/api/v1/ai/config \
  -H "authorization: Bearer $ADMIN_TOKEN" -H 'content-type: application/json' \
  -d '{"provider":"anthropic","model":"claude-haiku-4-5","apiKey":"sk-ant-…"}'

Test:

curl -X POST https://mail.example.com/api/v1/ai/config/test -H "authorization: Bearer $ADMIN_TOKEN"

{"ok":true,"models":[…],"modelAvailable":true} means the key works and the model exists.

An OpenAI-compatible provider

Identical, with provider set to openai, gemini, grok, llama, groq, vllm or openai-compatible. The endpoint is filled in from the catalogue; supply endpoint only for a self-hosted server or a gateway:

-d '{"provider":"gemini","model":"gemini-…","apiKey":"…"}'
-d '{"provider":"vllm","model":"…","endpoint":"http://gpu.internal:8000/v1"}'

Ollama: nothing leaves the server

Install Ollama on the mail server (or a machine on the same private network) and pull a model: ollama pull llama3.2. Model size decides which features work — see below.
Make sure the API container can reach it. localhost inside the container is the container itself, and the bundle defines neither an ollama service nor a host-gateway alias (extra_hosts: host-gateway) in compose/compose.yaml, so ollama and host.docker.internal resolve to nothing as shipped. Use the address of a machine on your private network (or the host’s address on the Docker bridge), or add an ollama service to the compose network yourself; the example below assumes you did.

Configure with local-only on:

-d '{"provider":"ollama","model":"llama3.2","endpoint":"http://ollama:11434","localOnly":true}'

localOnly is enforced, not advisory. A hosted provider is refused outright — by the request schema, as a 400 validation naming provider, before any address is looked up. An IP address must be private. localhost, a single-label name such as ollama and a name under a local-network suffix (.local, .internal, .lan, .home.arpa and the like) are trusted by name, without a lookup, so do not give one that points at a hosted API. Any other name is resolved, and every address it answers with must be on a private network. If you get ai_local_only_refused, that is the check working — the name you gave does not point where you think it does.

Which model to pull

Measured, not estimated — these are results from packages/ai/src/prompts/eval.test.ts, which you can run against your own model. Under Ollama the model decides what works, not the server: Test connection reports what the one you pulled can actually do, so check it there before assuming.

Model	Summaries	Explain email	Smart replies	Compose assist	Admin copilot
`llama3.2:1b`	yes	no	no	no	works, intermittent on args
`gemma3:4b`	yes	yes	no	yes	no tool support
`deepseek-r1:8b`	—	—	—	—	claims tools, does not call them
7–8B, tool-capable	expected	expected	expected	expected	expected

Five things worth knowing before choosing:

A 1B model summarises and little else. On the phishing fixture it answered unclear rather than judging it — the safe answer, and a useless one. Compose assist returned the text unchanged or with a “Here is the rewritten body:” preamble.
Smart replies need more than size. gemma3:4b handles everything else well but obeys instructions written in an email, returning attacker-chosen text as a ready-to-send suggestion. Do not enable smart replies on a small local model.
Tool support is per model, not per size. gemma3:4b (3.3 GB) has none and refuses the copilot outright; llama3.2:1b (1.3 GB) has it. ollama show <model> lists capabilities, and so does Test connection.
llama3.2:1b drives the copilot but is not a good one. It calls no-argument tools reliably and tools that take arguments only most of the time — one attempt came back as malformed text and produced no proposal at all, which is the safe failure but not a usable experience. Fine for proving a deployment works; pull something in the 7–8B tool-capable range for real use.
A model can claim tool support and not use it, which is worse than not having it. deepseek-r1:8b advertises tools, so the copilot offers itself — and then answers questions by inventing the data, formatted as JSON so it reads like a tool result. Asked for the domains on a one-domain server it returned three, one of them real. Verify a model yourself before trusting the copilot on it: ask “what domains are on this server?” and check that a listDomains line appears above the answer. The copilot labels any turn that ran no tools, so the check takes one question.

The Copilot page reads this for you and refuses rather than pretending: a model with no tool support gets “the configured model (<model>) does not support tool use, which the copilot needs”, not an empty answer. One thing it will tell you it cannot see on any model, because this build has no channel to it: live Rspamd statistics. Use rspamadm control stat inside that container for now. (The Postfix queue used to be the other; it is readable as of 2026-09-07 — see Mail queue (viewer, hold / release / delete).)

If the admin copilot matters, pull a 7–8B tool-capable model and verify it yourself:

AI_TEST_OLLAMA_ENDPOINT=http://localhost:11434 AI_TEST_OLLAMA_MODEL=<yours> \
  pnpm exec vitest run packages/ai/src/prompts/eval.test.ts

Before anyone can use it

Configuring a provider does not switch AI on for your users. Two more gates stand in the way, and both are deliberate:

Each domain is allowed by default, and an admin can veto one: PUT /api/v1/ai/domains/<id>/policy with {"enabled":false}. The same call sets allowAttachments, which is off by default — attachment text is not sent unless you turn it on for that domain. A vetoed domain is also left out of everything the admin copilot sends to the provider, and the copilot will not propose changes to it. Every AI record (calls, the copilot’s tool calls and proposals) is pruned after AI_LOG_RETENTION_DAYS (.env, default 30).
Each user opts in themselves, and nobody can do it for them. Until they do, GET /api/v1/ai/availability reports not_opted_in, and the webmail client shows none of the AI features — only a one-line notice that AI is available on this server but switched off for their mailbox, and only when every gate above the user is open (canOptIn).

GET /api/v1/ai/availability is the quickest way to see where a user stands — it names the closed gate rather than just saying no. Full detail in the feature-doc ai-privacy.

Turning it off

curl -X DELETE https://mail.example.com/api/v1/ai/config -H "authorization: Bearer $ADMIN_TOKEN"

Removing the configuration is the off switch; there is no separate toggle. The stored key is deleted with it.

Troubleshooting

Response	What it means	Fix
`ok:false, errorKind:"auth"`	The key was rejected.	Re-issue it at the provider and PUT it again.
`ok:false, errorKind:"network"`	The endpoint was unreachable.	For Ollama, check the container can reach the host — `localhost` inside a container is the container.
`ok:false, errorKind:"unsupported"`	Usually a model the endpoint does not have.	Compare against the `models` list in the test response.
`ok:false, errorKind:"rate_limit"`	Provider throttling or overloaded.	Retry; it is not a configuration problem.
`ok:false, errorKind:"billing"`	The key works but the account cannot pay — no credit, an exhausted quota.	Add credit or billing at the provider. Not retryable, and not a key problem.
`modelAvailable:false`	Key fine, model wrong.	Pick one from the returned `models`.
`400 ai_key_required`	This provider needs a key and none is stored.	Include `apiKey`. Note that changing `provider` deliberately drops the old key.
`400 ai_local_only_refused`	The endpoint is not verifiably private.	Use an address on a private network, or turn `localOnly` off deliberately.
`400 ai_key_unreadable` / `keyUnreadable: true`	The stored key cannot be decrypted — `SECRET_ENCRYPTION_KEY` was lost or replaced.	Restore `.env` from backup, or simply PUT the provider key again.

Editing without re-pasting the key

Omit apiKey to keep the stored one — changing the model does not need the key again. But changing provider drops it, because a key issued by one vendor is meaningless to another. Send "apiKey":"" to clear it deliberately.

What is stored, and what is logged

The key is encrypted at rest with the server’s SECRET_ENCRYPTION_KEY and never returned by the API — you get hasKey and the last four characters. The audit log records that the configuration changed, never the key. Back up .env along with the database: without SECRET_ENCRYPTION_KEY the stored key cannot be read, though re-entering it costs nothing.