HTTP chat endpoint — POST /api/chat¶
Why¶
The runtime was Discord-in, Discord-out. POST /webhook/{source} exists but is
GitHub-event specific: it formats an event into a prompt and dispatches it, it does not
do request/reply.
That made Discord the only place a human could hold a conversation with an agent. Rig
Cockpit (rig-cockpit-ios) had no way to reach one, and a Cockpit chat surface — open
the app, pick a tenant agent, talk to it — is the goal this unblocks.
What¶
createDashboard(character, memory, { webhookHandler, chatHandler }) now accepts a chat
handler alongside the existing webhook handler, and serves it at POST /api/chat.
The agent loop is untouched. The handler calls the same agent.process() the Discord
path calls, with the same context shape, so providers, tools, MCP servers and memory all
behave identically.
Request¶
Response¶
model (#760) is the model that produced the reply, as the provider reports it. It is
additive; reply and sessionId are unchanged.
| Provider | model value |
|---|---|
openai-api |
The upstream completion's model field (what the server actually served, e.g. a vLLM served-model-name). Falls back to the resolved llm.model when the server omits it. |
claude-cli, claude-tmux |
The resolved llm.model they already report on their result object (null if unset). |
codex-cli, anthropic (bare-string replies) |
null |
| Rule | Detail |
|---|---|
| Never a literal | chat.js only passes through what the provider returned. It never fills the field in from config itself. |
| Model turns only | Present (string or null) on every reply the agent produced. Runtime-authored 200s (the wake and context-overflow replies) and errors do not carry it: no model answered them. |
| Logged per turn | [chat] turn answered by model <model \| unknown> |
| Scope | openai-api returns { text, model } only when context.chatMode is set. Webhook, cron, worker and run-once callers still get a string. |
sessionId is passed straight through to agent.process()'s context — the same field
stream-consumer.js already uses — so multi-turn continuity needs no new machinery. A
session id is generated when the caller omits one.
Foreground Wake Continuation (#715)¶
An optional wakeMode accepts wait or resume; other supplied values return
400. Omission preserves the existing request/reply contract. Both modes check
the active provider before invoking the agent and pin that provider for the turn.
For the conductor-backed OpenAI provider, this uses a fresh pod lookup and
requires a safe endpoint plus a valid, non-future readySince. Missing provider
credentials fail before any wake. Neither mode retries a completion POST or
falls back to another provider after a failure.
wait may add pendingWake: { "pod": "qwen" } to the usual 200 response. Its
presence means the agent has not been invoked: no completion POST, tool call or
conversation write has happened. It is never inferred from reply text or a late
provider error. The existing wake cooldown still limits start requests; unreadable
or unconfigured status never produces this marker.
The foreground caller retains the original prompt and session, polls authorized
pod status with a deadline, then sends one resume after fresh readiness.
resume rechecks readiness, never wakes a pod and never returns pendingWake.
A stop/readiness race therefore ends the turn without another paid start. The
caller must consume the continuation before sending, cancel on ownership or
foreground changes, and never automatically retry an ambiguous response.
There is no server queue, durable replay, or new route; unchanged clients still
receive the normal 200 reply and session fields.
Status Codes¶
| Code | Meaning |
|---|---|
| 200 | reply produced |
| 400 | body is not JSON, or message is missing/empty/not a string |
| 401 | CHAT_API_TOKEN is set and X-Chat-Token does not match |
| 413 | body over 32 KB |
| 429 | a chat is already in flight (this conversation is mid turn) or too many chats in flight (the pod is at CHAT_MAX_CONCURRENT) |
| 502 | the provider failed |
| 503 | CHAT_API_TOKEN is unset — the route is disabled |
Decisions¶
Fails closed. Every call drives an LLM, so the route is cost-incurring. It stays
disabled and returns 503 until CHAT_API_TOKEN is set, mirroring rig-conductor's
RunPodClient.IsConfigured, which 503s its pod-control routes until the operator
provisions the secret. The alternative — ship open, harden later — leaves a window in
which an unauthenticated caller can spend the agent's quota.
The token comparison is constant-time and tolerates a length mismatch rather than
throwing, and an unset token is never satisfied by an unset header. That last one is the
trap worth naming: a naive configured === presented is true when both are undefined,
which would silently open the route on any pod without the secret.
One chat in flight per conversation, and a cap on the total. This was a single global boolean. It guarded two different things at once, and conflating them serialised conversations that share no state.
The per conversation half is a correctness rule and is not configurable: memory.saveMessage
writes the user turn and the assistant turn as two separate non transactional statements,
so two turns interleaving on one thread_id would corrupt that history. A second turn in
the same conversation is always 429 a chat is already in flight.
The total is a resource rule and is configurable through CHAT_MAX_CONCURRENT, default 1.
claude-cli and codex-cli fork a subprocess per turn, so unbounded parallel chats there
are a one line way to exhaust the host, and they should stay at 1. openai-api is a single
HTTPS call to a remote GPU and vLLM batches concurrent requests itself, so an agent on that
provider (Rig-E) can raise it. Callers past the ceiling get 429 too many chats in flight.
Nothing is queued; a refusal is immediate either way.
An unparseable or below 1 value falls back to 1 rather than widening the guard. The slot is
released in a finally, because releasing only on success means the first provider error
wedges that conversation at 429 for the lifetime of the pod.
Provider errors are not echoed to the caller. They routinely carry env var names, credential paths and prompt fragments. The detail is logged; the response says only that the provider failed. This reply crosses a network boundary to a phone.
No new public surface. The dashboard service stays ClusterIP. Cockpit reaches this
through rig-cockpit-worker, which is already the app's only edge.
Configuration¶
| Variable | Effect |
|---|---|
CHAT_API_TOKEN |
Shared secret. Unset disables the route entirely. |
CHAT_MAX_CONCURRENT |
How many DIFFERENT conversations may run at once. Default 1, which is the single global slot every agent had before. Unparseable or below 1 falls back to 1 rather than widening the guard. |
Callers present CHAT_API_TOKEN as X-Chat-Token.
Leave CHAT_MAX_CONCURRENT at 1 on any agent using a CLI provider: claude-cli and
codex-cli fork a subprocess per turn, so parallel chats there multiply host load. Raise
it only where a turn is a network call, such as openai-api against a vLLM pod, which
batches concurrent requests itself. It never affects one conversation talking to itself,
which is refused at any value.
Not in this change¶
- Streaming. A turn is a single request/reply. Long turns will feel slow in the app before they feel wrong; streaming is the follow-up if that becomes the complaint.
- Per-tenant authorization. One token, one agent, one pod. Tenant separation today is deployment-level: each tenant agent is its own release with its own token. That is sufficient while each pod serves exactly one character, and it is not sufficient once an agent holds credentials — see the note below.
Follow-up this exposes¶
In this runtime the persona and the tool grant live in the same file. character.json
carries both the personality and the tools block that decides what the agent may reach,
and buildTools() turns those endpoint declarations into callable tools with no policy
layer in between.
That is fine for a read-only agent. It stops being fine when an agent holds credentials, because the file describing how the agent talks is also the file that authorises what it can touch. A separate policy document — one the agent cannot see or edit, where deny wins — is the next structural piece, and it does not exist yet.