Your coding agent can do everything on this page for you: see the Quickstart. This page says
what it sets, and why.
- Chat Completions: OpenAI’s SDK (
chat.completions), any OpenAI-compatible client, or a framework configured with an OpenAI base URL. - Messages: Anthropic’s SDK (
messages.create,messages.stream), or a framework or agent built on it.
Workloads
A workload is one kind of task your agent repeats, such assupport-agent, and the model that serves it. Every
request names its workload, and the workload decides which model answers, so that model can change later with no
change to your agent. If your product runs different kinds of agent, a support agent and a coding agent, say,
each is its own workload, even when they share the code that calls the model.
List the org’s workloads and the models a workload can serve, then create one:
., _ and -, starting with a letter or digit, at most 63
characters. It never changes.
Start with a large frontier model, generally. The replies it records are what every replay run replays, and the
bar candidates are measured against: a scorer that compares each candidate’s reply with the recorded one can’t
tell you whether a cheaper model does as well as a strong one would, when the recorded reply is weak.
Titles and labels
When your code creates workloads itself, one per tenant or per generated agent, their names are often generated (agent-3f9c2a1e). Give each a title people can read, and label it with what someone will look for it by: the
tenant it serves, its tier, or the team that owns it.
key=value, or a bare key as a tag: --label beta matches any value of beta. Keep labels to
values that repeat across workloads, since an id unique to one workload belongs in its name.
Retitle a workload, and change its labels when what it serves does:
The Gateway’s base URL and key
ambral workloads get prints everything a client needs to connect to a workload. Use the values it prints:
--json, they are its connection:
The key
Requests carry an Ambral key, not a model provider’s key such as OpenAI’s or Anthropic’s. Create one on the API keys page of the web app, which shows it once, and store it where your agent reads its secrets: a.env file that git ignores, or your secret store. The CLI
can’t create a key, so the secret never passes through a terminal. Don’t paste it to a coding agent, which never
needs to see it.
Set up the client
Where your agent builds its client, set:
Vercel’s AI SDK provider for Anthropic (
@ai-sdk/anthropic) adds only /messages, so its base URL is
messagesBaseUrl with /v1 added.
A Chat Completions request then reads:
ambral/auto:support-agent. A request that names it both ways must name
the same workload. A request that names a provider’s model is refused with a 400: to serve another model, make it
the workload’s primary with ambral workloads set-primary.
The Gateway answers in the response format of the API it was called in, with model the model that served the
call. It answers errors in that API’s error format too, so your SDK raises its usual error, and the message says
what’s wrong: see Errors.
Session ids
A session is each time your agent does its task: a chat with a user, or one pass of an automated workflow, however many model calls it takes. Ambral records, evaluates and routes a session as a whole, so each request has to say which session it belongs to. Sendx-session-id on every request: the same id on each request of a session, and a new id for the next
session. An id is 8 to 128 characters of A-Z, a-z, 0-9, _ and -, so a UUID fits, and so does an id your
app already holds for the conversation. Any other value is ignored.
The session id is the easy one to miss. Without it, every request is recorded as a trace of its own, one step
long. The Gateway answers such a request with an x-session-id response header, the id of the session it made
for it.
A client that already names its session another way needs nothing more: the Gateway also reads session_id in
the request body, and a header named x-<vendor>-session-id.
A session is stored once it has gone quiet for a little while, so a new one takes about half a minute to appear
among the workload’s traces.
End users and labels
Two optional headers say more about each session. Send them where your agent sendsx-session-id, on every
request. Neither ever makes a request fail.
x-ambral-end-user
It names the person the session served, by your own id for them: a customer or account id such as cus_81723,
1 to 128 letters, digits and . _ : @ + -. Send it when your agent serves people, so each person’s sessions can
be found later. Prefer an opaque id to a name or an email.
Without the header, the Gateway reads the request body: safety_identifier or user (OpenAI’s), then
metadata.user_id (Anthropic’s). An agent that already sends one of those needs nothing more. A value that
doesn’t fit is ignored.
A session has one end user: the first one it was sent. Keep one session id per person’s conversation. A session
id reused for another person is still recorded, under the first person.
x-ambral-labels
It labels the session with key=value pairs or bare tags, comma-separated, up to 16:
. _ : / @ + -, at most 128
characters. A session gathers the labels of every request, and a later value for a key replaces the earlier one.
Once a session holds 16 labels, a later request’s new keys are dropped.
Label what tells one workload’s sessions apart:
- The deployment that sent it. Production sends
env=prod, a staging deploymentenv=staging, and a developer trying the agent by handenv=dev. Set the value from your app’s own environment, in the client’s default headers. Then--label env=prodshows only real people’s sessions, to read and to build evals from. - A change that ships to some of the traffic. Add a label that names it, such as
variant=prompt-v2, and compare its sessions with the others. A model change needs no label: each trace records the model that served it.
x-ambral-end-user, and the tenant
or team a whole workload serves goes in the workload’s own labels.
When a label doesn’t fit, the Gateway ignores the whole header. It still serves and records the request, without
labels, and says why in a response header: x-ambral-labels: ignored; reason=<reason>, where the reason is
invalid_key, invalid_value or too_many. Check a successful response for it once, after you add the header.
Find sessions
ambral traces list lists the org’s sessions, newest first, across every workload. Name a workload to keep only
its sessions, and narrow either with flags, each of which must hold:
--label matches a session’s own labels, not its workload’s. To find a tenant’s or team’s sessions, list their
workloads with ambral workloads list --label tenant=acme, then each workload’s sessions.
A test suite that runs your agent isn’t production traffic, and needs no env label once it reports through one
of Ambral’s SDKs: its sessions are recorded as the traces of its offline run, and
ambral traces list never lists them. Until a suite is wired that way, label its sessions env=test.
The system prompt
Ambral reads a request’s first message, when it is a system message, as your agent’s prompt: the instructions that stay the same from one session to the next. Everything after it, later system messages included, is the session’s own. Each session’s prompt version, whichambral traces get shows, is a fingerprint of that first
message and the tools, the same for every session sent with the same instructions.
So your agent’s requests should keep to that layout:
- The first system message is fixed. It holds the instructions, the same text on every request of the workload, changing only when the instructions do.
- What changes between sessions comes after it. The customer or account, retrieved documents, today’s date:
- in Chat Completions, in a later system message, or in the user message;
- in Messages, at the start of the first user message. Ambral reads
systemas one system message, however many blocks it has, so keepsystemto the instructions, in one text block.
Check the first trace
Run your agent through one short session of a few steps, wait half a minute, then list the workload’s traces:
Then read the trace whole, with the id the list printed:
When a session has more than one branch
A session has more than one branch when your agent sent a history that differs from what it was answered. A session can look healthy by every other check and still fork at every reply, so find out why before you rely on its traces.ambral traces get prints each branch: look at where each splits off the latest one.
- Your agent drops the model’s reasoning. This is the usual cause. The branches split at a reply: the same
reply, with the same text and the same tool calls, appears twice under the same message, and a session of ten
replies has around ten branches. A reply the model reasoned for comes back with that reasoning: in
reasoning_details(and inreasoning, when it is text) from Chat Completions, and asthinkingandredacted_thinkingblocks from Messages. Many frameworks and hand-written loops send the reply back without it, rebuilt fromrole,contentandtool_calls, so it isn’t the reply the model gave. The model then loses its earlier reasoning on every turn, which can lower the quality of multi-step work, and some providers refuse such a request outright. Where your agent appends the model’s reply to the conversation, append the assistant message as it came back, with those fields or blocks and each block’ssignature. - Your agent rewrites its history on purpose. It retries or regenerates a reply, summarizes or trims older messages when the conversation grows long, or edits a message before sending it again. Each rewrite starts a branch, and that is expected.
- A call failed, and your agent sent something else next. The failed call is left as a branch of its own, with its error.
- A voice platform builds the history. Vapi sends back its own version of each reply, so the two copies
differ: in their words, when it is its transcriber’s text of the audio, or only in their times and numbers,
when it is the reply rewritten for speech. The fix is in the Vapi assistant’s settings. For the transcriber’s
text, set
modelOutputInMessagesEnabledtotrue, which works only for a single assistant with an ElevenLabs voice. For the rewriting, setvoice.chunkPlan.formatPlan.enabledtofalse.
When no trace appears
Go through these in order:- The request failed. A 401 means the Ambral key is wrong or isn’t being sent: check that its secret is set
where your agent runs. A 402 means the org has no credits available. A 400 that asks for a workload means
x-workloadis missing, and a 404 that it isn’t the workload’s name asambral workloads getshows it. - The base URL. It must be the one
ambral workloads getprints for the client’s API:baseUrl,/v1included, for Chat Completions, andmessagesBaseUrlfor Anthropic’s SDK. Swapping them sends requests to a path the Gateway doesn’t serve. - The model. A request that names a provider’s model in place of
ambral/autois refused with a 400. - The tools. A response header
x-ambral-recording: skipped; reason=unsupported_toolsmeans a tool your agent offers can’t be recorded.
https://api.ambral.ai/v1/openapi.json, which needs no key.