Skip to main content

Agent Observability

Read one agent run in Traces

Open an agent-run trace in Agent Observability: which runs you are allowed to see, the list and its filters, the round-by-round timeline, the event inspector, and the trace assistant.

/traces is where the Agent Observability blocklet (blocklets/observability-explorer in the ARC monorepo) answers the question the dashboard cannot: what actually happened inside this run?

Verified against the live deployment at https://observability-explorer.afsd.dev, build 35de3a0b, signed in as a registered test user on 2026-08-20. Every screenshot below is that session.

The sibling sections are the dashboard — what ran lately — and AI usage — how much capacity was consumed. Traces and usage are written by different paths and do not reconcile with each other.

Which runs you can see

A trace carries the task text, the system prompt and every tool result, so read the gate before reading the list. Two axes admit a run, and nothing else does:

  • Your own runs, always — that data is a fragment of your own DID Space.
  • Every run of an app you administer — same-instance admin/owner, probed per app.

Being owner of one app grants nothing on another. That has only been true since 2026-08-18: before ARC #4170 / PR #4269, any owner/admin role was read as deployment-wide privilege here, and one app's owner could enumerate every app's runs — while the usage admin face refused the same caller for the same app. If you are on an older build, this section is not the boundary you actually have.

An app you may not see is indistinguishable from one that does not exist: the listing is empty, not an error. The dashboard page carries the worked comparison that shows both axes from a single account.

The list

The traces list: search box, blocklet filter, time filter, and one row per run with user input, blocklet, status, model, in/out/cache tokens, duration and start time

Newest first, one row per agent run, with three ways to narrow it:

ControlEffect
SearchFree text over task and status
BLOCKLETAll blocklets, or one app
TIMEAll · Today · 7d · 30d

The columns are USER INPUT · BLOCKLET · STATUS · MODEL · IN · OUT · CACHE · TIME · STARTED. Two things about them are worth knowing before you scan the list:

  • USER INPUT is the raw prompt, conversation history included. Rows from a chat surface start with <conversation_history> …, so the first line of the cell is often the previous exchange rather than the request that started this run. The full text is in the trace.
  • STATUS is ok or interrupted. A run that never reached the model shows interrupted with 0 in / 0 out and an empty duration — the same runs the dashboard counts as "failed".

One run

Clicking a row opens the run at its own addressable URL:

text
/traces/<blocklet>/<date>/<runId>
   e.g. /traces/todo/2026-08-16/195318-dd5abfb7

A run detail: header metrics, the timeline with INPUT, USER, ROUND 1, ROUND 2 and FINAL OUTPUT, the trace assistant below it, and the inspector showing the run's system prompt

Header. 3 rounds · 3 tool calls · 16851→164 tokens · 8153 cached · 50.8s · gemini-2-5-pro, plus Copy link — the run URL is shareable and hard-loadable.

Timeline (left). One row per step, with per-row IN / OUT / CACHE / TIME:

RowWhat it is
⚙️ INPUTThe call that started the run: action path, tool count, whether a system prompt was attached
👤 USERThe prompt text
🔄 ROUND nOne model round, labelled with the tool calls it made
🤖 FINAL OUTPUTThe answer, with the tokens that produced it

Note the arithmetic: a run reported as 3 rounds shows ROUND 1, ROUND 2 and then FINAL OUTPUT — the final answer is the third round, not a fourth step.

Inspector (right). Whatever row is selected, in full: type, time, and for the opening event the complete SYSTEM PROMPT, TOOLS grant and INPUT PARAMETERS (including the run budget — max_rounds, actions_per_round, total_tokens).

Expanding a round

A round with a chevron expands into the steps inside it:

ROUND 2 expanded to show TOOL USE and the two TOOL RESULT rows beneath it

🛠 TOOL USE is what the model asked for; each 🔧 TOOL RESULT is one call's outcome and its duration — afs_write … · ok in the screenshot. This is the level at which "the agent did the wrong thing" becomes visible.

A run that never started

An interrupted run: 0 rounds, 0 tool calls, 0→0 tokens, and a timeline with only INPUT and USER

An interrupted run keeps its INPUT and USER rows and has nothing after them. The inspector still shows the tools it was granted and the parameters it was given, which is usually enough to tell whether the run was mis-configured or the model was never reached.

The trace assistant

Below the timeline, an assistant is loaded with this trace as context. It is present only inside a run — the list, the dashboard and the usage pages do not show it.

The trace assistant answering "where do the tokens go?" for the open run, with a token footer under the answer

Four canned questions cover the common cases (这条 run 哪里有问题? · token 大头在哪?为什么? · 哪几轮可以省? · 总体分析一下), and the box below takes anything else. Its answer carries its own cost footer — 7877 tokens | 7393 in | 484 out | gemini-2-5-pro in the screenshot.

Asking the assistant is itself an AI call, billed to you and visible afterwards in the usage face under the observability-explorer app. Measured on this walk: two questions moved that app's row from 5 calls to 8.

Failures to expect

SymptomWhat it actually is
Sign in to view this content.Not an error — traces have no anonymous projection
Inspector reads Select a row to inspectClick any timeline row and it fills — see the note below
A row's first line is someone else's questionUSER INPUT includes conversation history; open the run for the real prompt
interrupted with 0 tokensThe run never reached the model
Round count is one higher than the ROUND n rowsThe final output is the last round

The empty inspector: no longer reproducible

On build 400ce96 the inspector came up empty after a multi-hop path — drill into a run, leave for /dashboard and /usage, come back to /traces and drill again — and stayed on Select a row to inspect indefinitely (3/3 at the time). On build 35de3a0b it does not reproduce: the same hop sequence, including the intermediate back-navigation, was driven three times and the inspector filled with the run's opening event every time.

The blocklet's own man page still records this as an open bug, and cites an issue number that belongs to a closed QA report — filed back to ARC so the note stops outliving the defect. If you do land on an empty inspector, clicking any timeline row fills it; the run itself is never the thing that broke.