Observability
Every agent run, tool call, and model response is captured as a trace. The Observability surface lets operators replay any execution end-to-end.
Nova Admin records every run as a trace — a tree of events spanning the triggering action, every model call, every tool invocation, every sandbox interaction, and the final output. The Observability surface is the operator's debugger.
What gets traced
| Event | Captured fields |
|---|---|
| Run start | Agent, trigger source, task, parent run (for delegations). |
| Model call | Model ID, input tokens, output tokens, latency, cost. |
| Tool call | Tool name, arguments, result, duration, error (if any). |
| Sandbox action | Sandbox ID, command, stdout/stderr, exit code. |
| Delegation | Child run ID, target agent, task passed in. |
| Run end | Final output, total tokens, total cost, success/error. |
The run timeline
Each run gets a full-page timeline view: collapsible groups for every model+tool turn, expandable JSON for arguments and results, and a side panel that shows the live system prompt and memory state at the moment of each step. If a tool call failed, the error and stack trace are inline.
Common debugging flows
- "Why did this agent call the wrong tool?" — open the run timeline, find the model call right before the bad tool invocation, and inspect the system prompt and the visible tool catalog at that point.
- "Why did the cron fire 12 times?" — open the cron job's run history. If you see runs spaced exactly one minute apart, the
lastRunAtcheck is broken (almost always a timestamp casting bug — see Flows). - "Why did the agent stop?" — sort by
errorin the run list. Token budget hits, sandbox timeouts, and tool failures all surface here.
Costs
Every trace rolls up into the Costs tab, broken down by agent, by model, and by day. Useful for budget alerts and for spotting agents that quietly went on a token-spending spree.
Retention
Traces live in Firestore and are not auto-pruned. The daily-maintenance cron removes traces older than 90 days; flip that knob in apps/admin/functions/src/daily-maintenance.ts if you need a different policy.
Integrations
Integrations are credentialled bridges from agents to external services. The platform ships with 60+ providers, each contributing typed actions to the tool registry.
Model Evaluation & Optimization
How Nova routes requests across 13 LLM providers, measures quality with a replay eval harness, and proposes cost-optimal model swaps for human review.