Observability & Debugging
Invariant exposes two different records:
- Runtime events are durable facts used to derive execution state.
- Diagnostic traces explain how handlers, models, and validation boundaries behaved. Traces do not decide execution truth.
Enable Console Tracing
ts
import { consoleTraceSink, invariant } from "@invariant-tech/sdk";
const app = invariant({
observability: {
level: "debug",
sinks: [consoleTraceSink({ level: "debug" })],
capture: {
reasoningBoundary: true,
},
},
});debug: true also installs a console sink, but reasoning payloads remain metadata-only by default. Set capture.reasoningBoundary: true only when the sink is approved to receive prompts, projected context, tools, and raw model responses.
Implement a Custom Trace Sink
A sink implements one non-interfering method. It may write synchronously or return a Promise:
ts
import type { InvariantTraceEvent, TraceSink } from "@invariant-tech/sdk";
export class ReasoningBoundaryJsonLineTraceSink implements TraceSink {
constructor(
private readonly writeLine: (line: string) => void | Promise<void>,
) {}
write(event: InvariantTraceEvent): void | Promise<void> {
if (event.type === "llm.call.started") {
return this.writeLine(JSON.stringify({
type: event.type,
callId: event.callId,
source: event.source,
model: event.model,
systemInstruction: event.request?.systemInstruction,
context: event.request?.context,
semanticActions: event.request?.semanticActions,
providerTools: event.request?.providerTools,
}));
}
if (event.type === "llm.call.completed") {
return this.writeLine(JSON.stringify({
type: event.type,
callId: event.callId,
response: event.response,
usage: event.usage,
durationMs: event.durationMs,
}));
}
return this.writeLine(JSON.stringify({ type: event.type, timestamp: event.timestamp }));
}
}Register an instance under observability.sinks and enable capture.reasoningBoundary to receive the optional request and response fields. The console card shows the complete system instruction, the context projection sent to the adapter, provider-facing tool declarations, the adapter's normalized response, and—after an accepted Agent action—the resulting runtime projection. Provider-native tool-result messages are not manufactured: an Agent action is validated and executed by the runtime, and the next model turn receives fresh state through its projected context. Runtime outcome remains a separate authoritative event (REASON_COMPLETED or REASON_FAILED) correlated by workflow/node metadata; never treat a diagnostic trace as execution truth.
Apply redaction, access control, and retention policy before storing these fields. TraceEmitter isolates synchronous throws and rejected Promises from execution, so a telemetry outage does not roll back or invalidate authoritative workflow progress. A sink is an observer: do not use it to perform business effects.
What You Can Inspect
Current SDK execution paths emit diagnostics for:
- Agent and workflow
.reason()call start, completion, failure, duration, and available token usage, - projected context and provider tool declarations when capture is enabled,
- Agent action receipt, acceptance, rejection, and validation details,
- workflow start and input rejection,
- node entry, completion, and failure,
- capability dispatch, completion, and failure,
- wait input receipt, validation rejection, and boundary conflict.
Not every event shape exported by the trace type union is emitted by every adapter. Treat the sink output from your chosen execution path as the operational contract.
Workflow Reason Traces
Every .reason() attempt emits the diagnostic LLM lifecycle, distinguished from Agent calls by source.kind: "reason":
| Outcome | Diagnostic traces | Durable execution event |
|---|---|---|
| Attempt begins | llm.call.started | None yet |
| Valid structured result | llm.call.completed, then node.completed | REASON_COMPLETED |
| Missing adapter, provider failure, or invalid result | llm.call.failed | REASON_FAILED |
The llm.call.* records are optional telemetry delivered to sinks. REASON_COMPLETED and REASON_FAILED are authoritative runtime events committed through the configured store. Reasoning request and raw response payloads appear only when capture.reasoningBoundary is enabled; metadata, duration, model identity, and available token usage remain observable by default.
A deterministic .step() emits node.completed with STEP_COMPLETED only after its handler returns an object. A missing handler, thrown error, or non-object result emits node.failed and the durable STEP_FAILED fact; it never emits a synthetic completion.
For conversational agent.run(), a missing or failed adapter emits llm.call.started followed by llm.call.failed and rejects with AgentModelError. Agent turns do not create REASON_FAILED durable events because they are not workflow .reason() nodes. A failed Agent call never emits llm.call.completed and never returns a synthetic greeting.
Agent Protocol Rejections over HTTP
agent.handleAction() returns an AgentActionResult; it does not choose an HTTP status. For a persistent model protocol, map a well-formed but unauthorized or schema-invalid action to HTTP 200 so the structured rejection can return to the model as semantic feedback:
ts
import type { AgentActionResult } from "@invariant-tech/sdk";
export function protocolHttpResponse(result: AgentActionResult) {
return {
status: 200 as const,
body: result,
};
}Reserve HTTP 4xx for failures at the application's transport boundary, such as malformed JSON, authentication failure, or a session identifier the route cannot resolve. This mapping is application policy, not an HTTP server supplied by Invariant.
An accepted result can contain execution.status: "failed": the proposal crossed authority and its durable consequence was failure. A database/socket/Host exception instead rejects the handler Promise and produces no AgentActionResult; do not catch it and fabricate settled execution metadata.
Debug a Rejected Action
- Read the
agent.action.rejectedor wait rejection trace. - Check the validation
codeand issues. - Inspect the fresh action surface or active
.wait()schema. - Compare the rejected proposal with
session.snapshot(). - Confirm that execution revision and event history did not change.
The rejection's boundaryView already contains the unchanged presentation, application projection, and authorized action surface from one snapshot. Prefer it over three independent session.snapshot() / projection reads.
Two-Phase LLM Tracing & Rejection Recovery
When rejectionRecovery is active (mode: 'explain' or mode: 'repair') and an action proposal is rejected due to schema/validation constraints (WAIT_INPUT_INVALID, ACTION_INPUT_INVALID, WORKFLOW_INPUT_INVALID), Invariant executes a bounded, single-shot recovery call. Observability sinks observe distinct phases:
In mode: 'explain' (Clarification):
text
1. llm.call.started (phase: "initial")
2. llm.call.completed (phase: "initial", proposal: submit_input)
3. agent.action.rejected (code: "WAIT_INPUT_INVALID", issues: [{ path: ["phone"], code: "required" }])
4. llm.call.started (phase: "rejection_recovery", actionCount: 0)
5. llm.call.completed (phase: "rejection_recovery", text: "I still need your phone number.")In mode: 'repair' (Proposal Repair):
text
1. llm.call.started (phase: "initial")
2. llm.call.completed (phase: "initial", proposal: submit_input)
3. agent.action.rejected (code: "WAIT_INPUT_INVALID", issues: [{ path: ["phone"], code: "required" }])
4. llm.call.started (phase: "rejection_recovery", actionCount: 1)
5. llm.call.completed (phase: "rejection_recovery", proposal: submit_input)
6. agent.action.accepted (status: "accepted", revision: 22)- Phase Provenance: Every
llm.call.*event carriesphase: "initial" | "rejection_recovery". - Token Aggregation: The returned
AgentTurnResult.usageaggregates token counts from both calls. - Trace Separation: Individual trace events retain the exact per-call usage and latency.
Debug Durable Progress
ts
const state = await app.storage?.loadState(runId);
const events = await app.storage?.readEventLog(runId);
console.log(state?.revision, state?.status, state?.currentNodeId);
console.table(events?.map(({ seq, type }) => ({ seq, type })));Use events to answer “what became true?” Use traces to answer “why did this handler or model produce that proposal?”
For Agent tool turns, the console card labels the snapshot sent to the model as Reasoned at and the causal action consequence as Settled at. These can have different revisions by design. Co-generated model text is marked withheld because it was authored at the former boundary, before the latter existed.
Sensitive Data
Projection boundaries should remove secrets and unnecessary PII before model calls. Full reasoning capture is opt-in and can record exactly what crossed that boundary, so production sinks require access controls, retention policy, and redaction appropriate to the application.
Default action traces never include argument values. They expose only sorted argument keys, disposition, and validation path/code; rejected values and custom validator messages are excluded. capture.reasoningBoundary: true is intentionally different: it can include projected context, the user message, provider tool declarations, and the normalized raw model response. Enable it only for a sink approved to receive that data.
Next Steps
- Agents & Action Authority — Understand projected context and valid actions.
- State & Event Sourcing — Inspect durable execution truth.
- SDK Trace Reference — Configure sinks and capture.