led by Ryan Garver · Boise, Idaho Get Smart AI, led by Ryan Garver
Email Ryan Hiring?
Back to work

Design + build · 2026 · public · 68 tests · reproducible

Agent Forensics

Post-mortem tooling that turns a Claude Code transcript into a redacted record and points at the exact turn where a run broke.

Source
50.8% sessions with a breakdown 96 of 189 of my own Claude Code sessions, 36,480 tool calls, Jul to Sep 2026; 95% Wilson CI 43.7 to 57.8
189 sessions analyzed the most frequent signature was TIMEOUT, in 62 of them
2.5% tool-call error rate 896 of 36,480 calls; 95% Wilson CI 2.3 to 2.6 
$ agent-forensics headline
Across 189 of my own Claude Code sessions (2026-07-20 to
2026-09-16; 36480 tool calls), 96 (50.8%, 95% Wilson CI
43.7-57.8%) contain at least one pre-registered breakdown
signature; the most frequent is TIMEOUT (62 sessions);
896 tool calls (2.5%) returned an error.
The record holds only counts, durations, and HMAC hashes; no prompt, tool output, or file path can enter it by construction. The redaction is verifiable in the committed data, not asserted.

I kept running Claude Code sessions that went sideways in ways I could not fully explain afterward: a tool-call loop, a retried command that never recovered, a session that just stopped. This is the tool I built to stop guessing, and to do it without keeping the raw transcripts anywhere.

What it found

Run against 189 of my own Claude Code sessions (July to September 2026, 36,480 tool calls), it found at least one pre-registered breakdown pattern in 96 of them: 50.8% (95% Wilson CI 43.7 to 57.8). The most frequent was a timeout, in 62 sessions. Across every session, 896 of 36,480 tool calls (2.5%) returned an error. Half of my own agent sessions hit a known failure mode, which is worth knowing before trusting one unattended.

Redacted by construction, not by filtering

Every transcript goes through one parser that builds a record which can only ever hold counts, durations, booleans, small labels, and HMAC tokens. There is no code path that copies a prompt, a tool input, a tool output, or a file path into the record. Eleven detectors, frozen and hashed before any transcript is read, look for loops, retries, API errors, refusals, dangling calls, hook errors, and timeouts; seven of them form the headline set. A threshold cannot be loosened after seeing a result without the protocol hash changing and the audit catching it.

How it is checked

68 tests pass and the type-check is clean. An audit sweeps everything under the data directory for redaction violations before it can be committed, and a reproduce command re-derives the findings from the committed records offline, failing on any byte-level mismatch. The redaction claim is therefore checkable in the data itself. The source is public: read it and run it yourself.