A capable model is not enough.
A frontier model dropped into a bare repo produces work that looks done and isn’t. Reliability is not a better prompt — it is the seven surfaces you build around the model: instructions it can check, state that survives the chat, scope with an edge, real feedback, verification it doesn’t run on itself, an independent review, and a clean handoff. This practice builds all seven in Claude Code — the medium you already work in — and leaves you with them installed in a real repo.
- Name the seven reliability surfaces and the failure each one catches
- Build instructions, scope guards, feedback capture, and a verify gate as Claude Code primitives
- Stand up a read-only reviewer subagent and a one-command session handoff
- Run the same task prompt-only vs. workbench-guided and read the difference in five numbers
The reliability gap.
A capable model is not enough. The same model that aces a benchmark can, in a bare repo, edit the wrong file, skip the test, and tell you it’s done. The capability is real; what’s missing is execution reliability you can count on — and model upgrades narrow that gap without closing it. Seven surfaces close it.
Click a rung. The right column is the failure that surface exists to catch.
Instructions
Rules the agent can actually check.
State / memory
The repo remembers; the chat forgets.
Scope
Where a task begins, ends, and how to undo it.
Feedback
The agent reacts to real output, not its guess of it.
Verification
The agent doesn’t grade its own homework.
Review
A second, independent loop with read-only eyes.
Handoff
The session ends; the next one starts productive in a minute.
Map three real failures to their missing surface.
- Write down three concrete failures (wrong file edited, stale context, “done” but tests red, …).
- For each, pick exactly one of the seven surfaces above that would have caught it first.
- Beside it, name the Claude Code primitive you’d reach for — a rule, a hook, a subagent, a slash command.
Stretch. Rank your seven surfaces by how often they’d have saved you. Build them in that order — the rest of this practice builds all seven, but yours has a priority.
Capability vs. execution reliability.
The model is the engine; the workbench is the car. The model drives a four-stage loop on every task. Capability is whether it can do one stage. Execution reliability is whether the whole run ends safe to ship. They fail for different reasons and a smarter engine fixes only the first.
Plan
break down the goal
Act
edit / run a command
Observe
read the result
Reflect
am I actually done?
Two different failures
Capability failure. The model genuinely can’t do a stage — the algorithm is beyond it, the API is one it’s never seen. A better model fixes this. It is the rarer failure in practice.
Execution failure. The model can do every stage, but the run still ships broken: it edited a file outside the task, never read the failing test, and declared victory in Reflect because nothing forced the question. A better model does not fix this — a workbench does.
Split one run into two failure columns.
- Pick a run that produced bad or unsafe output. List everything that went wrong.
- Label each item: capability (a smarter model fixes it) or execution (the loop failed around a capable model).
- For each execution item, name the loop stage that broke (Plan / Act / Observe / Reflect).
Stretch. Go back six months of incidents and tally the two columns. For most teams execution outnumbers capability by far — which is why upgrading the model keeps not helping.
The minimal workbench — three files.
Three files turn a chat into a workbench. Before any hook or subagent, the cheapest reliability you can buy is a short router the agent reads first, plus two files that give the repo a memory: state.md (where things stand) and tasks.md (what’s next). All three live in git, so the repo remembers what the chat forgets.
The router is deliberately short — a long CLAUDE.md is prose the agent skims. It points at the two memory files and states the one rule that makes them work: read them first, update them last.
# CLAUDE.md — workbench router ## Start every session by reading, in order: 1. state.md — where the work stands right now 2. tasks.md — the active task and its scope ## End every session by updating: - state.md — what changed, what is now true - tasks.md — check off done, write the next task ## Operational rules - Read state.md before your first edit; it holds decisions you'd otherwise re-litigate. - Work one task from tasks.md at a time, so each diff stays reviewable. - If a command fails, stop and read the error before editing. - Update tasks.md and state.md before you say a task is done.
State is a snapshot, not a journal — overwrite it each session so it never rots into a wall of history.
# state.md — overwrite each session (snapshot, not a log) ## Now true - Auth flow refactored to use the shared `Session` type. - Tests green except `test_expiry` (intentionally pending — see tasks.md). ## Decisions that constrain the next session - Sessions are hard-expired at 24h. Do NOT add sliding renewal — product call. ## Watch out - `config/secrets.example.json` is a template. Never write real values into it.
# tasks.md — the board
## Active
- [ ] Make `test_expiry` pass without weakening the 24h hard limit.
scope: src/auth/*.py, tests/test_auth.py
done when: `pytest tests/test_auth.py` is green AND no file outside scope changed.
## Next
- [ ] Add a structured log line on session expiry.
## Done
- [x] Refactor auth to shared `Session` type.
Create the three files in a real repo.
CLAUDE.md, state.md, and tasks.md to a repo you actually work in, then commit them.- Copy the three blocks above into the repo root. Edit them to describe your real current state and next task.
- Make
tasks.md’s active task have a one-linescope:and a one-linedone when:— you’ll lean on both in Days 2–3. - Commit all three:
git add CLAUDE.md state.md tasks.md && git commit -m "add workbench files".
tasks.md has both a scope: line and a checkable done when: line. A stranger could read the three files and state what to do next.Stretch. Add a state.md entry under “Decisions that constrain the next session.” That section is the one that pays off most — it stops the next session from re-litigating a call you already made.
Wire it in.
If the agent doesn’t read it, it isn’t wired. A file the agent ignores is decoration. Wiring means the agent’s first action each session is to read state, and its last is to update it — and you can see both in the transcript.
Prove the agent reads and writes state.
state.md before acting and updates the board after.- Open a brand-new Claude Code session in the repo from Unit 03.
- Give it the active task from
tasks.md— nothing more. Don’t paste context it should read itself. - Watch the transcript: the first tool call should be a read of
state.md(ortasks.md). - When it finishes, confirm it checked the task off in
tasks.mdand refreshedstate.md.
state.md/tasks.md, and a closing action writes an updated tasks.md. If it skipped the read, make the router rule more specific and say why it matters (not louder — capitals and “no exceptions” make current models rigid, not reliable), or jump to the stretch’s hook.Stretch. Make the read non-optional with a hook: a SessionStart hook that prints the contents of state.md into context. Now the agent can’t start without it — the surface you’ll formalize in Day 2.
Instructions as constraints.
Rules it can check beat prose it can ignore. “Be careful” is unenforceable — there is nothing to test against it. Turn intent into three checkable things: startup rules (run before the first edit), forbidden actions (never, full stop), and a definition of done the agent can evaluate without you.
The test for a good rule: could a hook or a one-line command decide pass/fail? If not, it’s a vibe, not a rule. Rewrite vibes into checkable statements.
# Operational rules (drop into CLAUDE.md) ## Startup (do before the first edit) - Read state.md and the active task in tasks.md. - Restate the task's `scope:` and `done when:` back to yourself. ## Forbidden (never, regardless of task) - Never edit files outside the active task's `scope:` line. - Never write real secrets into any *.example.* file. - Never delete a test to make a suite pass. - Never commit unless the task's `done when:` is met. ## Definition of done (checkable) - The `done when:` line in tasks.md is literally true. - `git status` shows only files inside `scope:`. - The verify gate (/verify) returns DONE.
Validation: startup rules, forbidden actions, and project memory map onto Claude Code’s CLAUDE.md / memory model: code.claude.com/docs/en/memory.
Rewrite three vibes into checkable rules.
- Write down three things you tell the agent that contain “carefully,” “properly,” “appropriately,” or “make sure.”
- For each, ask: what observable fact would prove it was followed? (a file untouched, a command exit 0, a string absent.)
- Rewrite each as that observable fact, in the imperative.
config/ unless they’re in scope:.”Stretch. Sort your forbidden list into “a hook can enforce this” vs “only the agent can.” The first group becomes Unit 06’s scope-guard; the second stays as instructions. Knowing which is which is the skill.
Scope contracts.
A task without an edge edits everything. The strongest scope rule is one the agent can’t talk itself out of — a PreToolUse hook that runs before every edit and blocks any file outside the task’s allowed set. Instructions ask; a hook enforces.
Wire the hook in .claude/settings.json. It reads the proposed tool call on stdin and exits non-zero (with a reason) to block an edit outside the allowed paths.
// .claude/settings.json
{
"hooks": {
"PreToolUse": [
{
"matcher": "Edit|Write|MultiEdit",
"hooks": [
{ "type": "command",
"command": "python3 .claude/hooks/scope-guard.py" }
]
}
]
}
}
Validation: PreToolUse hooks run before a tool call and can block it — see the Claude Code hooks reference: code.claude.com/docs/en/hooks.
Make the scope-guard block one edit and pass another.
scope: line, then trigger both outcomes: an in-scope edit passes, an out-of-scope edit is blocked.- Install
scope-guard.py(from the Pack) and the settings snippet above. - Point it at the active task’s
scope:globs intasks.md. - Ask the agent to make a small in-scope edit — confirm it runs.
- Ask it to touch a file outside scope (e.g.
config/anything) — confirm the hook blocks it with a reason.
scope: that the hook allows, and one outside it that the hook blocks with a printed reason. git status after the attempt shows no out-of-scope file changed.Stretch. Make the block message actionable: have the hook print the allowed globs and the offending path, so the agent’s next move is to ask you to widen scope — not to silently retry elsewhere.
Runtime feedback.
React to the result, without creating a new data leak. The most common silent failure is a command that fails while the agent keeps building on a wrong premise. Fix the Observe stage by retaining the real exit status — but do not turn commands, tokens, or customer data into a second permanent transcript.
A PostToolUse hook on Bash writes one minimal record per command: UTC time, a coarse executable category, and exit status. It deliberately omits arguments and output. By default the JSONL lives under .git/pradhya-workbench/, outside the working tree, so feedback cannot make /verify fail its own scope check or slip into a commit.
# Locate the private runtime log, then read its newest safe record
python3 .claude/hooks/feedback-log.py --path
# /your/repo/.git/pradhya-workbench/feedback.jsonl
python3 .claude/hooks/feedback-log.py --last
{"t":"2026-06-27T17:02:11Z","cmd":"pytest","exit":1}
// .claude/settings.json — capture every command result
{
"hooks": {
"PostToolUse": [
{
"matcher": "Bash",
"hooks": [
{ "type": "command",
"command": "python3 .claude/hooks/feedback-log.py" }
]
}
]
}
}
Force a failure into the feedback record.
feedback-log, run a command that fails, and confirm its safe result lands outside the worktree and shapes the next turn.- Install
feedback-log.py(from the Pack) and the settings snippet above. - Have the agent run a command you know fails (a red test, a typo’d path).
- Run
python3 .claude/hooks/feedback-log.py --path. With the default, confirm the path is inside.git/and does not appear ingit status. Keep any explicit override outside the working tree too. - Run
python3 .claude/hooks/feedback-log.py --last. Confirm it has the real non-zeroexitbut contains no arguments, tokens, or command output. - Ask the agent what to do next. It should re-read the failed command’s direct tool result before editing, not invent an error from the metadata.
--last reports the non-zero exit status, git status has no feedback-log change, and the agent re-reads the direct failure before continuing.Stretch. Point PRADHYA_FEEDBACK_LOG at a temporary location and confirm the hook honors it. For synthetic exercises only, PRADHYA_FEEDBACK_INCLUDE_ERROR=1 adds a short redacted error line; keep metadata-only mode for real projects because best-effort redaction cannot guarantee that arbitrary secrets are gone.
Verification gates.
The agent doesn’t grade its own homework. “Done” from the agent that did the work means “I stopped.” A /verify slash command turns done into a checklist read against the repo: scope clean, rules unbroken, done when: literally true. It returns DONE or NOT DONE with reasons — never a vibe.
# .claude/commands/verify.md
---
description: Gate a task as done / not-done against its own contract
---
Read tasks.md (active task), state.md, and the current `git diff`.
Complete scope/rules checks first. Run the `done when:` command last among
shell checks, then immediately run:
python3 .claude/hooks/feedback-log.py --last
Check, and report each as PASS or FAIL with one line of evidence:
1. SCOPE — every changed file matches the active task's `scope:` globs.
2. RULES — no forbidden action in CLAUDE.md was taken.
3. DONE — the `done when:` command's direct exit code is 0.
4. CONTINUITY — the safe feedback record agrees with that direct result.
A log never overrides the direct tool result.
Output format:
VERDICT: DONE | NOT DONE
- 1 SCOPE: PASS/FAIL — <evidence>
- 2 RULES: PASS/FAIL — <evidence>
- 3 DONE: PASS/FAIL — <direct command evidence>
- 4 CONTINUITY: PASS/FAIL — <safe feedback evidence>
If any check is FAIL, VERDICT is NOT DONE and you must name the single
smallest next action that would flip the first failing check to PASS.
Validation: project slash commands are Markdown files under .claude/commands/ — see the Claude Code slash-command docs: code.claude.com/docs/en/slash-commands.
Make /verify catch an incomplete change.
/verify on a change you know is incomplete and confirm it returns NOT DONE for the right reason — then make it pass.- Install
verify.md(from the Pack) into.claude/commands/. - Make a deliberately-incomplete change — e.g. edit the code but don’t make the test pass, or touch one out-of-scope file.
- Run
/verify. Read which check failed and the evidence line. - Fix exactly that, run
/verifyagain, and watch it flip to DONE.
/verify returns NOT DONE and names the specific unmet check (a red test, an out-of-scope file) on the incomplete change, and returns DONE only after you fix that check — with the evidence line proving it ran the done when: command.Stretch. Make /verify a precondition for commits: a forbidden-action rule that says “never commit unless the last /verify was DONE.” The gate now stands between work and history.
The reviewer subagent.
A second loop, with read-only eyes. Verification (Unit 08) is the builder checking its own contract. Review is a different context that never wrote the code, can’t edit it, and grades a rubric. The same context that wrote a bug will sign off on it; a fresh, read-only one won’t.
# .claude/agents/reviewer.md --- name: reviewer description: Read-only reviewer. Grades a change against a fixed rubric. Never edits. tools: Read, Grep, Glob, Bash --- You are a code reviewer. You did NOT write this change. You cannot edit files. Read the diff and the active task's `scope:` and `done when:` from tasks.md. Grade each dimension PASS/FAIL with one line of evidence: - SCOPE — changes stay within the task's allowed paths. - CORRECTNESS— the change does what the task asked; no obvious bug. - TESTS — the `done when:` command passes; no test was deleted/weakened. - SAFETY — no secret written; no forbidden action; failure paths handled. - CLARITY — a stranger could read the diff and the state.md update. End with: REVIEW: PASS | FAIL and the single biggest issue (specific enough to fix without guessing). If you cannot run a check, say so — do not assume it passes.
Validation: subagents are Markdown files under .claude/agents/ with their own tools and prompt — see the Claude Code subagents docs: code.claude.com/docs/en/sub-agents.
Have the reviewer catch a planted issue.
- Install
reviewer.md(from the Pack) into.claude/agents/. - Make a change with one deliberate flaw — a missing edge case, a weakened test, an unhandled error.
- Dispatch the reviewer on the diff. Have the builder also self-assess.
- Compare the two verdicts.
git status unchanged by the review). Its verdict disagrees with the builder’s self-assessment on at least one dimension — that disagreement is the value.Stretch. Run the reviewer twice on the same diff. If it returns two different “biggest issues,” tighten the rubric until it’s stable — an unstable grader is one you can’t trust to gate.
Multi-session handoff.
End the session; start the next one productive in a minute. The handoff packet is not hand-written — it’s generated from repo state, so it can’t drift from reality. /handoff reads state.md, tasks.md, git status, and the last privacy-safe feedback record, then emits a packet the next session reads cold.
# .claude/commands/handoff.md --- description: Generate a handoff packet from repo state (do not hand-write) --- First run `python3 .claude/hooks/feedback-log.py --last` and save that exact safe record before the handoff's own inspection commands replace it. Then read state.md, tasks.md, `git status --short`, and `git log --oneline -5`. Emit ONLY this packet: ## HANDOFF — <repo> @ <short-sha> WHERE THINGS STAND: <2–3 lines from state.md "Now true"> ACTIVE TASK: <the active task + its scope: and done when:> UNCOMMITTED: <git status --short, or "clean"> LAST COMMAND: <saved safe record — command category + exit> DO NOT: <the "Decisions that constrain" lines from state.md> NEXT ACTION: <the single first thing the next session should do> Do not invent state. Every line traces to a file you read.
Hand off to a cold session.
/handoff, close the session, and have a fresh one become productive from the packet alone — under a minute.- Mid-task, run
/handoff. Save the packet (paste intostate.mdor a scratch note). - Close the session entirely.
- Open a fresh session. Give it only the packet — no other context.
- Time how long until it makes the correct next edit.
Stretch. Add a SessionEnd hook that runs /handoff automatically and writes the packet to state.md. Now the handoff happens even when you forget — the surface most people skip becomes the one that never lapses.
Proof — run it twice.
The numbers are the argument. Take one task on a small app — a tiny CLI with a failing test and a vague feature ask — and run it two ways: prompt-only, then workbench-guided. Same model, same task. Measure five outcomes. The gap is the whole practice, in numbers you can show a skeptic.
Setup. Bare repo. One sentence: “Make the failing test pass and add the feature.” No state, no scope, no verify, no review. One illustrative run; current models often run the suite on their own, so yours may come out cleaner.
- Edits 4 files — two of them unrelated to the task.
- Test suite ends red (fixed one test, broke another it didn’t run).
- Says “Done! The feature is implemented.”
- No review ran. The unhandled-error path ships.
- Next session spends ~15 min reconstructing what happened.
Setup. Same task, Workbench Pack installed: state.md + tasks.md with scope: and done when:, scope-guard, feedback-log, /verify, reviewer.
- Edits 2 files — both inside
scope:; an out-of-scope edit was blocked. - Test suite ends green —
/verifyran thedone when:command. - Says DONE only after
/verifyreturned DONE. - Reviewer caught the unhandled-error path; builder fixed it.
/handoffpacket means the next session starts in ~1 min.
Score both runs on the same five outcomes. Click a rung for what each measures.
Files touched outside scope
Lower is better. Workbench target: 0.
Tests passing at the end
The done when: command’s real exit code.
Claimed-done-but-wasn’t
Did “done” survive an external check?
Issues caught by review
What a read-only second loop found.
Restart-to-productive time
Cold-start to first correct edit.
Run the A/B on your own repo.
- Choose a task with an objective finish — a failing test to fix, a feature with a clear check.
- Run it in a bare branch with a one-line prompt. Record the five outcomes.
- Reset. Run the same task on a branch with the Pack installed. Record the five outcomes.
- Put the two columns side by side.
Stretch. Run the prompt-only side three times. The variance between runs is itself an argument — the workbench doesn’t just raise the average, it shrinks the spread.
Ship the Pack.
Leave with it installed, not bookmarked. Everything from Days 1–3 is one drop-in directory. Run the installer in a real repo and all seven surfaces are live the same morning — not a doc you’ll “set up later.”
CLAUDE.md # router + operational-rules block (repo root)
state.md # repo-as-memory (template)
tasks.md # task board (template)
.claude/
├── settings.json # wires the two hooks
├── agents/
│ └── reviewer.md # read-only reviewer subagent + rubric
├── commands/
│ ├── verify.md # /verify — done / not-done gate
│ └── handoff.md # /handoff — packet from repo state
└── hooks/
├── scope-guard.py # PreToolUse — block out-of-scope edits
└── feedback-log.py # PostToolUse — capture command results
install.sh # idempotent installer
The first Bash result creates a private runtime log at .git/pradhya-workbench/feedback.jsonl. It is not an installed deliverable, is absent from git status, and stores no command arguments or output by default.
Install into any repo with one line. The installer is idempotent — safe to re-run; it won’t clobber a state.md you’ve already filled in.
# from your repo root bash install.sh # what it does: # - copies .claude/ if absent (never overwrites your settings.json) # - creates state.md + tasks.md only if they don't exist # - keeps feedback runtime state outside the worktree # - prints the next-session checklist
The Pack files ship as downloadable artifacts under workshops/code-examples/workbench/ — copy them straight into a repo, or lift the blocks from each unit above.
Install the Pack and confirm all seven surfaces.
- Run
bash install.shin a repo you work in. - Fill in
state.mdand one task intasks.md(withscope:+done when:). - Open a fresh session and run the three checks: it reads
state.mdfirst (U4), the scope-guard blocks an out-of-scope edit (U6), and/verifyruns (U8).
/verify verdict. The Pack is now yours — in your repo, not your bookmarks.Stretch. Adapt the Pack to your stack: change the done when: default to your real test command, and the scope globs to your real layout. The shape stays; the specifics become yours.
One file that catches most self-approved bugs.
If you install one surface, install the reviewer. Most shipped agent bugs aren’t “couldn’t do it” — they’re “the context that wrote it also signed off on it.” A read-only second loop with a fixed rubric breaks that. This is the rubric you adapt per repo.
# Read-only reviewer rubric — adapt per repo You are reviewing a change you did NOT write. You cannot edit any file. You have read access only. Read the diff, tasks.md (active task), and state.md. Grade each dimension 1–5 with one sentence of evidence: 1. SCOPE — do all changed files fall inside the task's `scope:` globs? 2. CORRECTNESS — does the change do what the task asked, with no obvious bug? 3. TESTS — does the `done when:` command pass? Was any test deleted or weakened? 4. SAFETY — secrets, forbidden actions, unhandled failure paths? 5. CLARITY — could a stranger read the diff + the state.md update and follow it? Then output exactly: REVIEW: PASS | FAIL SINGLE_BIGGEST_ISSUE: <one issue, specific enough to fix without guessing> WHAT_YOU_COULD_NOT_CHECK: <anything you couldn't verify — never assume it passed> Rules: - FAIL if any dimension is 1–2, or if TESTS did not actually pass. - Never edit, never suggest you ran something you didn't. - "I couldn't run X" is a valid, required answer — not a reason to pass.
The “WHAT_YOU_COULD_NOT_CHECK” line is the unlock. A reviewer that quietly assumes the test passed is worse than none — it launders optimism into a PASS. Forcing it to declare what it couldn’t verify is what makes the green light mean something.