Usage is very high

[ableinc]

Ensure that we are only prompting the AI with the minimal data it needs. Upon auditing i'm finding a single session taking upwards of 1 millions tokens collectively between tokens up and tokens down. This is way too much and its causing claude rate limits.

[ableinc]

Plan

Cut per-run token usage: trim the harness prompt, stop the CLI loading ambient operator context, cap per-run spend, and report cache reads separately from fresh input

Context

Issue #18 reports single sessions totalling ~1M tokens "up and down", causing Claude rate limits, and asks that we prompt the model with only the minimal data it needs.

Auditing the code turns up two separate things, and both need addressing:

1. The 1M number is mostly cache reads. claude.Result.TokensIn() (internal/claude/runner.go:110) sums input_tokens + cache_creation_input_tokens + cache_read_input_tokens into one figure, which is stored as runs.tokens_in and rendered by the web console as plain "Tokens in" (internal/web/assets/app.js:649,685) and by Discord (internal/discord/notifier.go:269). On an agentic run the whole conversation prefix is re-read from cache on every turn, so a 40-turn run with a 25K prefix reports ~1M input tokens while only tens of thousands were ever processed fresh. Cache reads are billed at a fraction of fresh input and are not what drives a rate limit. The number is real but it is not the number the operator thinks it is, and today there is no way to see the split.
2. There is genuine, fixable bloat, in two places: what the harness puts into the prompt, and what the Claude Code CLI loads into every turn's system prompt because we never tell it not to. Neither is 1M tokens on its own, but both are paid on every turn of every run.

The intended outcome: the same work, with a materially smaller context re-sent per turn, a hard per-run spend ceiling the daemon can enforce, and a dashboard that distinguishes cache traffic from fresh input so "usage is very high" can be judged from data rather than a conflated total.

Approach

Part A — Report cache reads separately from fresh input

Nothing here changes what is sent to Claude; it changes what the operator sees, and it is the part that actually answers "is 1M tokens a problem?".

• internal/claude/runner.go: keep TokensIn() as the total (so existing runs.tokens_in rows keep their meaning) and add three accessors alongside it, mirroring the existing TokensOut() style: FreshTokensIn() (Usage.InputTokens), CacheWriteTokens() (CacheCreationInputTokens), CacheReadTokens() (CacheReadInputTokens). The Usage struct already parses all four fields.
• internal/store/store.go: append a third entry to the migrations slice — the file already establishes this pattern at line 285 (ALTER TABLE runs ADD COLUMN kind ...) and the migration loop bumps PRAGMA user_version per entry, so existing databases upgrade in place:

  ALTER TABLE runs ADD COLUMN tokens_cache_read  INTEGER NOT NULL DEFAULT 0;
  ALTER TABLE runs ADD COLUMN tokens_cache_write INTEGER NOT NULL DEFAULT 0;

  Add the two fields to store.Run, to the runColumns const, and to scanRun.
• RecordUsage already takes seven positional arguments; adding two more makes it unreadable. Replace the tail arguments with a small store.RunUsage{ModelID, SessionID, CostUSD, TokensIn, TokensOut, CacheRead, CacheWrite, Turns} value and update the four call sites (internal/orchestrator/loop.go:660, :740, internal/orchestrator/prcomments.go:385, :435). Keep the existing "empty model/session does not overwrite" CASE WHEN semantics — the pre-record call at loop.go:660 depends on it.
• Surface the split:
  • internal/web/assets/app.js — runs table cell (line 649) and detail grid (lines 685-686): render 1.0M in (18K new · 24K written · 950K cached) / 42K out, reusing the existing fmtTokens helper (line 132). No CSS or server-side change is needed: /runs already marshals the whole store.Run.
  • internal/discord/notifier.go:269 — same breakdown in the Tokens embed field.
  • internal/orchestrator/loop.go:794 — extend the claude_done event detail (and the equivalent in prcomments.go:479) with fresh_in=… cached_in=… out=… so the per-run audit trail carries it.

Part B — Trim what the harness puts in the prompt

All in internal/orchestrator/prompt.go. The current worst case for issueContext is ~36K chars (12K body + 12 × 2K comments) and implementTaskPrompt appends the approved plan untruncated on top of that; prCommentTaskPrompt is unbounded in the number of comments it renders.

1. Stop feeding the harness's own comments back to the model. issueContext (line 190) renders every comment, including the plan comment, the PR announcement, and every failure comment the daemon itself wrote. Filter with the existing isAgentComment helper (phase.go:37) before the maxCommentsInclu window is applied. This is the single largest and safest win: on a re-plan or implement run the plan is currently sent twice (truncated inside the discussion, in full under "Approved plan"), and failure comments are pure noise. The re-plan path's wording ("see the newest comment in the discussion above") still holds — reviewer feedback is a human comment and survives the filter. Optionally also drop bare implement approvals via isApproval (phase.go:51); they carry no information the "Approved plan" header does not already state.
2. Tighten the constants (lines 11-15), which are the only knobs the truncation uses: maxBodyChars 12000 → 6000, maxCommentChars 2000 → 1200, maxCommentsInclu 12 → 6. Every one already routes through the existing truncate helper, which appends truncationSuffix — and extractPlan (phase.go:134) keys off that suffix to refuse a truncated plan, so the plan cap below must stay well clear of any real plan length.
3. Cap the plan. Add maxPlanChars = 20000 and apply truncate to the plan in implementTaskPrompt (line 265) and to previousPlan in planTaskPrompt (line 244). 20K chars is roughly 5K tokens and comfortably above a normal plan, so TestImplementTaskPromptCarriesTheApprovedPlanVerbatim keeps passing; the cap only bites on a runaway plan.
4. Bound prCommentTaskPrompt (line 140), which today has no limit on comment count: add maxPRCommentsInclu = 10 (keep the newest, state how many were dropped, mirroring the issueContext "showing the last N of M" line), give diff hunks their own smaller maxDiffHunkChars = 800, and cap review summaries at 3. pendingMentions (prcomments.go:106) can legitimately return dozens of inline comments after a big review pass.

Note that the system prompts themselves (systemPrompt, planSystemPrompt, prCommentSystemPrompt) are ~1.5K chars each and are not worth touching — they are the cheapest part of the prompt and each line in them is load-bearing for a safety rule.

Part C — Stop the CLI loading the operator's ambient context

This is the lever with the largest per-turn effect and it is currently entirely unmanaged. The daemon shells out to claude with the operator's own environment (internal/claude/runner.go:198-222), so every run inherits their user-level MCP servers, plugins, skills, and settings. MCP tool schemas and the skills listing sit in the system prompt on every turn — this is exactly the "prompting the AI with data it does not need" the issue describes, and it is invisible in the harness's own code.

In internal/claude/runner.go, extend Options and the args construction (after the existing --permission-mode block, before opts.ExtraArgs so an operator can still override):

Flag	Why	Default
--strict-mcp-config	With no --mcp-config, this loads zero MCP servers. The harness's runs need none — they get the built-in file/bash tools.	on
--disable-slash-commands	Drops the skills listing from the system prompt; the harness drives the model from stdin and never types a slash command.	on
--exclude-dynamic-system-prompt-sections	Moves per-machine sections (cwd, env, git status) out of the system prompt, improving prompt-cache reuse across runs. Applies here because we use --append-system-prompt, not --system-prompt.	on
--autocompact <tokens>	Caps how large the conversation grows before it is compacted. This is what bounds the per-turn cache-read cost that produced the 1M figure. Accepts 100k–1M.	200000
--setting-sources <list>	Controls whether user/project/local settings load at all.	leave unset — see Risks

Expose these as first-class config.ClaudeConfig fields (StrictMCPConfig, DisableSlashCommands, ExcludeDynamicSystemPrompt, AutocompactTokens, SettingSources) rather than asking operators to know the flags, with the defaults above set in config.Default() (internal/config/config.go:244). config.Migrate is Default()-driven and generic, so existing config.json files pick the new fields up through -migrate-config with no change to internal/config/migrate.go.

Validate AutocompactTokens in Config.Validate(): zero means "don't pass the flag", otherwise it must be within 100000–1000000, matching what the CLI accepts.

Part D — A hard per-run spend ceiling

The README states as a principle that there is "no cost budget", and the gate only reacts after Claude reports a limit. The CLI offers --max-budget-usd <amount> (print mode only), which stops a run at a dollar figure instead of letting a pathological run consume an hour of usage before the run.timeout fires.

• Add Options.MaxBudgetUSD float64 to internal/claude/runner.go and pass --max-budget-usd when it is > 0.
• Add run.max_budget_usd to config.RunConfig and set it from execute (loop.go:700) and executePRComments (prcomments.go:403).
• Default: 0 (off), so behaviour is unchanged unless the operator opts in. config.example.json should ship a non-zero suggestion in the documented reference so the knob is discoverable.
• A budget-terminated run surfaces through the existing failure path: diagnose (runner.go:293) already renders subtype, terminal_reason and the CLI's own message into the error, so the run is recorded as failed with an explanatory reason and backs off normally.

There is no --max-turns in the installed CLI (2.1.x) — do not reach for it.

Files touched

• internal/claude/runner.go — new usage accessors; new Options fields; new args.
• internal/orchestrator/prompt.go — comment filtering, tightened constants, plan and PR-comment caps.
• internal/orchestrator/loop.go, internal/orchestrator/prcomments.go — RecordUsage call sites, new claude Options, richer claude_done events.
• internal/store/store.go — third migration entry, two Run fields, runColumns, scanRun, RecordUsage signature.
• internal/config/config.go — new ClaudeConfig and RunConfig fields, defaults, validation.
• internal/web/assets/app.js, internal/discord/notifier.go — token breakdown display.
• config.example.json (also the embedded default via embedded.go); models.json untouched.
• README.md — "How Claude Code is invoked" (line 319, the claude invocation block is verbatim and must be updated), the "Prompts" section's description of what goes into the prompt, the configuration reference, and the "Usage-limit-aware, not schedule-aware" principle at line 86, which no longer holds literally once run.max_budget_usd exists.
• Tests: internal/orchestrator/prompt_test.go (add cases for agent-comment filtering and the PR-comment cap), internal/store/store_test.go (a migration test in the shape of TestMigrationAddsKindAndPRCommentTasks), internal/claude/runner_test.go (assert the new flags appear, in the style of the existing arg assertions).

Verification

1. go build ./... && go test ./... — the store migration test and the prompt tests are the ones that should visibly change.
2. Prompt size, measured rather than assumed: add a temporary test (or a -run one-off) that builds implementTaskPrompt from a fixture issue with a long body, 20 comments (half of them agent comments) and a long plan, and prints len() before and after. Expect roughly a 2-3× reduction.
3. ./coding-agent-loop -dry-run -once — exercises discovery, phase decision and model selection without spending anything; confirms the config additions load and validate.
4. A real single run against a scratch issue: ./coding-agent-loop -no-mutate -once (runs Claude for real, pushes nothing). Then inspect the transcript:

   jq -c 'select(.type=="system") | {tools: (.tools|length), mcp: (.mcp_servers//[]|length)}' ~/.agent-loop/logs/<run>.jsonl | head -1
   jq  'select(.type=="result") | .usage' ~/.agent-loop/logs/<run>.jsonl

   The first line confirms --strict-mcp-config took effect (no MCP servers); the second gives the four-way usage split to compare against a pre-change run on the same issue.
5. Open /ui, check the run row and detail page show the in (new · written · cached) / out breakdown, and confirm a Discord notification (if enabled) carries the same.

Risks and decisions for the reviewer

• --setting-sources is deliberately left unset. Restricting it would cut the most ambient context, but user-level settings can carry auth helpers, permission rules and env that a bypassPermissions daemon run may depend on. Turning it off could break runs on the operator's machine in a way that is hard to diagnose. It is exposed as config, defaulted to unchanged behaviour. Decide whether to default it to project instead — that is the aggressive choice and would need a manual run to confirm auth still works.
• --disable-slash-commands also disables skills. If a target repository ships .claude/skills that the agent is expected to use, this would silently remove them. The harness has never invoked a skill explicitly, so this is judged safe, but it is a behaviour change worth a conscious yes.
• --autocompact 200000 changes behaviour on long runs: the model's earlier context gets summarized rather than carried verbatim. This is the trade the issue is asking for (fewer tokens per turn) but it can cost quality on a long implement run. 200K is a conservative starting point — the CLI's own default is higher.
• Redefining tokens_in was considered and rejected. Making tokens_in mean "fresh input only" would give a more honest headline number, but silently changes the meaning of every existing row. The plan keeps the total and adds the breakdown instead. Say so if you would rather have the cleaner semantics and accept the discontinuity.
• Retry behaviour is deliberately untouched. retryDelay (loop.go:384) caps back-off at retry_backoff_max (24h default) and never gives up, so a permanently-broken issue costs a full plan-or-implement run every day forever. Capping that would contradict the documented "the trigger label is the only thing that decides whether it is worked" principle, so it is flagged rather than changed. If reducing usage matters more than that principle, the cheap version is: after N consecutive failures, drop the trigger label and say so in the failure comment.
• The plan/implement split inherently pays for exploration twice — the implement run cannot see the plan run's session (--no-session-persistence, and a human gate sits between them). Nothing in this plan changes that; a shared-session design would be a much larger change and would weaken the "the plan a human approved is the one on the issue" guarantee.

---

Reply with exactly implement to approve this plan and start the change. Reply with anything else and the plan will be revised to address it.

_{coding-agent-loop run ac782780-2909-4afc-a4ef-a8d3f1dcc03b, model claude-opus-5, cost $2.8418}