Skip to content

fix(api): scope tool policies to agent vocabulary - #114

Merged
moshloop merged 2 commits into
mainfrom
fix/codex-agent-foreign-tool-policy
Sep 8, 2026
Merged

fix(api): scope tool policies to agent vocabulary#114
moshloop merged 2 commits into
mainfrom
fix/codex-agent-foreign-tool-policy

Conversation

@adityathebe

@adityathebe adityathebe commented Sep 4, 2026

Copy link
Copy Markdown
Member

Gavel AI-fix can select codex-agent while carrying a Claude-style permissions.tools map. Captain treated every non-empty map as a Codex policy and rejected the run, even though those tool names do not exist in Codex.

Scope policy checks to the selected agent's declared built-ins. Known foreign names are inapplicable, while selected-runtime and unknown names remain fail-closed; runtime profile validation uses the same rule.

Fixes #110

Summary by CodeRabbit

  • Bug Fixes

    • Allow policies for built-in tools belonging to another runtime are now safely ignored when resolving permissions.
    • Deny and ask policies for foreign tools continue to be rejected.
    • Unknown tool names remain enforceable and fail validation rather than being treated as foreign.
    • Tool-policy validation now handles mixed lists consistently across supported runtimes.
  • Documentation

    • Clarified the behavior and limits of per-tool allow policies for foreign built-in tools.

Gavel AI-fix could select codex-agent while carrying a Claude tool policy. The blanket runtime guard rejected the run even though Codex exposes none of those named tools.

Treat positively known foreign built-ins as inapplicable for support checks and profile validation. Keep selected-runtime and unknown names fail-closed so unsupported Codex policies are never silently dropped.
@coderabbitai

coderabbitai Bot commented Sep 4, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 59cb3f14-c1dd-4885-adb6-092faa09a33c

📥 Commits

Reviewing files that changed from the base of the PR and between 8c7bc2d and 7913459.

📒 Files selected for processing (5)
  • docs/src/pages/agents/approvals.mdx
  • pkg/api/permissions.go
  • pkg/api/runtime_profiles.go
  • pkg/api/runtime_profiles_ginkgo_test.go
  • pkg/api/tool_policy_support_test.go

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.


Walkthrough

The change allows known foreign built-in tools in allow policies to remain inert for a selected runtime. It continues to reject foreign deny and ask policies, unknown tools, and tools for runtimes without vocabularies. Tests and documentation cover these rules.

Changes

Foreign tool policy handling

Layer / File(s) Summary
Policy support validation
pkg/api/permissions.go, pkg/api/tool_policy_support_test.go, docs/src/pages/agents/approvals.mdx
RequireToolPolicySupport identifies known foreign built-ins and ignores them only for allow policies. Tests and documentation cover supported, foreign, unknown, deny, and ask policies.
Resolved runtime policy handling
pkg/api/runtime_profiles.go, pkg/api/runtime_profiles_ginkgo_test.go
Runtime profile resolution skips foreign built-in allow policies and continues to reject other unavailable policies.

Suggested reviewers: moshloop

Merge Risk: ⚪ Minimal · up to 79134

Policies can now include inert allow entries for another agent’s built-in tools without blocking the selected runtime, while restrictive, unknown, and supported-tool policies retain validation. No concrete current-head merge-blocking risk remains.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the primary change: scoping tool policies to the selected agent's vocabulary.
Linked Issues check ✅ Passed The changes address issue #110 by allowing foreign built-in allow policies without weakening enforcement for owned, unknown, deny, or ask policies. Tests cover the required behavior.
Out of Scope Changes check ✅ Passed The code, tests, and documentation changes directly support tool-policy scoping and the requirements in issue #110. No unrelated changes are evident.
Docstring Coverage ✅ Passed Docstring coverage is 85.71% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 7 functions across 4 files. (1 skipped: 1 u…
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/codex-agent-foreign-tool-policy
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch fix/codex-agent-foreign-tool-policy

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown

Gavel summary

Source Pass Fail Skip Duration

Totals: 0 passed · 0 failed · 0 skipped · -

View full results

@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown

Gavel summary

Source Pass Fail Skip Duration
approval 4 1 11 2.2s
agent 5 0 0 284.61µs
agentcreds 11 0 0 815.322µs
ai 90 0 0 92ms
aichat 151 0 2 42.8s
anthropicmock 1 0 0 3ms
api 292 0 0 162ms
attachments 5 0 0 24ms
bash 4 0 0 185.914µs
callertools 10 0 0 110ms
captain 31 0 0 28.8s
claude 22 0 0 6ms
claudeagent 13 0 0 286ms
cli 334 0 0 10.1s
cmux 4 0 0 246.855µs
collections 12 0 0 1ms
credentials 5 0 0 36ms
credsync 11 0 0 410ms
database 22 0 0 23.0s
deploy 110 0 0 780ms
genkit 25 0 0 193ms
gitagent 123 0 0 37.1s
github.com/flanksource/captain/migrations 11 0 0 2.9s
github.com/flanksource/captain/pkg/ai 245 0 0 630ms
github.com/flanksource/captain/pkg/ai/agent 26 0 0 450ms
github.com/flanksource/captain/pkg/ai/agent/commit 56 0 0 4.5s
github.com/flanksource/captain/pkg/ai/agent/setup 16 0 0 60ms
github.com/flanksource/captain/pkg/ai/agent/verify 32 0 0 400ms
github.com/flanksource/captain/pkg/ai/agent/worktree 6 0 0 -
github.com/flanksource/captain/pkg/ai/assistanttags 15 0 0 -
github.com/flanksource/captain/pkg/ai/fixture 43 0 0 160ms
github.com/flanksource/captain/pkg/ai/fixture/kubeproxy 2 0 0 30ms
github.com/flanksource/captain/pkg/ai/fixture/mcpproxy 6 0 0 -
github.com/flanksource/captain/pkg/ai/history 54 0 0 -
github.com/flanksource/captain/pkg/ai/internal/gen-model-registry 12 0 0 -
github.com/flanksource/captain/pkg/ai/middleware 25 0 0 -
github.com/flanksource/captain/pkg/ai/observation 2 0 0 -
github.com/flanksource/captain/pkg/ai/pricing 9 0 0 10ms
github.com/flanksource/captain/pkg/ai/prompt 24 0 0 10ms
github.com/flanksource/captain/pkg/ai/provider 177 0 1 -
github.com/flanksource/captain/pkg/ai/provider/claudeagent 54 0 0 1.4s
github.com/flanksource/captain/pkg/ai/provider/cmux 128 0 0 870ms
github.com/flanksource/captain/pkg/ai/provider/genkit 39 0 0 -
github.com/flanksource/captain/pkg/ai/provider/jsonrpc 6 0 0 50ms
github.com/flanksource/captain/pkg/ai/provider/openai 8 0 0 20ms
github.com/flanksource/captain/pkg/aichat 9 0 0 100ms
github.com/flanksource/captain/pkg/aiflags 9 0 0 -
github.com/flanksource/captain/pkg/aimock 50 0 7 210ms
github.com/flanksource/captain/pkg/aimock/anthropicmock 14 0 0 30ms
github.com/flanksource/captain/pkg/aimock/openaimock 18 0 0 70ms
github.com/flanksource/captain/pkg/api 141 0 0 390ms
github.com/flanksource/captain/pkg/api/registry 127 0 0 -
github.com/flanksource/captain/pkg/bash 348 0 0 10ms
github.com/flanksource/captain/pkg/captainconfig 29 0 0 -
github.com/flanksource/captain/pkg/captaintoken 23 0 0 1.3s
github.com/flanksource/captain/pkg/claude 138 0 0 -
github.com/flanksource/captain/pkg/claude/tools 18 0 0 -
github.com/flanksource/captain/pkg/cli 818 0 1 10m0s
github.com/flanksource/captain/pkg/cmux 1 0 0 -
github.com/flanksource/captain/pkg/codexconfig 10 0 0 -
github.com/flanksource/captain/pkg/container 72 0 1 30ms
github.com/flanksource/captain/pkg/database 152 0 0 44.7s
github.com/flanksource/captain/pkg/gitagent 31 0 0 480ms
github.com/flanksource/captain/pkg/gitagent/proxy 12 0 0 10ms
github.com/flanksource/captain/pkg/monitor 61 0 0 4.0s
github.com/flanksource/captain/pkg/sandbox 13 0 0 -
github.com/flanksource/captain/pkg/sandbox/adapter 39 0 0 20ms
github.com/flanksource/captain/pkg/sandbox/presets 13 0 0 -
github.com/flanksource/captain/pkg/session 70 0 0 30ms
history 72 0 0 180ms
jsonl 6 0 0 60ms
migrations 17 0 0 24.8s
openaimock 2 0 0 149.17µs
promptrun 40 0 0 84ms
provider 15 0 0 13ms
registry 52 0 0 2ms
runtimeprofiles 27 0 0 7.8s
session 33 0 0 33ms
tools 58 0 0 7ms
verify 28 0 0 516ms

Totals: 4847 passed · 1 failed · 23 skipped · 14m2s

Failing tests

approval — Approval broker > cancels the durable row when the calling context ends

Expected
    <database.TurnRequestState>: pending
to equal
    <database.TurnRequestState>: cancelled

View full results

The previous commit treated every tool name from another agent's vocabulary as inapplicable, so a Claude-style `Bash: deny` or `Bash: ask` on a codex run passed RequireToolPolicySupport and codex ran with its shell fully available — the silent widening the guard exists to refuse. Captain's own history normaliser maps codex shell to Bash, so those names describe the same capability, not a tool codex lacks. The rule also made enforcement depend on how complete the hand-kept vocabulary table is: adding a Claude tool name to it would have loosened every codex run.

Narrow the exemption to allow entries. An allow only pre-approves a tool the agent already has, so on an agent without that tool it constrains nothing and can be skipped safely; this is enough for the portable allowlist in #110. A deny or ask on any name, foreign or not, is refused exactly as before, and ask is once again refused unconditionally. validateResolvedPermissions applies the same policy-aware rule.

Restore the deny/ask tests to their original form, since that behaviour is unchanged, and add tests that pin the new behaviour directly: a foreign allowlist is accepted on codex-agent, codex-cli and gemini-cli; a mixed map still fails on the runtime's own tools; a foreign deny or ask still fails; a name no agent declares still fails; the API modes treat no name as foreign; and profile resolution skips a foreign allow but refuses a foreign deny. Document the exemption in the approvals page.
@moshloop moshloop closed this Sep 6, 2026
@moshloop moshloop reopened this Sep 7, 2026
@moshloop
moshloop marked this pull request as ready for review September 7, 2026 06:47
@adityathebe
adityathebe requested a review from moshloop September 7, 2026 06:59
@moshloop
moshloop merged commit c11998f into main Sep 8, 2026
15 of 16 checks passed
@moshloop
moshloop deleted the fix/codex-agent-foreign-tool-policy branch September 8, 2026 06:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

codex-agent rejects policies for Claude-only tool names

2 participants