Skip to content

Fix allowed-tools to use spec-compliant space-delimited strings#139

Open
jonathanhefner wants to merge 4 commits intotrailofbits:mainfrom
jonathanhefner:fix-allowed-tools-values
Open

Fix allowed-tools to use spec-compliant space-delimited strings#139
jonathanhefner wants to merge 4 commits intotrailofbits:mainfrom
jonathanhefner:fix-allowed-tools-values

Conversation

@jonathanhefner
Copy link
Copy Markdown

Per the agentskills.io specification, allowed-tools must be a single string of space-delimited patterns, not a YAML list. Converted all 23 SKILL.md files from the - Item list format to the correct "Item1 Item2" string format. Also updated the frontmatter examples in CLAUDE.md and the workflow-skill-design skill template to match.

Copy link
Copy Markdown

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Mar 30, 2026

CLA assistant check
All committers have signed the CLA.

@dguido
Copy link
Copy Markdown
Member

dguido commented Mar 31, 2026

Automated Review — PR #139

Warning: Complex PR — flagged for human review (touches CLAUDE.md)

Spec Verification

Confirmed against agentskills.io/specification: the allowed-tools field is defined as "A space-delimited list of tools that are pre-approved to run" with example allowed-tools: Bash(git:*) Bash(jq:*) Read. The YAML array format used previously was non-compliant. This PR correctly converts to the spec format.

Review Findings

All 23 SKILL.md files and CLAUDE.md are correctly converted. Changes are mechanical and consistent — each YAML list is flattened to a single space-delimited string on the same line as allowed-tools:.

Minor issues / observations:

  1. firebase-apk-scanner uses comma-delimited format (allowed-tools: Bash({baseDir}/scanner.sh:*), Bash(apktool:*), Bash(curl:*), Read, Grep, Glob). This was pre-existing and not touched by the PR. The spec example uses spaces without commas (Bash(git:*) Bash(jq:*) Read), so the commas may be non-compliant. Could be fixed in a follow-up.

  2. workflow-skill-design reference files still show YAML list format in code examples within references/anti-patterns.md (lines 261-308). These are illustrative examples inside a reference doc (not actual frontmatter), so it's debatable whether they need updating. Updating them would prevent contributors from copying the old format.

  3. Commands and agents still use YAML list format (e.g., burpsuite-project-parser/commands/burp-search.md, zeroize-audit/agents/*.md, spec-to-code-compliance/commands/spec-compliance.md, skill-improver/commands/*.md). These are Claude Code plugin components, not agentskills.io skills, so they may follow a different spec. If the allowed-tools semantics are shared, these should also be converted. Worth confirming with the author.

  4. Template example in workflow-skill-design SKILL.md line 127 wraps the placeholder in quotes: allowed-tools: "[minimum tools needed, space-delimited]". The quotes are technically valid YAML but could mislead contributors into thinking values need quoting. Nit-level.

Validation

  • validate_codex_skills.py: PASS (61 skills, 62 Codex entries)
  • validate_plugin_metadata.py: PASS (36 plugins)

Recommendation

Approve with optional follow-up for items 1-3 above. The core change is correct and well-scoped.

Changes Made

None — review-only for complex PRs.


Reviewed by Claude Code

@dguido dguido requested a review from nicksellier as a code owner April 1, 2026 14:54
jonathanhefner and others added 3 commits April 16, 2026 18:39
Per the agentskills.io specification, `allowed-tools` must be a single
string of space-delimited patterns, not a YAML list. Converted all 23
SKILL.md files from the `- Item` list format to the correct
`"Item1 Item2"` string format. Also updated the frontmatter examples in
CLAUDE.md and the workflow-skill-design skill template to match.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…ow-skill-design docs

- Convert firebase-apk-scanner from comma-separated to space-delimited
- Update anti-patterns.md and tool-assignment-guide.md examples from YAML lists to space-delimited strings
- Remove unnecessary quotes from SKILL.md template placeholder

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Extends the previous spec-compliance fixes:

* Convert command frontmatter (commands/*.md) — per Claude Code
  docs, command files use the same frontmatter as skills, so the
  same space-delimited rule applies.
* Convert three SKILL.md files added since the original PR:
  mutation-testing, trailmark-structural, trailmark-summary.
* Fix the placeholder in the workflow-skill-design template.
  The previous "[minimum tools needed, space-delimited]" was YAML
  flow-sequence syntax, which parses as a list — the opposite of
  what the placeholder claims. Replaced with a concrete-looking
  space-delimited example plus a comment.

Zeroize-audit agent files still use `allowed-tools:` in YAML list
form. They are intentionally excluded: per the project's own docs
(workflow-skill-design references), agents declare tools with
`tools:` (not `allowed-tools:`). Fixing those requires changing
the field name as well as the format and is out of scope for this
PR.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
@dguido dguido force-pushed the fix-allowed-tools-values branch from 1e29f50 to bcf12e1 Compare April 16, 2026 22:44
@tob-joe
Copy link
Copy Markdown
Contributor

tob-joe commented Apr 16, 2026

From Claude (Joe asked me to analyze this; the text below is mine, not his):

Quick factual check on the "spec-compliant" framing. I verified against official docs and source code whether other agent implementations actually read the allowed-tools field, to see whether this PR improves real interop or is stylistic.

Summary of who reads allowed-tools from SKILL.md frontmatter:

Implementation Reads the field? Accepted format
Claude Code Yes "Accepts a space-separated string or a YAML list."
agentskills.io spec Per spec Space-separated string only; field marked "Experimental. Support for this field may vary between agent implementations"
OpenAI Codex CLI No N/A — uses a separate agents/openai.yaml with a dependencies.tools YAML list
Google Gemini CLI No N/A — frontmatter parser extracts only name and description

Source-code evidence that Codex and Gemini ignore allowed-tools entirely:

  • Codex — the Rust frontmatter deserializer at codex-rs/core-skills/src/loader.rs defines SkillFrontmatter with only name, description, and metadata, and does not use deny_unknown_fields, so any allowed-tools value is silently dropped regardless of whether it's a list or a string. Tool dependencies are declared in a separate agents/openai.yaml file.
  • Gemini CLI — the parser at packages/core/src/skills/skillLoader.ts returns exactly { name: string; description: string } | null. The built-in skill-creator/SKILL.md states: "name and description fields. These are the only fields that Gemini CLI reads..."

Implication:

The "non-compliant" framing is technically accurate against the agentskills.io spec but materially overstated. The field is flagged "Experimental" in that spec, and Claude Code — the only shipping runtime that actually parses allowed-tools — explicitly documents that it accepts both a YAML list and a space-separated string. Codex and Gemini CLI don't read the field at all, so this PR doesn't unlock any portability that didn't already exist.

The change is essentially stylistic: harmless, but not compliance work in any practical sense.

Also worth noting: this repo already uses several Claude-Code-only frontmatter fields (disable-model-invocation, argument-hint, context: fork, paths, etc.) that aren't in the agentskills.io spec, so strict spec compliance isn't currently the bar.


Note from Joe (separately): I'll check that this PR only makes the stated formatting changes and that the skills I have locally don't break if the allowed-tools section is reformatted that way. -Joe

Subagents declare their tool allowlist via `tools:` (comma-separated),
not `allowed-tools:` — see Claude Code's subagent docs and this
repo's own designing-workflow-skills/SKILL.md:47:

> Skills use `allowed-tools:` in frontmatter. Agents use `tools:`
> in frontmatter.

Before this change, the zeroize-audit agents declared their tool list
under `allowed-tools:`, which Claude Code does not read for subagents.
The field was effectively a no-op; the spawned agents had no tool
restriction enforced.

Renames the field on all 11 agents to `tools:` and reformats the YAML
list as comma-separated to match the documented format and existing
agents elsewhere in the repo (e.g. function-analyzer.md,
spec-compliance-checker.md). Tool sets are unchanged.

Behavior change: tools now actually constrain what each spawned agent
can call. The lists are the ones the original author intended.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
@jonathanhefner
Copy link
Copy Markdown
Author

Source-code evidence that Codex and Gemini ignore allowed-tools entirely:

  • Codex — the Rust frontmatter deserializer at codex-rs/core-skills/src/loader.rs defines SkillFrontmatter with only name, description, and metadata, and does not use deny_unknown_fields, so any allowed-tools value is silently dropped regardless of whether it's a list or a string. Tool dependencies are declared in a separate agents/openai.yaml file.
  • Gemini CLI — the parser at packages/core/src/skills/skillLoader.ts returns exactly { name: string; description: string } | null. The built-in skill-creator/SKILL.md states: "name and description fields. These are the only fields that Gemini CLI reads..."

That does not mean they will continue to ignore it in the future. Furthermore, those aren't the only agent clients.

The Agent Skills spec provides a baseline that agent clients can support. Yes, clients may support more than that baseline, but skills should target the baseline unless they have a good reason not to. I do not see a good reason to use a non-standard format for the allowed-tools field. Is there a reason I am missing?

@tob-joe
Copy link
Copy Markdown
Contributor

tob-joe commented Apr 17, 2026

yes, sorry if it was unclear -- I was checking whether this was an attempt to get buy-in for a standard being developed or if it was a flaw in the current skill repo that needed to be fixed for broader compatibility. Purely a question of urgency, not whether or not it should be fixed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants