Skip to content

[Mate] Materialize mate.invocation into the installed skills #2445

Description

@wachterjohannes

mate init records how the coding agent must invoke Mate as mate.invocation, and #2380
materializes that command into mate/AGENT_INSTRUCTIONS.md and the managed AGENTS.md block.
The installed skills do not get it: 11 shell snippets across 4 of the 6 shipped SKILL.md files
hardcode vendor/bin/mate.

In a containerized project this is the one place the pinning does not reach. An agent following a
skill runs the host binary. If the host PHP version differs, the guard catches it and names the
right command, so the agent recovers after one failed call. If it happens to match, the call
succeeds and reports on a runtime that is not the application under test, which is exactly what
mate.invocation exists to prevent.

Proposed approach, mirroring what SkillInstaller::rewriteSkillName() already does for the
frontmatter name:

  • put a placeholder in the shipped skill sources, and substitute it in the installed copy after
    copyDirectory(), so a reader of the package source can see that substitution happens
  • substitute the bare vendor/bin/mate literal as well, so skills from third-party extensions
    that do not know the placeholder are not silently left wrong
  • reuse the callback form of SkillFrontmatter::rewriteName(); an invocation such as
    docker compose exec app php vendor/bin/mate must stay literal rather than be read as a
    backreference

One thing to watch: SkillInstaller::isUpToDate() compares source_hash and the hash of the
installed folder. Changing mate.invocation changes neither, so a plain reinstall would keep the
old prefix without saying so. The invocation needs to become part of the recorded state and of
that freshness check, otherwise this fix introduces the kind of silent staleness it is meant to
remove.

Alternative: keep the skills path-neutral instead of substituting

Raised by @chr-hertel in the review of #2380: rather than materializing the invocation, write the
skills as mate tools:call ... or plainly tools:call ... and let the agent resolve the prefix
from the managed AGENTS.md block, which already states it.

Worth weighing properly, because it is cheaper and avoids the freshness problem above entirely:

  • no installer change, no extra state, and changing mate.invocation cannot leave a skill stale
  • the installed skill stays byte-identical to the package source, which keeps skills:validate
    honest and gives third-party skill authors nothing new to know
  • it reads as "the Mate command" rather than as a path, which is what a skill is describing

Against it:

  • the command as written is not runnable. tools:call alone is not a program, and mate is only
    on PATH if the project put it there, so every invocation depends on the agent applying an
    indirection that lives in a different file
  • that indirection is exactly the kind of instruction Fix package name for demo #81 found models handle unreliably: the
    reworded block went from 2/10 to 10/10 invocations precisely by not relying on the agent to
    carry a rule across a boundary
  • it was a model complaining about the mismatch that surfaced this in the first place, so at
    least one agent did notice and did not silently do the right thing

The underlying question to settle first: is an installed SKILL.md a document the agent copies
commands out of, or a description it adapts? Substitution assumes the former, path-neutral assumes
the latter. Whichever we pick should then also apply to mate/AGENT_INSTRUCTIONS.md, which today
takes the substituted form, so the two do not end up answering that question differently.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    MateIssues & PRs about the AI Mate component

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions