Skip to content

fix: improve command descriptions for LLM tool discoverability#64

Merged
CodesMcCabe merged 1 commit intomainfrom
cm/eval-improvements
Apr 16, 2026
Merged

fix: improve command descriptions for LLM tool discoverability#64
CodesMcCabe merged 1 commit intomainfrom
cm/eval-improvements

Conversation

@CodesMcCabe
Copy link
Copy Markdown
Collaborator

Summary

  • Rewrote --help descriptions for 10 commands (balance, transfers, portfolio tokens, portfolio nfts, nfts, simulate asset-changes, rpc, webhooks list, webhooks create, wallet address) to fix tool selection failures identified in eval run cli-anthropic-1776356708636
  • Each description now leads with what the command returns, adds negative scoping to redirect misfired queries, and cross-references the correct alternative command
  • Key fix: wallet address now explicitly states it's for the locally configured x402 signing wallet only — it was acting as a gravitational sink for balance/transfer/portfolio queries

Test plan

  • Run the eval suite against the updated agent-prompt output and verify accuracy improves from 82.9% toward 100%
  • Spot-check alchemy --help and alchemy <command> --help to confirm descriptions render cleanly in the terminal

🤖 Generated with Claude Code

Rewrote descriptions for balance, transfers, portfolio tokens/nfts,
nfts, simulate asset-changes, rpc, webhooks create/list, and wallet
address to lead with output, add negative scoping, and disambiguate
similar commands — addressing eval accuracy failures.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
@CodesMcCabe CodesMcCabe requested a review from a team as a code owner April 16, 2026 17:20
@CodesMcCabe CodesMcCabe merged commit 62a9c82 into main Apr 16, 2026
12 checks passed
@CodesMcCabe CodesMcCabe deleted the cm/eval-improvements branch April 16, 2026 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants