Skip to content

feat(development-codebase-tools): add scan-secrets skill#461

Open
wkoutre wants to merge 1 commit intonextfrom
feat/add-skill-scan-secrets
Open

feat(development-codebase-tools): add scan-secrets skill#461
wkoutre wants to merge 1 commit intonextfrom
feat/add-skill-scan-secrets

Conversation

@wkoutre
Copy link
Copy Markdown
Contributor

@wkoutre wkoutre commented Apr 14, 2026

What gap this fills

The development-codebase-tools plugin had no credential scanning capability. While security-analyzer-agent performs high-level threat modeling and architectural security analysis, there was no skill for the very common task of scanning source files for hardcoded secrets, API keys, passwords, and tokens.

This is a critical pre-release and pre-OSS-publish gap — hardcoded credentials are one of the most common and damaging security mistakes developers make.

How it was identified

During capability gap analysis, the secrets hygiene dimension was entirely absent. The existing security-analyzer-agent is for full threat modeling sessions — it's not suitable for quick, targeted "do I have any hardcoded API keys?" scans. No other skill in any plugin addresses this.

What it does

The scan-secrets skill:

  1. Enumerates all source files in the target scope, excluding binaries, lock files, and generated output
  2. Pattern matches against a library of secret signatures covering: AWS keys, GitHub PATs, Stripe keys, Slack webhooks, SendGrid, Twilio, JWTs, private keys, npm/Vercel/Linear tokens, DB connection strings, and generic password/token assignments
  3. Filters false positives: placeholder values (TODO, CHANGEME, YOUR_, etc.), test/fixture files (downgraded severity), short/trivial values
  4. Classifies each finding by type (critical / high / medium) and secret category
  5. Optionally scans git history (--history) for secrets committed in past commits that may have been removed but are still exposed
  6. Reports a structured summary grouped by severity with specific remediation guidance per finding

Example usage

"Scan for hardcoded secrets"
"Find hardcoded API keys in src/"
"Are there any credentials in this code?"
"Check for leaked secrets --history"
"Secrets audit --severity critical"
"Scan for exposed tokens before open-sourcing"

Test plan

  • Skill triggers on "scan for hardcoded secrets", "find hardcoded API keys", "check for credentials in the codebase"
  • File enumeration excludes node_modules/, .git/, dist/, lock files
  • Critical patterns match real AWS keys, GitHub PATs, Stripe keys
  • Placeholder values (YOUR_API_KEY, CHANGEME, ${VAR}) are filtered
  • Test/fixture files are downgraded in severity (critical → high, high → medium)
  • --history flag triggers git log scan
  • --scope src/ limits scan to that directory
  • --severity critical only reports critical findings
  • Output always redacts secret values (shows *** instead of real value)
  • Clean result reported when no findings
  • Plugin validates: node scripts/validate-plugin.cjs packages/plugins/development-codebase-tools
  • Markdown lints clean
AI-Generated Description

Summary

  • Adds a new scan-secrets skill to the development-codebase-tools plugin for scanning codebases for hardcoded secrets, API keys, tokens, and credentials
  • The skill fills a gap in credential hygiene — the existing security-analyzer-agent handles high-level threat modeling but not targeted secret detection
  • Bumps plugin version from 2.1.1 → 2.2.0

What it does

The scan-secrets skill:

  1. Enumerates source files in target scope, excluding binaries, lock files, and generated output
  2. Pattern matches against a library of secret signatures covering: AWS keys, GitHub PATs, Stripe keys, Slack webhooks, SendGrid, Twilio, JWTs, private keys, npm/Vercel/Linear tokens, DB connection strings, and generic password/token assignments
  3. Filters false positives: placeholder values (TODO, CHANGEME, YOUR_), test/fixture files (downgraded severity), short/trivial values
  4. Classifies each finding by type (critical / high / medium) and secret category
  5. Optionally scans git history (--history) for secrets committed in past commits
  6. Reports a structured summary grouped by severity with remediation guidance

Changes

File Change
packages/plugins/development-codebase-tools/skills/scan-secrets/SKILL.md New skill with pattern library, false-positive filtering, severity classification, and remediation guidance
packages/plugins/development-codebase-tools/.claude-plugin/plugin.json Add scan-secrets to skills array; bump version 2.1.1 → 2.2.0
packages/plugins/development-codebase-tools/CLAUDE.md Add scan-secrets to skills list and file structure
packages/plugins/development-codebase-tools/README.md Add scan-secrets to skills table and usage examples

Test plan

  • Skill triggers on "scan for hardcoded secrets", "find hardcoded API keys", "check for credentials in the codebase"
  • File enumeration excludes node_modules/, .git/, dist/, lock files
  • Critical patterns match real AWS keys, GitHub PATs, Stripe keys
  • Placeholder values (YOUR_API_KEY, CHANGEME, ${VAR}) are filtered
  • Test/fixture files are downgraded in severity (critical → high, high → medium)
  • --history flag triggers git log scan
  • --scope src/ limits scan to that directory
  • --severity critical only reports critical findings
  • Output always redacts secret values (shows *** instead of real value)
  • Clean result reported when no findings
  • Plugin validates: node scripts/validate-plugin.cjs packages/plugins/development-codebase-tools
  • Markdown lints clean

@wkoutre wkoutre requested a review from a team as a code owner April 14, 2026 11:06
@vercel
Copy link
Copy Markdown

vercel bot commented Apr 14, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
ai-toolkit-slack-oauth-backend Ready Ready Preview, Comment Apr 14, 2026 11:06am

Request Review

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 14, 2026

🤖 Claude Code Review

Review complete

Summary

This PR adds a new scan-secrets skill to the development-codebase-tools plugin, bumping the version from 2.1.1 to 2.2.0. The skill provides heuristic pattern-based scanning for hardcoded secrets, API keys, and credentials with severity classification and remediation guidance.

The documentation updates (CLAUDE.md, README.md, plugin.json) are consistent and complete. The SKILL.md is well-structured with clear steps, a comprehensive pattern library, and thoughtful false-positive filtering.

Feedback

UUID pattern labeled as "Critical" will generate heavy false positives

The "Heroku API key" pattern on SKILL.md line 140:

[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}

This is a standard UUID v4 format. UUIDs are ubiquitous in codebases (database record IDs, correlation IDs, trace IDs, config identifiers, etc.). Labeling every UUID match as Critical will flood results with noise and erode trust in the scanner. Consider either:

  • Removing this pattern entirely (UUIDs are not secrets by default)
  • Restricting it to context where the variable name suggests a credential (e.g., heroku_api_key\s*=\s*<uuid-pattern>)
  • Downgrading to Medium severity at most

Base64 blob pattern is very broad

The pattern [A-Za-z0-9+/]{64,}={0,2} under "High" severity (line 165) will match any base64 content — encoded images, hashes, serialized data, large constants, minified code fragments, etc. Without anchoring to a credential-related variable name or context, this will be a significant false-positive source. Consider requiring a preceding assignment like (key|secret|private|credential)\s*[:=]\s* to narrow matches.

Minor: mermaid-diagram skill missing from README table

The README skills table doesn't include the mermaid-diagram skill, though this predates this PR and isn't something to block on.


💡 Want a fresh review? Add a comment containing @request-claude-review to trigger a new review at any time.

@github-actions
Copy link
Copy Markdown
Contributor

📚 Documentation Check ✅

Verdict: Passed

Plugin version was bumped (2.1.1 → 2.2.0), CLAUDE.md and README.md were updated, and the new SKILL.md was created. All required documentation is present.


PR #461 Documentation Review

This PR adds the scan-secrets skill to the development-codebase-tools plugin. All critical documentation requirements are satisfied:

  • Version bumped: plugin.json version updated from 2.1.12.2.0 (correct minor bump for new feature)
  • CLAUDE.md updated: New skill listed in the Skills section and directory tree
  • README.md updated: New skill added to the skills table and usage examples
  • SKILL.md created: Full skill definition with frontmatter, options, and detailed process steps

Two minor/informational items noted below.

Missing Updates

Type File Severity Reason
📄 readme CLAUDE.md ℹ️ info Root CLAUDE.md plugin version table shows development-codebase-tools at 2.1.1 but plugin.json now specifies 2.2.0

Suggestions (2)

💡 Inline suggestions have been posted as review comments. Click "Commit suggestion" to apply each fix directly.

  • ℹ️ CLAUDE.md: The root CLAUDE.md contains a plugin version table that still shows 2.1.1 for development-codebase-tools. It should be updated to reflect the new 2.2.0 version from this PR.
  • ℹ️ Notion Plugin Marketplace (external): Per project rules, the Notion Plugin Marketplace doc must be updated when plugin inventory changes. The new scan-secrets skill should be added to the development-codebase-tools section and the total Skills count incremented.

🤖 Generated by Claude Documentation Validator | Mode: suggest

Copy link
Copy Markdown
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📋 Review verdict: APPROVE

👆 The main review comment above is the source of truth for this PR review. It is automatically updated on each review cycle, so always refer to it for the most current feedback.

This formal review submission is for the verdict only. 2 inline comment(s) are attached below.

-----BEGIN (RSA |EC |OPENSSH |PRIVATE |PGP PRIVATE )KEY-----

# Heroku API key
[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This UUID pattern ([0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}) will match every UUID in the codebase and is classified as Critical. UUIDs are used extensively for non-secret purposes (database IDs, trace/correlation IDs, config identifiers). This will generate heavy false positives and undermine the scanner's credibility.

Consider either removing it or requiring a credential-related variable name context, e.g.:

(heroku_api_key|HEROKU_API_KEY)\s*[:=]\s*["']?[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}["']?

https?:\/\/[^:@\s]+:[^@\s"']+@

# Private RSA/PEM material inline (base64 blob ≥64 chars)
[A-Za-z0-9+/]{64,}={0,2}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pattern [A-Za-z0-9+/]{64,}={0,2} is extremely broad — it matches any 64+ character base64 string. This will flag encoded images, hashes, serialized data, minified code, and many other non-secret values as High severity.

Consider requiring a credential-related variable name or assignment context to reduce false positives, e.g.:

(private_key|secret|credential|pem)\s*[:=]\s*["'][A-Za-z0-9+/]{64,}={0,2}["']

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: eb3a4a0dc0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +87 to +89
-o -name "*.yml" -o -name "*.yaml" -o -name "*.toml" -o -name "*.ini" \
-o -name "*.conf" -o -name "*.config" -o -name "*.json" \
-o -name "*.sh" -o -name "*.bash" -o -name "Makefile" -o -name "Dockerfile" \) \
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Exclude lockfiles from file enumeration

The skill states that *.lock files should always be skipped, but the find recipe still includes *.json/*.yaml without any filename-level exclusions for package-lock.json, yarn.lock, or pnpm-lock.yaml. In practice this makes scans process large lockfiles and can produce noisy secret matches, which contradicts the documented behavior and the test plan expectation that lock files are excluded.

Useful? React with 👍 / 👎.

-----BEGIN (RSA |EC |OPENSSH |PRIVATE |PGP PRIVATE )KEY-----

# Heroku API key
[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Narrow Heroku token regex to avoid UUID matches

The Heroku pattern is currently a bare UUID regex, so any ordinary UUID literal (IDs in config, fixtures, migrations, etc.) will be classified as a critical secret. Because UUIDs are common and this pattern has no surrounding key/context constraint, the scanner will generate high-confidence false positives and desensitize users to real leaks.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant