Skip to content

feat(auth): add API key helper command support#2918

Open
t3hk0d3 wants to merge 11 commits into
tailcallhq:mainfrom
t3hk0d3:feat/api-key-helper
Open

feat(auth): add API key helper command support#2918
t3hk0d3 wants to merge 11 commits into
tailcallhq:mainfrom
t3hk0d3:feat/api-key-helper

Conversation

@t3hk0d3
Copy link
Copy Markdown

@t3hk0d3 t3hk0d3 commented Apr 9, 2026

Problem

In many enterprise and security-conscious environments, API keys are not static secrets — they are ephemeral tokens generated by a secrets manager (e.g. HashiCorp Vault, AWS Secrets Manager), rotated on a schedule, or single-use. Currently, Forge only supports static API keys entered manually or read from environment variables. Users in these environments have to manually refresh keys whenever they expire, which breaks their workflow.

Solution

Add a helper command mechanism that allows users to specify a shell command whose stdout is used as the API key. The command is re-executed automatically when the key expires (or on every request if no TTL is specified).

Three ways to configure

1. Environment variable convention (zero config):

# Just append _HELPER to your provider's API key variable
export ANTHROPIC_API_KEY_HELPER="vault read -field=token secret/ai/anthropic"

2. Provider config (forge.toml):

[[providers]]
id = "my_provider"
url = "https://api.example.com/v1/chat"
api_key_helper = "vault read -field=token secret/ai/key"
response_type = "OpenAI"

3. Interactive UI — during provider login, users now see:

Authentication type:
> Static API Key
  Helper Command (script that generates a key)

Helper command output format

Simple (refresh every request):

sk-ephemeral-key-abc123

With TTL (cache for 1 hour):

sk-ephemeral-key-abc123
---
TTL: 3600

With absolute expiry:

sk-ephemeral-key-abc123
---
Expires: 1750000000

Architecture

Domain layer — why ApiKeyProvider wraps AuthDetails::ApiKey

The simpler approach would have been adding optional generator and expires_at fields directly on AuthCredential. However, this creates technical debt: those fields would only be meaningful for the ApiKey variant, not for OAuth, GoogleAdc, or OAuthWithApiKey. Optional fields that apply to one variant but sit on the parent struct are a code smell — it's exactly the pattern enums are supposed to eliminate. Future developers would wonder why generator exists on OAuth credentials.

Instead, AuthDetails::ApiKey now wraps an ApiKeyProvider enum that owns all key-source concerns — the same way OAuth owns OAuthTokens and OAuthConfig. This keeps the data model honest: StaticKey is a bare key, HelperCommand carries the command + runtime state. The needs_refresh() logic lives on ApiKeyProvider itself, not as a special case in AuthCredential.

The tradeoff is ~35 mechanical match-site updates (AuthDetails::ApiKey(key)AuthDetails::ApiKey(provider) then provider.api_key()), but factory methods (AuthDetails::static_api_key() / AuthDetails::api_key_from_helper()) keep construction sites clean.

ApiKeyProvider enum (new, in credentials.rs):

pub enum ApiKeyProvider {
    StaticKey(ApiKey),           // Existing behavior
    HelperCommand {
        command: String,         // Shell command to execute
        last_key: ApiKey,        // Runtime: last obtained key (not persisted)
        expires_at: Option<DateTime<Utc>>,  // Runtime: TTL/expiry (not persisted)
    },
}
  • AuthDetails::ApiKey now wraps ApiKeyProvider instead of bare ApiKey
  • Factory methods: AuthDetails::static_api_key() and AuthDetails::api_key_from_helper()
  • #[serde(untagged)] ensures backward compatibility — old "sk-123" format still deserializes as StaticKey
  • Only the command field is persisted to credentials.json; last_key and expires_at are #[serde(skip)] and re-obtained at runtime by executing the command

Infra layer

api_key_helper module (new, in forge_infra/src/auth/):

  • execute(&ApiKeyProvider) — async, runs sh -c <command> via tokio::process::Command
  • Configurable timeout via FORGE_API_KEY_HELPER_TIMEOUT env var (default 30s)
  • kill_on_drop(true) ensures child process is cleaned up
  • CRLF normalization for cross-platform compatibility
  • parse_output() handles key-only, TTL, and Expires formats

Refresh flow

  1. On startup: create_provider() loads credential from file → detects HelperCommand with empty last_key → calls api_key_helper::execute() → populates key
  2. Before requests: refresh_provider_credential() checks needs_refresh() → if expired or no TTL, re-executes the command
  3. Migration: migrate_env_to_file() detects {API_KEY_VAR}_HELPER env vars and creates HelperCommand credentials

Credential persistence

[
  {
    "id": "xai",
    "auth_details": {
      "api_key": {
        "command": "vault read -field=token secret/ai/xai"
      }
    }
  }
]

Only the command is stored. The key is always obtained fresh by executing the command on load.

Files changed (25 files, +799 -95)

Layer File Change
Domain credentials.rs ApiKeyProvider enum, needs_refresh() delegation, serde untagged
Domain auth_context.rs helper_command: Option<String> on ApiKeyResponse, api_key_with_helper() factory
Domain new_types.rs Default derive on ApiKey
Domain provider.rs, node.rs Match site updates for ApiKeyProvider
Config config.rs api_key_helper: Option<String> on ProviderEntry
Infra api_key_helper.rs New — async execute + parse + timeout
Infra strategy.rs ApiKeyStrategy::refresh() and complete() handle HelperCommand
Repo provider_repo.rs create_credential_from_env() detects helper command (config or env var), refresh on credential load
Services provider_auth.rs ApiKey added to refresh match arm
UI ui.rs Auth type selection ("Static API Key" / "Helper Command") in handle_api_key_input()
Various 12 files Mechanical: AuthDetails::ApiKey(key)ApiKey(provider) + provider.api_key()

Test plan

  • ApiKeyProvider::StaticKeyapi_key() returns key, serde round-trip, serializes as bare string
  • ApiKeyProvider::HelperCommandapi_key() returns last_key, serializes only command, deserializes with empty last_key
  • needs_refresh() — helper without TTL → true, with future TTL → false, with past TTL → true, static → false
  • parse_output() — key only, key + TTL, key + Expires, CRLF, empty output, unknown metadata
  • execute() — static no-op, helper returns key, failing command returns error
  • ApiKeyStrategy::refresh() — HelperCommand re-executes, StaticKey unchanged
  • Backward compat — legacy {"api_key": "sk-123"} JSON deserializes as StaticKey
  • Config round-trip — api_key_helper serializes/deserializes in TOML
  • Full cargo insta test --accept — all existing tests pass
  • Manual smoke test — provider login → "Helper Command" → command validated → provider configured → chat works

Closes #2888

🤖 Generated with Claude Code

@github-actions github-actions Bot added the type: feature Brand new functionality, features, pages, workflows, endpoints, etc. label Apr 9, 2026
@t3hk0d3 t3hk0d3 force-pushed the feat/api-key-helper branch from 7091bca to 589066a Compare April 9, 2026 23:23
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Apr 9, 2026

CLA assistant check
All committers have signed the CLA.

@t3hk0d3 t3hk0d3 force-pushed the feat/api-key-helper branch 6 times, most recently from 0519d10 to 26419d7 Compare April 10, 2026 08:31
@t3hk0d3
Copy link
Copy Markdown
Author

t3hk0d3 commented Apr 11, 2026

@amitksingh1490 Any chance to check this PR pls?

This is really a blocker for me

@github-actions
Copy link
Copy Markdown

Action required: PR inactive for 5 days.
Status update or closure in 10 days.

@github-actions github-actions Bot added the state: inactive No current action needed/possible; issue fixed, out of scope, or superseded. label Apr 17, 2026
@t3hk0d3
Copy link
Copy Markdown
Author

t3hk0d3 commented Apr 17, 2026

Action required: PR inactive for 5 days. Status update or closure in 10 days.

Thats stupid.

@github-actions github-actions Bot removed the state: inactive No current action needed/possible; issue fixed, out of scope, or superseded. label Apr 24, 2026
@amitksingh1490
Copy link
Copy Markdown
Contributor

@t3hk0d3 Apologies for delay can you rebase this once. I will review it asap

@t3hk0d3 t3hk0d3 force-pushed the feat/api-key-helper branch 2 times, most recently from 20a4dd1 to 735abca Compare April 24, 2026 12:53
@t3hk0d3
Copy link
Copy Markdown
Author

t3hk0d3 commented Apr 24, 2026

@t3hk0d3 Apologies for delay can you rebase this once. I will review it asap

No worries. I've rebased PR.

Allow users to specify a shell command that generates API keys
dynamically, for environments where keys are ephemeral, one-use, or
rotated periodically.

- Add `ApiKeyProvider` enum (`StaticKey` / `HelperCommand`) to model
  both static and command-based API key sources
- Helper commands are configured via env var (`{API_KEY_VAR}_HELPER`
  convention), `api_key_helper_var` in provider config, or interactively
  through the provider login UI
- Commands are executed asynchronously with configurable timeout
  (`FORGE_API_KEY_HELPER_TIMEOUT`, default 30s) and `kill_on_drop`
- Output format supports optional TTL: `<key>\n---\nTTL: <seconds>` or
  `Expires: <unix_timestamp>`
- Only the command is persisted to credentials file; the key is always
  obtained fresh by executing the command on load
- Backward-compatible serde: old `"sk-123"` format still deserializes
  correctly via `#[serde(untagged)]`

Co-Authored-By: Claude Code <noreply@anthropic.com>
@t3hk0d3 t3hk0d3 force-pushed the feat/api-key-helper branch from 735abca to 03cc279 Compare April 24, 2026 13:01
Copy link
Copy Markdown
Contributor

@amitksingh1490 amitksingh1490 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comprehensive Code Review by ForgeCode Agent

Overall Assessment: GOOD PR with recommended fixes

This PR adds API key helper command support — a well-structured feature with thorough testing and correct backward compatibility. The untagged serde enum design is elegant.

Summary of Findings

Priority Count Key Items
Must-fix 1 Command strings leaked in error messages/logs (security)
Should-fix 2 No confirmation for config-file commands; undocumented startup re-execution
Nice-to-have 4 ApiKey::Default concern, TTL overflow, unnecessary async for static keys, process tree kill

See inline comments for details on each finding.

Comment thread crates/forge_infra/src/auth/api_key_helper.rs Outdated
timeout,
tokio::process::Command::new("sh")
.arg("-c")
.arg(command)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SECURITY — MEDIUM] No command sanitization or validation

The command string is passed directly to sh -c with zero validation. Commands come from:

  1. forge.toml config (checked-in to repos)
  2. Environment variables
  3. Interactive user input

A malicious forge.toml in a cloned repository could execute arbitrary code when a user runs forge provider login.

Recommendation: Consider adding a confirmation prompt when the command originates from a config file rather than direct user input. At minimum, log a warning when executing helper commands.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@amitksingh1490 Shall we add trusted flag (stored in credentials.json) to repository-provided configurations? This would be a bigger change, slightly outside of scope for this PR.

Warnings on every helper execution can be really annoying IMO.

Copy link
Copy Markdown
Author

@t3hk0d3 t3hk0d3 Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another approach would be to have trusted flag that is stored as part of helper configuration. If helper is provided by UI or ENV variable - we set it to trusted. Otherwise we ask user prompt on first execution, and set it to trusted for further calls.

But seems like reaching out TUI (text UI) layer from auth provider repository could be a bit challenging.
@amitksingh1490 what would you suggest here?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May be we should ask user to once for permission to run and persist it. And add a config to allow always? I am also thinking best way to solve this.

Copy link
Copy Markdown
Author

@t3hk0d3 t3hk0d3 May 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@amitksingh1490
I need an architectural advise. Problem is that api helper executor is deep inside forge-infra library. Text UI is running on forge-main level. We can't just call UI prompt from inside forge-infra :(

There are several options:

  • We pass a "prompter" trait/callback all the thru callstack.
    We define trait on forge-infra/forge-core level and implement it on forge-main level.
    This could cause a huge blast radius of changes, but can be useful in the future.
  • Same as above but using static mut global variable. This should reduce blast radius, but is a bit smelly.
  • We use a makeshift stdin prompter. Hacky workaround.
  • Something else?

Comment thread crates/forge_infra/src/auth/api_key_helper.rs Outdated
Comment thread crates/forge_infra/src/auth/api_key_helper.rs Outdated
Comment thread crates/forge_infra/src/auth/api_key_helper.rs Outdated
Comment thread crates/forge_infra/src/auth/strategy.rs
Comment thread crates/forge_domain/src/auth/credentials.rs
Comment thread crates/forge_domain/src/auth/new_types.rs
Copy link
Copy Markdown
Contributor

@amitksingh1490 amitksingh1490 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have marked the bot comments which were invalid as resolved. Remaining needs to be fixed/discussed

t3hk0d3 and others added 5 commits April 24, 2026 16:26
Avoid re-running the auth helper on every startup by persisting
last_key and expires_at to credentials.json. The helper only
re-executes when the cached key has expired or on first use.

This is important for zsh mode where the binary is re-invoked per
command — eliminating unnecessary helper execution on startup.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Truncate the command string to 40 characters in error messages to
avoid leaking sensitive vault paths or secret names through logs
and UI error output. Also fix potential u64 to i64 overflow in TTL
parsing by using i64::try_from with a meaningful error.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
kill_on_drop only kills the direct child (sh), leaving grandchildren
orphaned when the helper command uses pipes or subshells. Place the
child in its own process group via process_group(0) and send SIGKILL
to the entire group on timeout.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Match on HelperCommand variant before calling execute() in refresh(),
avoiding unnecessary async overhead for StaticKey credentials.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Prevents FORGE_API_KEY_HELPER_TIMEOUT from being set to an
unreasonably large value that would effectively disable the timeout.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown

Action required: PR inactive for 5 days.
Status update or closure in 10 days.

@github-actions github-actions Bot added the state: inactive No current action needed/possible; issue fixed, out of scope, or superseded. label Apr 30, 2026
@t3hk0d3
Copy link
Copy Markdown
Author

t3hk0d3 commented Apr 30, 2026

Action required: PR inactive for 5 days. Status update or closure in 10 days.

not stale

@github-actions github-actions Bot removed the state: inactive No current action needed/possible; issue fixed, out of scope, or superseded. label Apr 30, 2026
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 7, 2026

Action required: PR inactive for 5 days.
Status update or closure in 10 days.

@github-actions github-actions Bot added the state: inactive No current action needed/possible; issue fixed, out of scope, or superseded. label May 7, 2026
@t3hk0d3
Copy link
Copy Markdown
Author

t3hk0d3 commented May 17, 2026

Action required: PR inactive for 5 days.
Status update or closure in 10 days.

Not stale. Waiting for maintainer's response

@github-actions github-actions Bot removed the state: inactive No current action needed/possible; issue fixed, out of scope, or superseded. label May 17, 2026
@github-actions
Copy link
Copy Markdown

Action required: PR inactive for 5 days.
Status update or closure in 10 days.

@github-actions github-actions Bot added the state: inactive No current action needed/possible; issue fixed, out of scope, or superseded. label May 22, 2026
@t3hk0d3
Copy link
Copy Markdown
Author

t3hk0d3 commented May 31, 2026

Action required: PR inactive for 5 days.
Status update or closure in 10 days.

Still waiting for reply

@github-actions github-actions Bot removed the state: inactive No current action needed/possible; issue fixed, out of scope, or superseded. label May 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type: feature Brand new functionality, features, pages, workflows, endpoints, etc.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Add support for API key helpers

3 participants