Skip to content

Limit SHOPIFY_ environment variables and credential flags in analytics - #8198

Closed
isaacroldan wants to merge 13 commits into
mainfrom
analytics-redact-shopify-env
Closed

Limit SHOPIFY_ environment variables and credential flags in analytics#8198
isaacroldan wants to merge 13 commits into
mainfrom
analytics-redact-shopify-env

Conversation

@isaacroldan

@isaacroldan isaacroldan commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

WHY are these changes introduced?

The analytics payload is printed by outputDebug, including under --verbose and when analytics are disabled, so credential values shouldn't be in it. Two gaps let them through, plus a bug in the redaction itself:

  • sanitizePayload() only knew about shptka_ tokens and --store-password, so flags like --client-secret passed through, and so did the environment variables the CLI reads its own tokens from.
  • It rewrites the already-serialised payload and parses it back, and its patterns didn't account for JSON escaping. --store-password "" unbalanced the string, the closing JSON.parse threw, and the whole event was dropped. Credential keys nested inside metadata were missed for the same reason.

WHAT is this pull request doing?

Widening what the sanitizer redacts. Nothing is dropped from the payload that main reports today, and getShopifyEnvironmentVariables is untouched.

Two passes over the payload, and one extra over the copies that get printed.

sanitizePayload matches any flag or key whose name contains password, token, secret or credential — covering args, a command line quoted into error_message, and keys at any nesting depth inside metadata, cmd_all_environment_flags and env_shopify_variables. This replaces the two --store-password rules, which it is a superset of. Values are matched escape-aware so a replacement can't cut a \" in half and leave the payload unparseable — --store-password "" used to do exactly that, and the event was dropped.

auth is excluded from the name matching on purpose, so env_auth_method keeps reporting.

key is handled separately, in private/node/analytics/redact-output.ts. It marks a credential in the environment — SHOPIFY_PROXY_KEY holds a signed token and SHOPIFY_FLAG_GRAPHIQL_KEY is derived from the app secret — but api_key is an app's public client ID that Monorail is meant to receive. So the name is redacted only in the copies outputDebug prints, which is what --verbose turns on. Telemetry keeps the values; a terminal, and whatever scrapes it, does not see them.

Still best effort. A credential whose name gives no hint is not redacted, and neither is a non-string value like {"password": 12345}, since the key rule expects a quoted value.

#8205 stacks on this and notes in the --verbose help that the output may include sensitive data.

How to test your changes?

pnpm test packages/cli-kit/src/public/node/analytics.test.ts

SHOPIFY_FLAG_VERBOSE=1 SHOPIFY_CLI_PARTNERS_TOKEN=atkn_fake_value \
  node packages/cli/bin/run.js version --verbose --client-secret cs_fake_value

env_shopify_variables shows SHOPIFY_CLI_PARTNERS_TOKEN with its value as *****, and args shows --client-secret *****.

Checklist

  • I've considered possible cross-platform impacts (Mac, Linux, Windows)
  • I've considered possible documentation changes
  • I've considered analytics changes to measure impact
  • The change is not user-facing — internal hardening of what the analytics payload reports, so no changeset

🤖 Generated with Claude Code

@isaacroldan
isaacroldan requested review from a team as code owners July 29, 2026 12:01
@github-actions github-actions Bot added the Area: @shopify/cli @shopify/cli package issues label Jul 29, 2026

@graygilmore graygilmore left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense!

@isaacroldan
isaacroldan force-pushed the analytics-redact-shopify-env branch from 7463d4e to 0140d86 Compare July 30, 2026 10:33

isaacroldan commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

@github-actions github-actions Bot added no-changelog This PR doesn't include a changeset entry. Is an internal only change not relevant to end users. and removed Area: @shopify/cli @shopify/cli package issues labels Jul 31, 2026
isaacroldan and others added 12 commits August 3, 2026 13:33
The analytics payload collected every SHOPIFY_* environment variable, which
includes the credentials the CLI reads from the environment, and the payload
sanitizer only recognised the shptka_ theme token format. Values for other
credential env vars and for credential-bearing flags were reported verbatim
and printed by the verbose debug sinks.

- Collect an allowlist of the agent identification variables the field was
  added for, instead of matching on the SHOPIFY_ prefix.
- Generalise the flag redaction from --store-password to any flag whose name
  contains password, token, secret or credential.
- Apply the same name matching to object keys, so credentials arriving through
  plugin metadata or environment flags are covered too.

Excludes `key` and `auth` from the name matching on purpose: api_key is an
app's public client ID and env_auth_method is legitimate telemetry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The flag rule matched a value as `"[^"]*"` or `[^\s"]+`, neither of which
accounts for the payload already being JSON: a literal quote inside a value is
`\"` there, so the quoted alternative could only fire by matching the closing
quote of `args` and running into the next key. `--client-secret ""` was enough to
produce unbalanced JSON, and the JSON.parse at the end of sanitizePayload then
threw, dropping the whole event and reporting to Bugsnag. Match escape sequences
as a unit instead.

The key rule allowed one optional backslash before each quote, so it only
reached keys nested a single serialised JSON string deep. `metadata` and
`cmd_all_environment_flags` are serialised JSON whose values may be serialised
JSON in turn, leaving those credentials unredacted. Allow any number.
The allowlist test already asserts the exact contents of env_shopify_variables, so
the separate credential-env-var test added nothing.
The flag regex was complicated only because it ran after JSON.stringify, where it
had to recognise escape sequences to avoid cutting one in half. Redacting the
argument list before it is joined removes the need for it entirely, and covers a
value containing a space, which was impossible to delimit once joined.
The allowlist test only covers env_shopify_variables; this covers the rest.
env_shopify_variables exists so a caller can set its own SHOPIFY_* variable and
have it reach Monorail -- SHOPIFY_INVOKED_BY, from shopify-function-test-helpers
in PR #6509, is set outside this repo entirely. An allowlist of the names we know
about drops those callers silently.

Filter on the name instead, with `key` treated as a credential marker for
environment variables specifically, since SHOPIFY_PROXY_KEY holds a signed token.
Dropping credential variables from the payload also dropped the fact that they
were set, and an allowlist would have dropped callers this repo has never heard
of -- SHOPIFY_INVOKED_BY comes from shopify-function-test-helpers.

Go back to reporting every SHOPIFY_ variable, as main does, and mask the value
when the name marks it as a credential. That is what main already did for
SHOPIFY_FLAG_STORE_PASSWORD; the rule was just too narrow to cover the rest.
The sanitizer already redacts every credential the CLI reads from the environment,
since they are all named for what they hold. Filtering at collection time as well
was a second mechanism for the same job.
Removing the string-level flag rule was not the pure simplification it looked
like. Redacting the argument list covers `args`, but the old rule also matched
anywhere else in the serialised payload -- a failing subprocess reports the
command line it ran, and that reaches Monorail through error_message.

Keep both: the list-level pass for `args`, where it can delimit values exactly,
and an escape-aware string-level pass as the backstop for copies elsewhere.
Internal hardening of what the analytics payload reports; nothing here needs a
line in the changelog.
The doubled backslashes came from building the patterns with new RegExp from a
string, not from the matching itself. As literals they are the same shape as the
two --store-password rules they replace, with the name generalised.
`key` marks a credential in the environment -- SHOPIFY_PROXY_KEY holds a signed
token and SHOPIFY_FLAG_GRAPHIQL_KEY is derived from the app secret -- but
`api_key` is an app's public client ID that Monorail is meant to receive, so the
payload rules leave the name alone.

Add the name to an extra pass over the copies that outputDebug prints, which is
what --verbose turns on. Telemetry keeps the values; a terminal, and whatever
scrapes it, does not see them.
@isaacroldan
isaacroldan force-pushed the analytics-redact-shopify-env branch from 4eb005e to be539ad Compare August 3, 2026 11:35
Redacting the argument list before joining was a second mechanism for what the
flag rule already does. It only added a value containing a space, which needs
quotes inside argv to arise, at the cost of a helper, a predicate and the wiring
into buildPayload.

redactForOutput also subsumes monorail's api_key masker, since api_key is a
key-named entry, so that function goes too.
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Differences in type declarations

We detected differences in the type declarations generated by Typescript for this branch compared to the baseline ('main' branch). Please, review them to ensure they are backward-compatible. Here are some important things to keep in mind:

  • Some seemingly private modules might be re-exported through public modules.
  • If the branch is behind main you might see odd diffs, rebase main into this branch.

New type declarations

packages/cli-kit/dist/private/node/analytics/redact-output.d.ts
/**
 * Redacts the copy of the analytics payload that gets printed.
 *
 * The payload itself is already sanitized for Monorail. This is the extra pass for
 * the `outputDebug` sinks that `--verbose` turns on, where the audience is a
 * terminal and whatever scrapes it rather than a sensitive Monorail field.
 *
 * `key` is the whole reason this exists. It marks a credential in the environment
 * -- SHOPIFY_PROXY_KEY holds a signed token, SHOPIFY_FLAG_GRAPHIQL_KEY is derived
 * from the app secret -- but `api_key` is an app's public client ID that Monorail
 * is meant to receive, so the payload rules leave the name alone.
 *
 * @param payload - The already-sanitized analytics payload.
 * @returns A copy with the values of `key`-named entries replaced.
 */
export declare function redactForOutput<T>(payload: T): T;

Existing type declarations

We found no diffs with existing type declarations

@isaacroldan isaacroldan closed this Aug 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-changelog This PR doesn't include a changeset entry. Is an internal only change not relevant to end users.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants