Limit SHOPIFY_ environment variables and credential flags in analytics - #8198
Closed
isaacroldan wants to merge 13 commits into
Closed
Limit SHOPIFY_ environment variables and credential flags in analytics#8198isaacroldan wants to merge 13 commits into
isaacroldan wants to merge 13 commits into
Conversation
isaacroldan
force-pushed
the
analytics-redact-shopify-env
branch
from
July 30, 2026 10:33
7463d4e to
0140d86
Compare
Contributor
Author
This stack of pull requests is managed by Graphite. Learn more about stacking. |
4 tasks
The analytics payload collected every SHOPIFY_* environment variable, which includes the credentials the CLI reads from the environment, and the payload sanitizer only recognised the shptka_ theme token format. Values for other credential env vars and for credential-bearing flags were reported verbatim and printed by the verbose debug sinks. - Collect an allowlist of the agent identification variables the field was added for, instead of matching on the SHOPIFY_ prefix. - Generalise the flag redaction from --store-password to any flag whose name contains password, token, secret or credential. - Apply the same name matching to object keys, so credentials arriving through plugin metadata or environment flags are covered too. Excludes `key` and `auth` from the name matching on purpose: api_key is an app's public client ID and env_auth_method is legitimate telemetry. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The flag rule matched a value as `"[^"]*"` or `[^\s"]+`, neither of which accounts for the payload already being JSON: a literal quote inside a value is `\"` there, so the quoted alternative could only fire by matching the closing quote of `args` and running into the next key. `--client-secret ""` was enough to produce unbalanced JSON, and the JSON.parse at the end of sanitizePayload then threw, dropping the whole event and reporting to Bugsnag. Match escape sequences as a unit instead. The key rule allowed one optional backslash before each quote, so it only reached keys nested a single serialised JSON string deep. `metadata` and `cmd_all_environment_flags` are serialised JSON whose values may be serialised JSON in turn, leaving those credentials unredacted. Allow any number.
The allowlist test already asserts the exact contents of env_shopify_variables, so the separate credential-env-var test added nothing.
The flag regex was complicated only because it ran after JSON.stringify, where it had to recognise escape sequences to avoid cutting one in half. Redacting the argument list before it is joined removes the need for it entirely, and covers a value containing a space, which was impossible to delimit once joined.
The allowlist test only covers env_shopify_variables; this covers the rest.
env_shopify_variables exists so a caller can set its own SHOPIFY_* variable and have it reach Monorail -- SHOPIFY_INVOKED_BY, from shopify-function-test-helpers in PR #6509, is set outside this repo entirely. An allowlist of the names we know about drops those callers silently. Filter on the name instead, with `key` treated as a credential marker for environment variables specifically, since SHOPIFY_PROXY_KEY holds a signed token.
Dropping credential variables from the payload also dropped the fact that they were set, and an allowlist would have dropped callers this repo has never heard of -- SHOPIFY_INVOKED_BY comes from shopify-function-test-helpers. Go back to reporting every SHOPIFY_ variable, as main does, and mask the value when the name marks it as a credential. That is what main already did for SHOPIFY_FLAG_STORE_PASSWORD; the rule was just too narrow to cover the rest.
The sanitizer already redacts every credential the CLI reads from the environment, since they are all named for what they hold. Filtering at collection time as well was a second mechanism for the same job.
Removing the string-level flag rule was not the pure simplification it looked like. Redacting the argument list covers `args`, but the old rule also matched anywhere else in the serialised payload -- a failing subprocess reports the command line it ran, and that reaches Monorail through error_message. Keep both: the list-level pass for `args`, where it can delimit values exactly, and an escape-aware string-level pass as the backstop for copies elsewhere.
Internal hardening of what the analytics payload reports; nothing here needs a line in the changelog.
The doubled backslashes came from building the patterns with new RegExp from a string, not from the matching itself. As literals they are the same shape as the two --store-password rules they replace, with the name generalised.
`key` marks a credential in the environment -- SHOPIFY_PROXY_KEY holds a signed token and SHOPIFY_FLAG_GRAPHIQL_KEY is derived from the app secret -- but `api_key` is an app's public client ID that Monorail is meant to receive, so the payload rules leave the name alone. Add the name to an extra pass over the copies that outputDebug prints, which is what --verbose turns on. Telemetry keeps the values; a terminal, and whatever scrapes it, does not see them.
isaacroldan
force-pushed
the
analytics-redact-shopify-env
branch
from
August 3, 2026 11:35
4eb005e to
be539ad
Compare
Redacting the argument list before joining was a second mechanism for what the flag rule already does. It only added a value containing a space, which needs quotes inside argv to arise, at the cost of a helper, a predicate and the wiring into buildPayload. redactForOutput also subsumes monorail's api_key masker, since api_key is a key-named entry, so that function goes too.
Contributor
Differences in type declarationsWe detected differences in the type declarations generated by Typescript for this branch compared to the baseline ('main' branch). Please, review them to ensure they are backward-compatible. Here are some important things to keep in mind:
New type declarationspackages/cli-kit/dist/private/node/analytics/redact-output.d.ts/**
* Redacts the copy of the analytics payload that gets printed.
*
* The payload itself is already sanitized for Monorail. This is the extra pass for
* the `outputDebug` sinks that `--verbose` turns on, where the audience is a
* terminal and whatever scrapes it rather than a sensitive Monorail field.
*
* `key` is the whole reason this exists. It marks a credential in the environment
* -- SHOPIFY_PROXY_KEY holds a signed token, SHOPIFY_FLAG_GRAPHIQL_KEY is derived
* from the app secret -- but `api_key` is an app's public client ID that Monorail
* is meant to receive, so the payload rules leave the name alone.
*
* @param payload - The already-sanitized analytics payload.
* @returns A copy with the values of `key`-named entries replaced.
*/
export declare function redactForOutput<T>(payload: T): T;
Existing type declarationsWe found no diffs with existing type declarations |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

WHY are these changes introduced?
The analytics payload is printed by
outputDebug, including under--verboseand when analytics are disabled, so credential values shouldn't be in it. Two gaps let them through, plus a bug in the redaction itself:sanitizePayload()only knew aboutshptka_tokens and--store-password, so flags like--client-secretpassed through, and so did the environment variables the CLI reads its own tokens from.--store-password ""unbalanced the string, the closingJSON.parsethrew, and the whole event was dropped. Credential keys nested insidemetadatawere missed for the same reason.WHAT is this pull request doing?
Widening what the sanitizer redacts. Nothing is dropped from the payload that
mainreports today, andgetShopifyEnvironmentVariablesis untouched.Two passes over the payload, and one extra over the copies that get printed.
sanitizePayloadmatches any flag or key whose name containspassword,token,secretorcredential— coveringargs, a command line quoted intoerror_message, and keys at any nesting depth insidemetadata,cmd_all_environment_flagsandenv_shopify_variables. This replaces the two--store-passwordrules, which it is a superset of. Values are matched escape-aware so a replacement can't cut a\"in half and leave the payload unparseable —--store-password ""used to do exactly that, and the event was dropped.authis excluded from the name matching on purpose, soenv_auth_methodkeeps reporting.keyis handled separately, inprivate/node/analytics/redact-output.ts. It marks a credential in the environment —SHOPIFY_PROXY_KEYholds a signed token andSHOPIFY_FLAG_GRAPHIQL_KEYis derived from the app secret — butapi_keyis an app's public client ID that Monorail is meant to receive. So the name is redacted only in the copiesoutputDebugprints, which is what--verboseturns on. Telemetry keeps the values; a terminal, and whatever scrapes it, does not see them.Still best effort. A credential whose name gives no hint is not redacted, and neither is a non-string value like
{"password": 12345}, since the key rule expects a quoted value.#8205 stacks on this and notes in the
--verbosehelp that the output may include sensitive data.How to test your changes?
pnpm test packages/cli-kit/src/public/node/analytics.test.tsenv_shopify_variablesshowsSHOPIFY_CLI_PARTNERS_TOKENwith its value as*****, andargsshows--client-secret *****.Checklist
🤖 Generated with Claude Code