Skip to content

Add run config passthrough to launch and materialise#1

Merged
lukew-cogapp merged 1 commit into
mainfrom
feat/launch-run-config
Jun 1, 2026
Merged

Add run config passthrough to launch and materialise#1
lukew-cogapp merged 1 commit into
mainfrom
feat/launch-run-config

Conversation

@lukew-cogapp
Copy link
Copy Markdown
Contributor

What

colflow launch and colflow materialise now accept Dagster run config, matching dg launch semantics:

  • --config <path> — JSON or YAML file. Repeatable; files shallow-merge left to right.
  • --config-json <json> — inline JSON. Merged last, wins over --config.

Config resolves through resolveRunConfig (src/commands/_runconfig.ts) and flows into the runConfigData field of ExecutionParams in launchRun / launchAssetRun, which previously hardcoded {}. With no flag the config is {}, so default-config behaviour is unchanged.

Why

colflow launch previously launched every job with default config only (the TUI even says "with default config"). No way to set resource/op config from the CLI. Motivating case: enabling use_vision on image_qc_pipeline without editing the resource default.

colflow launch image_qc_pipeline \
  --config-json '{"resources":{"colour_target_qc":{"config":{"use_vision":true}}}}'

Changes

  • src/commands/_runconfig.ts — shared resolver (file + inline, merge order, validation).
  • src/client/index.tslaunchRun / launchAssetRun take an optional runConfigData (default {}).
  • src/commands/{launch,materialise}.ts + src/cli.tsx — flag wiring + help text.
  • yaml dependency for YAML config files.
  • tests/runconfig.test.ts — 10 unit tests (merge order, file/inline precedence, error paths).
  • README + CLAUDE docs note the flag.

Verification

Tested live against a running Dagster dev server:

colflow launch image_qc_pipeline --config-json '{"resources":{"colour_target_qc":{"config":{"use_vision":true}}}}'
# run logged: QC starting: 133 images ... vision=True

bun run typecheck clean, bun test 173 pass.

Release

Feature bump to v0.5.0 after merge (tag on main triggers the compile + homebrew-tap release workflow).

Both commands now accept run config like dg launch does:

  --config <path>       JSON or YAML file, repeatable, shallow-merged
                        left to right
  --config-json <json>  inline JSON, merged last (wins over --config)

Config resolves via resolveRunConfig (src/commands/_runconfig.ts) and
flows into the runConfigData field of ExecutionParams in launchRun /
launchAssetRun, which previously hardcoded {}. With no flag the config
is {}, so default-config behaviour is unchanged.

Motivating case: setting resource config on image_qc_pipeline
(use_vision) without editing the resource default. colflow launch
previously launched every job with default config only.

Adds the yaml dependency for YAML config files and unit tests for the
resolver (merge order, file/inline precedence, error paths).
@lukew-cogapp lukew-cogapp merged commit 5da4a8e into main Jun 1, 2026
4 checks passed
@lukew-cogapp lukew-cogapp deleted the feat/launch-run-config branch June 1, 2026 09:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant