Add 5 new Qwen models and update DeepSeek V4 Flash to the 0731 checkpoint (OpenRouter) - #1637
Merged
Merged
Conversation
Add five new Qwen models on OpenRouter, each verified with paid integration tests: - Qwen 3.7 Flash, Qwen 3.6 Flash, Qwen 3.6 35B A3B, Qwen 3.5 9B: multimodal (image/video), json_instruction_and_object. - Qwen 3 Max Thinking: reasoning_capable left False (adaptive reasoning, does not always emit reasoning). Only providers that actually serve inference were included. Fireworks lists these Qwen slugs but they 404 (not deployed); Together hosts the Plus variants as streaming-only; Featherless returns degenerate/prose output for the small Qwen deployments. DeepSeek V4 Flash 0731 was requested but has no live OpenRouter endpoints yet, so the existing deepseek-v4-flash entry is left unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019JtLfiQC2NQKm3jz34soxT
Contributor
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
Contributor
WalkthroughFive Qwen models were added to ChangesQwen model catalog
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (1)
libs/core/kiln_ai/adapters/ml_model_list.py (1)
212-226: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winFix these Ruff errors before merge.
ruff checkfails with 8 lint errors inlibs/core/kiln_ai/adapters/ml_model_list.py: replace deprecated mutableListimports/annotations withlist, and fix the mutable default argument inensure_valid_structured_output_mode.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@libs/core/kiln_ai/adapters/ml_model_list.py` around lines 212 - 226, Resolve the Ruff errors in ml_model_list.py by replacing deprecated typing.List imports and annotations with the built-in list type, and update ensure_valid_structured_output_mode to avoid a mutable default argument by using a safe None-based initialization pattern. Preserve the existing validation behavior and list contents.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@libs/core/kiln_ai/adapters/ml_model_list.py`:
- Around line 212-226: Resolve the Ruff errors in ml_model_list.py by replacing
deprecated typing.List imports and annotations with the built-in list type, and
update ensure_valid_structured_output_mode to avoid a mutable default argument
by using a safe None-based initialization pattern. Preserve the existing
validation behavior and list contents.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 45c61890-a36e-4acb-b459-9c1e3da82dec
📒 Files selected for processing (1)
libs/core/kiln_ai/adapters/ml_model_list.py
📊 Coverage ReportOverall Coverage: 92% Diff: origin/main...HEADNo lines with coverage information in this diff.
|
…nRouter) The 0731 checkpoint is now live on OpenRouter (it had no endpoints earlier) and advertises native structured outputs, so the OpenRouter slug is updated from deepseek/deepseek-v4-flash to the 0731 checkpoint, keeping json_schema and the existing thinking levels. Fireworks is unchanged. Config verified: Fireworks passed its full suite, and the OpenRouter provider passed structured-output, COT, and plaintext tests when requests got through. The remaining OpenRouter failures were all HTTP 429 "temporarily rate-limited upstream" errors on the freshly-launched 0731 shared pool (transient), not config or schema errors. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019JtLfiQC2NQKm3jz34soxT
tawnymanticore
changed the title
leonardmq
approved these changes
Aug 5, 2026
2 tasks
tawnymanticore
added a commit
that referenced
this pull request
Aug 17, 2026
… Qwen 3.8, and Featherless (#1631) * Bump js-yaml from 4.1.1 to 4.3.0 in /app/web_ui Bumps [js-yaml](https://github.com/nodeca/js-yaml) from 4.1.1 to 4.3.0. - [Changelog](https://github.com/nodeca/js-yaml/blob/master/CHANGELOG.md) - [Commits](https://github.com/nodeca/js-yaml/compare/4.1.1...4.3.0) --- updated-dependencies: - dependency-name: js-yaml dependency-version: 4.3.0 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump soupsieve from 2.8 to 2.8.4 Bumps [soupsieve](https://github.com/facelessuser/soupsieve) from 2.8 to 2.8.4. - [Release notes](https://github.com/facelessuser/soupsieve/releases) - [Commits](https://github.com/facelessuser/soupsieve/compare/2.8...2.8.4) --- updated-dependencies: - dependency-name: soupsieve dependency-version: 2.8.4 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * Fix broken/flaky Playwright discover e2e tests on main Both fixes are test-level; no product code changed. data-generation.spec.ts (hard failure): the synth flow gained a SynthBatchChooser step (added 2026-07-11) between "Continue Without Data Guide" and the DataGenIntro. The test (last updated 2026-05-07) expected "Add Topics" immediately after skipping the guide. Updated the test to click "Create Manually" in the chooser before asserting the DataGenIntro buttons, matching the current intended flow. extractors.spec.ts (flaky): the advanced-options step used check({ force: true }) on the DaisyUI collapse checkbox. The app's global loading overlay (+layout.svelte, z-[1000], shown while projects load) can still cover the page at that point; force clicks straight through onto the overlay, so the checkbox never toggles ("Clicking the checkbox did not change its state"), and toBeVisible passes anyway because it ignores obscuring elements. Replaced with a normal (actionability-checked) click, which auto-waits for the overlay to clear before hitting the checkbox. Independent of the KIL-771 prompt-caching work. * Increase compare screen max run configs from 6 to 10 (#1602) Raise MAX_COLUMNS on the run config comparison screen so users can compare up to 10 run configs side-by-side instead of 6. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Add Kimi K3, Grok 4.5, Qwen 3.7 Max, Mistral Medium 3.5, and Thinking Machines Inkling (#1593) * Add Kimi K3, Grok 4.5, Qwen 3.7 Max, Mistral Medium 3.5, and Inkling to model list * Reorder Qwen 3.7 Max to top of Qwen group and add Inkling on OpenRouter Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AWmHGnzanDt8c1sBBwFpLW --------- Co-authored-by: Claude <noreply@anthropic.com> * Add Gemini 3.6 Flash and Gemini 3.5 Flash Lite to model list (#1604) * Add Gemini 3.6 Flash and Gemini 3.5 Flash Lite to model list * Rename RAG extractor config to Gemini 3.5 Flash Lite to match model * Update RAG template preview labels to Gemini 3.5 Flash Lite --------- Co-authored-by: Claude <noreply@anthropic.com> * Deprecate 54 dead provider entries (Fireworks + Together serverless purges, Anthropic/OpenRouter/SiliconFlow removals) (#1588) * Deprecate 44 removed provider entries across 5 providers Fireworks purged serverless deployment for 32 of Kiln's 43 entries (verified via 1-token inference probes; catalog detail API still returns 200/READY for undeployed models). Also: Anthropic retired Claude 4 (Opus/Sonnet 20250514), OpenRouter removed 7 models, SiliconFlow removed Pro GLM-4.7/GLM-5, Together removed the Qwen3-235B-A22B-Instruct-2507-tput slug. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Inference-verify all entries: +11 Together removals, revert SiliconFlow GLM-5 1-token inference probes (with live positive controls per provider) against every deprecated entry revealed: (a) Together has also pulled 11 more Kiln entries off serverless (model_not_available, confirmed twice), and (b) SiliconFlow Pro/zai-org/GLM-5 still serves inference despite being delisted from /v1/models - reverted its deprecation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * New openrouter logo * Bump dompurify from 3.4.11 to 3.4.12 in /app/web_ui Bumps [dompurify](https://github.com/cure53/DOMPurify) from 3.4.11 to 3.4.12. - [Release notes](https://github.com/cure53/DOMPurify/releases) - [Commits](https://github.com/cure53/DOMPurify/compare/3.4.11...3.4.12) --- updated-dependencies: - dependency-name: dompurify dependency-version: 3.4.12 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * Bump fast-uri from 3.1.2 to 3.1.4 in /app/web_ui Bumps [fast-uri](https://github.com/fastify/fast-uri) from 3.1.2 to 3.1.4. - [Release notes](https://github.com/fastify/fast-uri/releases) - [Commits](https://github.com/fastify/fast-uri/compare/v3.1.2...v3.1.4) --- updated-dependencies: - dependency-name: fast-uri dependency-version: 3.1.4 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * Bump pyasn1 from 0.6.3 to 0.6.4 Bumps [pyasn1](https://github.com/pyasn1/pyasn1) from 0.6.3 to 0.6.4. - [Release notes](https://github.com/pyasn1/pyasn1/releases) - [Changelog](https://github.com/pyasn1/pyasn1/blob/main/CHANGES.rst) - [Commits](https://github.com/pyasn1/pyasn1/compare/v0.6.3...v0.6.4) --- updated-dependencies: - dependency-name: pyasn1 dependency-version: 0.6.4 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * Bump setuptools from 80.9.0 to 83.0.0 Bumps [setuptools](https://github.com/pypa/setuptools) from 80.9.0 to 83.0.0. - [Release notes](https://github.com/pypa/setuptools/releases) - [Changelog](https://github.com/pypa/setuptools/blob/main/NEWS.rst) - [Commits](https://github.com/pypa/setuptools/compare/v80.9.0...v83.0.0) --- updated-dependencies: - dependency-name: setuptools dependency-version: 83.0.0 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * Add Muse Spark 1.1 (Meta) to model list via OpenRouter (#1611) * Add Muse Spark 1.1 (Meta) to model list via OpenRouter * Move Muse Spark 1.1 entry next to the Llama models --------- Co-authored-by: Claude <noreply@anthropic.com> * Bump echarts from 6.0.0 to 6.1.0 in /app/web_ui Bumps [echarts](https://github.com/apache/echarts) from 6.0.0 to 6.1.0. - [Release notes](https://github.com/apache/echarts/releases) - [Commits](https://github.com/apache/echarts/compare/6.0.0...6.1.0) --- updated-dependencies: - dependency-name: echarts dependency-version: 6.1.0 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * Revert "Merge pull request #1610 from Kiln-AI/dependabot/uv/setuptools-83.0.0" This reverts commit 7c87137e3d4140daba24289fbfa01ebb573ffd7f, reversing changes made to 0a65e637a13e0489e671b45023474d1c3b7cac48. * CI: run Build Desktop Apps on dependency PRs The desktop packaging job only ran on push-to-main / release, so a lockfile bump that breaks the bundle (e.g. setuptools 83 dropping pkg_resources, which broke all 5 platforms on main) passed every PR check and only failed after merge. Add a path-filtered pull_request trigger so dependency PRs run the same desktop build pre-merge. The build uses no secrets, so Dependabot PRs run it fine. * Bump pypdf from 6.13.0 to 6.14.2 Bumps [pypdf](https://github.com/py-pdf/pypdf) from 6.13.0 to 6.14.2. - [Release notes](https://github.com/py-pdf/pypdf/releases) - [Changelog](https://github.com/py-pdf/pypdf/blob/main/CHANGELOG.md) - [Commits](https://github.com/py-pdf/pypdf/compare/6.13.0...6.14.2) --- updated-dependencies: - dependency-name: pypdf dependency-version: 6.14.2 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * Add Featherless AI as a model provider (7 models, paid-tested) (#1618) * Add Featherless AI as a model provider, with GLM 5.2 Featherless is a serverless host for open models. LiteLLM already ships a native featherless_ai provider, so this wires it through Kiln's provider plumbing rather than routing it as a custom OpenAI-compatible endpoint. Adds GLM 5.2 (zai-org/GLM-5.2) as the first model, matching the json_instructions structured output mode already used for GLM 5.2 on the other third-party hosts (together_ai, siliconflow_cn, fireworks). Key validation is unusual for this provider: Featherless has no authenticated GET endpoint to ping — /v1/models is public and returns 200 without a key, so it can't validate anything. connect_featherless instead POSTs to chat/completions with a model slug that intentionally doesn't exist. Featherless checks auth before resolving the model, so a bad key 401s while a good key falls through to a model error. That validates the key without spending tokens and without depending on any real model slug staying available. Not included: the generated Copilot API client (app/desktop/studio_server/api_client/) still lacks featherless_ai, since that enum mirrors the remote Copilot service's schema and needs a server-side change first. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Make Featherless icon match the monochrome provider-icon convention The initial asset was Featherless's favicon: a full-color app tile with a yellow (#FEF47A) rounded-rect background. Every other provider icon is a bare monochrome glyph with no background box, so it stood out. Drops the background rect, switches the mark to currentColor (matching openai/groq/siliconflow/openrouter), and crops the viewBox to the artwork bounds — without the tile's built-in padding the glyph filled only 76x178 of a 256 viewBox and rendered small in the 24px slot. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Backfill Featherless AI: latest model per family Adds the newest model in each family that Featherless actually hosts, on top of the existing GLM 5.2 entry. Nine new provider entries: deepseek DeepSeek V4 Pro deepseek-ai/DeepSeek-V4-Pro gemma Gemma 4 31B google/gemma-4-31B-it gpt GPT OSS 120B openai/gpt-oss-120b kimi Kimi K2.6 moonshotai/Kimi-K2.6 llama Llama 3.3 70B meta-llama/Llama-3.3-70B-Instruct minimax Minimax M3 MiniMaxAI/MiniMax-M3 mistral Mistral Small 3 mistralai/Mistral-Small-24B-Instruct-2501 phi Phi 4 - 14B microsoft/phi-4 qwen Qwen 3.5 397B A17B Qwen/Qwen3.5-397B-A17B Every slug was verified to exist verbatim in Featherless's live /v1/models response, and each entry's capability flags were cross-checked against what that endpoint advertises per model -- supports_function_calling is False wherever Featherless doesn't report tool_use (GPT OSS 120B, Minimax M3), and vision is only enabled where it reports image_input. Config rule: mirror the together_ai sibling where one exists, since it's the closest analog (same HF slug convention, same class of third-party host). Otherwise use json_instructions, the mode already validated against this provider. OpenRouter-specific flags (openrouter_reasoning_object, r1_openrouter_options, require_openrouter_reasoning, thinking levels) are dropped since they don't apply here. Two judgement calls worth flagging for review: - Minimax M3 and Qwen 3.5 397B use parser=optional_r1_thinking with reasoning_capable left False. Featherless serves raw weights that emit <think> inline, but the existing Fireworks entry notes M3 doesn't always emit reasoning -- requiring it would error on those responses, so the optional parser strips tags when present and tolerates their absence. - suggested_for_evals / suggested_for_data_gen are deliberately NOT set on any of these, since none are tested on this provider yet. Easy to flip. None of these are runtime-tested; only GLM 5.2 has had a live run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Paid-test Featherless models; drop the three that fail; document provider Ran the full paid suite against the live Featherless API. Final result for the shipped set: 73 passed, 10 skipped, 1 failed. The single failure is pre-existing and provider-independent: test_structured_input_cot_prompt_builder asserts len(trace) == 5, which is incompatible with any provider setting reasoning_capable=True (that selects the single-call strategy, 3 messages). Verified it fails identically for gpt_oss_120b on fireworks_ai on a clean tree. Dropped three models that failed persistently: - Llama 3.3 70B: HTTP 403 model_gated_needs_oauth. Featherless gates it behind per-user HuggingFace org linkage, so it cannot work for arbitrary Kiln users. All 20 official meta-llama repos there are gated, so no Llama variant is usable. My original matcher didn't filter is_gated -- fixed in the skill. - Qwen 3.5 397B: the Featherless deployment returns degenerate output, rambling to the 4096-token cap with unrelated word lists. Fails even the plaintext test. Not a config issue. - Phi 4 14B: wraps JSON in prose and markdown fences, and Featherless cannot constrain it (see below). llm_as_judge and structured-output COT fail. Confirmed json_instructions is the only viable structured output mode. LiteLLM's featherless_ai provider rejects response_format outright (UnsupportedParamsError), ruling out json_schema, json_mode and json_instruction_and_object. Routing as a custom openai provider bypasses that gate but is not reliable: GLM 5.2 accepts json_schema and silently ignores it (returned bare text for a schema'd request), while DeepSeek V4 Pro and Kimi K2.6 return APIErrors. Not worth the risk, so the native provider stays. Skill updates (.agents/skills/claude-maintain-models/SKILL.md): - New Featherless section in Provider Quirks: json_instructions only, the gating trap, no cost reporting, per-deployment quality variance. - New Lagging Providers entry with jq recipes. Their /v1/models is the only authoritative source (absent from both models.dev and the LiteLLM catalog), and at ~22k models it must never be WebFetched or dumped raw. Includes an is_gated + official-org filter and a verbatim-slug check against what Kiln already ships for together_ai / siliconflow_cn. - Corrected two stale instructions that cost time this run: there is no pytest.ini (config is in pyproject.toml, -n auto already active), and paid tests read keys from the ENVIRONMENT -- conftest's autouse use_temp_settings_dir fixture means a key added via the app's provider page is invisible to tests. Documented the bridging one-liner, the --ollama marker requirement, and a step to cross-check failures against an existing provider before calling them regressions. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Add Claude Opus 5 and Kimi K3 (Fireworks) to model list (#1625) * Add Claude Opus 5 to model list (anthropic, openrouter) Adds claude_opus_5 with Anthropic (claude-opus-5) and OpenRouter (anthropic/claude-opus-5) providers, a new CLAUDE_OPUS_5_ANTHROPIC_THINKING_LEVELS constant (low/medium/high/xhigh/max, default high), and moves featured_rank + suggested_for_evals/data_gen from Opus 4.8 to Opus 5. * Add Kimi K3 Fireworks AI provider (backfill) Adds accounts/fireworks/models/kimi-k3 to the existing kimi_k3 entry (was OpenRouter-only), mirroring the K2.6 Fireworks config, and corrects the stale "not yet on Fireworks" comment. Still absent from Together AI and SiliconFlow (.cn). --------- Co-authored-by: Claude <noreply@anthropic.com> * Return JSON 404s for API routes instead of the web app's 404 page Unmatched /api paths were answered with the compiled Svelte app's 404.html: StaticFiles is mounted at "/" in html mode, and on a miss it returns a 404 FileResponse of 404.html rather than raising, so the app's 404 exception handler never ran and HTMLStaticFiles' .html fallback re-served the same page. HTMLStaticFiles now raises a 404 for API paths before consulting the file system, letting the existing handler answer with JSON. The guard only applies to GET/HEAD so other methods keep falling through to StaticFiles' own 405, and API path matching is segment-aware so web app routes like /api-keys still get the 404 page. The JSON body now uses "message", matching every other Kiln API error and the key the web UI reads; two error paths in task_sample_example.ts that read the old "detail" key are updated to match. Test coverage for the gap: the studio fixture now ships a 404.html (as the real build does), which is the condition that triggered the bug, and tests cover unmatched API paths, HEAD, non-GET methods, matched routes, non-API paths that start with "api", and handler precedence when the shared error handlers are registered alongside the web host. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Pf6BHCk1YW3JF993WZPxsi * Bump nltk from 3.9.4 to 3.10.0 Bumps [nltk](https://github.com/nltk/nltk) from 3.9.4 to 3.10.0. - [Release notes](https://github.com/nltk/nltk/releases) - [Changelog](https://github.com/nltk/nltk/blob/develop/ChangeLog) - [Commits](https://github.com/nltk/nltk/compare/3.9.4...v3.10.0) --- updated-dependencies: - dependency-name: nltk dependency-version: 3.10.0 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * Bump @sveltejs/kit from 2.61.1 to 2.70.2 in /app/web_ui Bumps [@sveltejs/kit](https://github.com/sveltejs/kit/tree/HEAD/packages/kit) from 2.61.1 to 2.70.2. - [Release notes](https://github.com/sveltejs/kit/releases) - [Changelog](https://github.com/sveltejs/kit/blob/version-3/packages/kit/CHANGELOG.md) - [Commits](https://github.com/sveltejs/kit/commits/@sveltejs/kit@2.70.2/packages/kit) --- updated-dependencies: - dependency-name: "@sveltejs/kit" dependency-version: 2.70.2 dependency-type: direct:development ... Signed-off-by: dependabot[bot] <support@github.com> * Bump postcss from 8.5.14 to 8.5.25 in /app/web_ui Bumps [postcss](https://github.com/postcss/postcss) from 8.5.14 to 8.5.25. - [Release notes](https://github.com/postcss/postcss/releases) - [Changelog](https://github.com/postcss/postcss/blob/main/CHANGELOG.md) - [Commits](https://github.com/postcss/postcss/compare/8.5.14...8.5.25) --- updated-dependencies: - dependency-name: postcss dependency-version: 8.5.25 dependency-type: direct:development ... Signed-off-by: dependabot[bot] <support@github.com> * Bump brace-expansion from 1.1.14 to 1.1.18 in /app/web_ui Bumps [brace-expansion](https://github.com/juliangruber/brace-expansion) from 1.1.14 to 1.1.18. - [Release notes](https://github.com/juliangruber/brace-expansion/releases) - [Commits](https://github.com/juliangruber/brace-expansion/compare/v1.1.14...v1.1.18) --- updated-dependencies: - dependency-name: brace-expansion dependency-version: 1.1.18 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * Bump undici in /app/web_ui Bumps and [undici](https://github.com/nodejs/undici). These dependencies needed to be updated together. Updates `undici` from 6.26.0 to 6.28.0 - [Release notes](https://github.com/nodejs/undici/releases) - [Commits](https://github.com/nodejs/undici/compare/v6.26.0...v6.28.0) Updates `undici` from 7.27.2 to 7.29.0 - [Release notes](https://github.com/nodejs/undici/releases) - [Commits](https://github.com/nodejs/undici/compare/v6.26.0...v6.28.0) --- updated-dependencies: - dependency-name: undici dependency-version: 6.28.0 dependency-type: indirect - dependency-name: undici dependency-version: 7.29.0 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * Teach the root-first protocol in the skill tool description (#1635) The `skill` tool's description carried three literal example resource paths (`references/guide.md`, `references/subdir/notes.txt`, `assets/template.csv`) and never stated that a skill's instruction page is where real resource paths come from. A concrete-looking path in a tool schema gets read as a real path: on an external 187-run agent-port corpus, `references/guide.md` was issued verbatim as an actual resource request 32 times, including at skills that ship no resource files, and 91% of resource reads (227/250) happened with no prior skill-root load. The description now states the progressive-disclosure protocol Kiln's skills already follow (agentskills.io / SKILL.md): call `skill(name)` first for the instruction page, which is the only place resource files are listed, then `skill(name, resource)` with a path copied from it. Paths are described by shape (`references/` or `assets/` prefix) rather than spelled out, and guessing is ruled out explicitly. The `resource` parameter description says the same thing at the argument itself. Description text only: no behavior, error `output` strings, or function signature changed. New tests pin the root-first wording and assert no concrete resource path appears anywhere in the tool schema. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> * Bump aiohttp from 3.13.5 to 3.14.3 Bumps [aiohttp](https://github.com/aio-libs/aiohttp) from 3.13.5 to 3.14.3. - [Changelog](https://github.com/aio-libs/aiohttp/blob/master/CHANGES.rst) - [Commits](https://github.com/aio-libs/aiohttp/compare/v3.13.5...v3.14.3) --- updated-dependencies: - dependency-name: aiohttp dependency-version: 3.14.3 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * Bump fast-uri from 3.1.4 to 3.1.5 in /app/web_ui Bumps [fast-uri](https://github.com/fastify/fast-uri) from 3.1.4 to 3.1.5. - [Release notes](https://github.com/fastify/fast-uri/releases) - [Commits](https://github.com/fastify/fast-uri/compare/v3.1.4...v3.1.5) --- updated-dependencies: - dependency-name: fast-uri dependency-version: 3.1.5 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * Bump cryptography from 48.0.1 to 50.0.0 Bumps [cryptography](https://github.com/pyca/cryptography) from 48.0.1 to 50.0.0. - [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst) - [Commits](https://github.com/pyca/cryptography/compare/48.0.1...50.0.0) --- updated-dependencies: - dependency-name: cryptography dependency-version: 50.0.0 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * Add 5 new Qwen models and update DeepSeek V4 Flash to the 0731 checkpoint (OpenRouter) (#1637) * Add Qwen 3.7 Flash, 3.6 Flash, 3.6 35B A3B, 3.5 9B, and 3 Max Thinking Add five new Qwen models on OpenRouter, each verified with paid integration tests: - Qwen 3.7 Flash, Qwen 3.6 Flash, Qwen 3.6 35B A3B, Qwen 3.5 9B: multimodal (image/video), json_instruction_and_object. - Qwen 3 Max Thinking: reasoning_capable left False (adaptive reasoning, does not always emit reasoning). Only providers that actually serve inference were included. Fireworks lists these Qwen slugs but they 404 (not deployed); Together hosts the Plus variants as streaming-only; Featherless returns degenerate/prose output for the small Qwen deployments. DeepSeek V4 Flash 0731 was requested but has no live OpenRouter endpoints yet, so the existing deepseek-v4-flash entry is left unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019JtLfiQC2NQKm3jz34soxT * Point DeepSeek V4 Flash at the deepseek-v4-flash-0731 checkpoint (OpenRouter) The 0731 checkpoint is now live on OpenRouter (it had no endpoints earlier) and advertises native structured outputs, so the OpenRouter slug is updated from deepseek/deepseek-v4-flash to the 0731 checkpoint, keeping json_schema and the existing thinking levels. Fireworks is unchanged. Config verified: Fireworks passed its full suite, and the OpenRouter provider passed structured-output, COT, and plaintext tests when requests got through. The remaining OpenRouter failures were all HTTP 429 "temporarily rate-limited upstream" errors on the freshly-launched 0731 shared pool (transient), not config or schema errors. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019JtLfiQC2NQKm3jz34soxT --------- Co-authored-by: Claude <noreply@anthropic.com> * Bump dompurify from 3.4.12 to 3.4.13 in /app/web_ui Bumps [dompurify](https://github.com/cure53/DOMPurify) from 3.4.12 to 3.4.13. - [Release notes](https://github.com/cure53/DOMPurify/releases) - [Commits](https://github.com/cure53/DOMPurify/compare/3.4.12...3.4.13) --- updated-dependencies: - dependency-name: dompurify dependency-version: 3.4.13 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * Add DeepSeek V4 Flash 0731 providers, MiniMax M2.7 & Inkling (Fireworks), and Muse Spark 1.2 (OpenRouter) (#1665) * Add DeepSeek V4 Flash 0731 providers, MiniMax M2.7 (Fireworks), re-enable Inkling on Fireworks - DeepSeek V4 Flash: pin Fireworks to the 0731 snapshot and add Together AI provider - MiniMax M2.7: add Fireworks provider - Inkling: re-enable Fireworks provider (now deployed) * Add Muse Spark 1.2 (OpenRouter) Meta Muse Spark 1.2 via OpenRouter (meta/muse-spark-1.2), mirroring the 1.1 config; html/csv excluded from doc extraction since the Meta backend rejects those uploads. --------- Co-authored-by: Claude <noreply@anthropic.com> * Bump js-yaml from 4.3.0 to 4.3.1 in /app/web_ui Bumps [js-yaml](https://github.com/nodeca/js-yaml) from 4.3.0 to 4.3.1. - [Changelog](https://github.com/nodeca/js-yaml/blob/4.3.1/CHANGELOG.md) - [Commits](https://github.com/nodeca/js-yaml/compare/4.3.0...4.3.1) --- updated-dependencies: - dependency-name: js-yaml dependency-version: 4.3.1 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * Bump pypdf from 6.14.2 to 6.15.0 Bumps [pypdf](https://github.com/py-pdf/pypdf) from 6.14.2 to 6.15.0. - [Release notes](https://github.com/py-pdf/pypdf/releases) - [Changelog](https://github.com/py-pdf/pypdf/blob/main/CHANGELOG.md) - [Commits](https://github.com/py-pdf/pypdf/compare/6.14.2...6.15.0) --- updated-dependencies: - dependency-name: pypdf dependency-version: 6.15.0 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * Add cloud sandbox DX project: overview and research findings Starts a spec project for improving work on Kiln inside Claude Code cloud sandboxes. Includes the project overview and a research phase with measurements taken on a live sandbox: - The sandbox ships uv 0.8.17, which cannot parse the repo's relative `exclude-newer = "7 days"`. It silently re-resolves the graph on every plain `uv run`, rewriting uv.lock and breaking imports. First working version is 0.10.0. - Setup is already ~37s cold vs ~39s warm, so a dependency cache would save nothing. - Building the venv on a uv-managed Python 3.13 (which bundles Tk) removes all 5 tkinter collection errors and unblocks the OpenAPI schema scripts. - Test parallelism is healthy at ~3.0x on 4 cores; `-n auto` is optimal and `-n 8` is 42% slower. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GbnWHq5aFRM3gviE2UBPiE * Record planning decisions for cloud sandbox DX project Captures the settled choices: reuse and de-interactivize setup_env.sh, enforce a uv floor of >=0.10, pin Python 3.13 via a generated gitignored .python-version, restore agent config via the existing claude/setup.sh, and run the two dependency installs in parallel in the foreground. Also records that .agents/mcp.json needs an `mcp<2` pin: hooks-mcp 0.2.4 resolves against mcp 2.0.0, which dropped Server.list_tools, so the MCP server currently fails to start. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GbnWHq5aFRM3gviE2UBPiE * Record --agent flag decision for setup_env.sh setup_env.sh also runs the agent config setup via --agent all|claude|cursor|none, defaulting to all. Everything the agent scripts write is gitignored and the whole copy is 176K with no network, so running both by default is free and avoids privileging one editor. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GbnWHq5aFRM3gviE2UBPiE * Drop mcp<2 pin: hooks-mcp incompatibility is being fixed upstream Pinning mcp<2 in this repo would hold back the upstream fix. Once a fixed hooks-mcp ships, set a version floor in .agents/mcp.json instead, since uvx reuses cached tool environments and could keep serving the broken pair. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GbnWHq5aFRM3gviE2UBPiE * Add round 2 research: uv upgrade path, MCP trust, pytest startup, litellm - uv self update fails structurally here (GitHub API rate limit is per egress IP). `pip install --user --upgrade uv` takes 2.4s and drops a plain `uv run` from 16.4s to 0.072s with uv.lock untouched. - Project .mcp.json is never auto-trusted. Only user-level ~/.claude/settings.json with enableAllProjectMcpServers works; project-scoped settings files do not. - Deferring the module-level litellm import in the root conftest takes a single test file from ~7.4s to ~0.95s with identical pass/skip counts. Ignoring the paid-heavy test files saves only ~1.7s and is rejected. - import litellm does fetch the model cost map over the network, but that is only ~0.6s of ~3.9s; the rest is 2148 modules of eager imports. Forcing the local map is unsafe in production (290 models missing) but safe for tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GbnWHq5aFRM3gviE2UBPiE * Record uv upgrade, conftest, and .env decisions - uv upgrade moves into setup_env.sh behind an opt-in --upgrade-tools flag, using `uv tool install --force uv` (no pin, no pip). Without the flag it prompts with a 10s timeout, or warns when there is no TTY. - Drop the cloud_env_setup.sh idea; the shared cross-repo wrapper just calls setup_env.sh --upgrade-tools. - Adopt the deferred-litellm conftest change; reject ignoring paid test files. - LITELLM_LOCAL_MODEL_COST_MAP via .env is P2, and requires moving load_dotenv() to conftest module scope to take effect before collection. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GbnWHq5aFRM3gviE2UBPiE * Add functional spec, architecture, and implementation plan Covers the setup_env.sh rework, the uv >=0.10 floor, the Python 3.13 pin via a generated .python-version, agent config restoration, and the deferred litellm import in the root conftest. Records the environment-side changes that no repo code can make, and drops the LITELLM_LOCAL_MODEL_COST_MAP work. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GbnWHq5aFRM3gviE2UBPiE * Mark UV_SYSTEM_CERTS env change as done Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GbnWHq5aFRM3gviE2UBPiE * Make the repo work in cloud sandboxes: setup scripts, uv floor, faster tests Enforce a uv floor, build environments on Python 3.13, split environment setup from per-session startup, and stop paying a litellm import on every pytest invocation. pyproject.toml gains `required-version = ">=0.10"`. Older uv cannot parse the repo's relative `exclude-newer`; rather than failing it ignores the setting, re-resolves the whole graph and rewrites uv.lock with a broken dependency set. This makes that fail loudly instead. setup_env.sh is reworked into the single environment build. It is non-interactive by default, gates on the uv version, installs Python 3.13 and pins it via a gitignored .python-version (uv-managed CPython bundles Tk, which fixes tkinter for the test modules and both OpenAPI schema scripts), runs `uv sync --frozen --all-packages` and `npm ci` in parallel, and writes agent configs. A delimited CONFIGURATION block at the top holds the defaults, so the file can be pasted verbatim into a Claude Code cloud environment's setup script with only that block edited. --best-effort suppresses the exit code, which that context requires: a setup script exiting non-zero stops the session starting. setup_startup.sh is new, and is what AGENTS.md tells agents to run before their first build or test. A cloud setup script runs once per environment and is then snapshotted and skipped, so nothing repo-aware runs per session and a session can land on a stale VM. It verifies the hard dependencies the environment was meant to provide, printing the repair command when they are missing, then tops up uv sync and npm install for the current branch. The uv check runs before the sync, since a too-old uv is exactly what would corrupt uv.lock during it. It uses npm install rather than npm ci because the working directory is on the snapshotted filesystem and npm ci would discard the cached node_modules. conftest.py no longer imports litellm at module scope. pytest imports the root conftest on every invocation, so every run paid for it even though most test modules never touch litellm. The autouse fixture now looks litellm up in sys.modules and does its setup and teardown only when something has already imported it, which cannot skip a needed cache flush: if litellm was never imported there are no cached clients. A single test file goes from ~7.4s to ~0.9s. Full suite counts are unchanged at 6369 passed, 10020 skipped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GbnWHq5aFRM3gviE2UBPiE * Harden cloud sandbox setup: root discovery, per-session startup, phase plans Follow-up to 5da1958, which implemented this project without going through review. This is the reviewed pass: it verifies each feature against the spec by executing it, fixes what diverged, closes the one item that was blocked on upstream, and adds the phase plans the process requires. The central fix is project root discovery. Both setup scripts derived their root from ${BASH_SOURCE[0]}, which is only correct when the file runs from inside a checkout. setup_env.sh is meant to be pasted verbatim into a Claude Code cloud environment's setup script field, where it is not in a checkout and where, since that script runs once per environment and is then snapshotted and shared across repos, there is frequently no checkout at all. The pasted copy wrote a stray .python-version into an unrelated directory and then reported a run of errors for work that had no target; fed on stdin it hit an unbound BASH_SOURCE, and as root created /.python-version. Both scripts now validate candidate roots (pyproject.toml + libs/core/kiln_ai) from the script location, git rev-parse, and a walk up from $PWD. setup_env.sh treats a miss as a normal outcome — it does the environment-level work worth snapshotting and skips the rest with one notice — while setup_startup.sh, which cannot do anything useful without a checkout, treats it as fatal. That reduction in what setup_env.sh can promise is why setup_startup.sh now also writes agent configuration. It is the only component that reliably has a checkout in hand, so without it CLAUDE.md, .claude/skills/ and .mcp.json never exist in a cloud session and success criterion 7 is unreachable there. It runs first, ahead of the pin and sync, because it is offline, sub-second, and is what makes the repo's own instructions readable. setup_startup.sh also writes .python-version before syncing, which is what gives the tkinter fix a path in a fresh sandbox rather than a round trip through its own failure message. Both scripts re-exec under bash when started by a POSIX shell, since pipefail would otherwise kill them before --best-effort could suppress the exit code — in a cloud setup script, a non-zero exit stops the session from starting. Failure handling is tightened throughout: uv absence is separated from uv being too old, an upgrade is followed by hash -r and a version re-read so a shadowed binary is not reported as success, the --human block routes failures through fail now that set -e is gone, sync failures name the real cause, and setup_startup.sh's repair lines are per-reason rather than one line that cannot work for two of them. Neither script resets a .python-version pin at or above the floor, including the patch-versioned form pyenv writes. setup_startup.sh uses npm install rather than npm ci to keep the cached node_modules that make it cheap, which means it can rewrite the tracked package-lock.json. Rather than give up the incremental behavior, it hashes the file around the install and reports a change, before the failure guards so an install that rewrote the lock and then failed still says so. The functional spec and success criterion 8 name the exception instead of leaving the "mutates no tracked file" claim standing unqualified. .agents/mcp.json gains the hooks-mcp>=0.2.5 floor that was blocked on upstream. A floor, not a pin: uvx reuses cached tool environments and could otherwise keep serving the broken 0.2.4 + mcp 2.0 pair. Verified by driving the configured command over stdio. conftest.py's fixture is renamed _litellm_per_test_setup, since it no longer only clears httpx clients and has no teardown, and its comment now names both cases the sys.modules guard leaves unconfigured. The architecture's open verification item is closed: the model-calls log still lands in ~/.kiln_ai/logs, because get_log_file_path reads Config.settings_dir(), not the settings_path the autouse fixture patches. The ordering dependency this introduced is recorded rather than glossed. Documentation now covers both stated audiences: CONTRIBUTING.md gets the uv floor and its repair command, the Python pin, and the pyenv interaction, so a contributor on uv 0.9 is not walked into a bare required-version refusal. Two things are deliberately left open and tracked rather than quietly resolved. The bootstrap is circular — the only instruction to run setup_startup.sh lives in AGENTS.md, which the repo copies to CLAUDE.md precisely because Claude Code reads CLAUDE.md, so in a fresh sandbox nothing tells the agent to run the script that creates it. That is stated as a known limitation in F5, criterion 7 names the precondition it depends on, and the implementation plan carries checklist items for the operator-side prompt and for the SessionStart hook decision. Separately, the measured speedup from the conftest change does not reproduce as a single multiple: it ranges from ~2.3x on a warm machine to ~7x on a cold one, so the spec now makes the qualitative claim and lets the tables carry the numbers. Full suite unchanged at 6369 passed, 10020 skipped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GbnWHq5aFRM3gviE2UBPiE * Seed node_modules from a warm VM tree, and gate startup to containers Cache warming before the VM snapshot fixed the Python half of a fresh session outright (Prepared 182 packages in 14.67s -> 3 packages in 428ms) but barely moved the Node half (24s -> 21s): npm copies out of its cache where uv hardlinks, so filling an empty node_modules still costs ~21s. So keep the tree instead of refilling it. setup_env.sh gains WARM_CACHE / --warm-cache, which — with no checkout, the cloud environment-build case — clones Kiln to a throwaway directory, syncs it to warm ~/.cache/uv and ~/.npm, keeps its node_modules outside any repo, and deletes the clone. The flag previously existed only in the copy pasted into the cloud environment dialog, which broke the contract that the pasted script is this file with only its CONFIGURATION block edited. It also writes a .setup_for_kiln_repo_v1 marker recording what this machine was provisioned with; the marker's tree fields describe the tree on disk, carrying a prior commit forward, so a repair re-run cannot make it lie about what every session is seeded from. It is written to a temp file and renamed, so a failed write can never leave a zero-byte marker that reads as "provisioned". setup_startup.sh hardlink-copies that tree when the checkout has no node_modules: 0.54s for 601MB, against ~21s of npm. It stages, unshares the root metadata files, then renames — in that order, because npm rewrites node_modules/.package-lock.json in place through a shared inode, and node_modules must never exist in a fully shared state. Every failure (no marker, no tree, cross-device image, failed unshare) falls back to the plain npm install, with its output capped at three lines and a count. A missing marker also skips the hardlink outright: a tree of unknown provenance is never linked into a checkout. The script now runs only when CLAUDE_CODE_REMOTE or IS_CONTAINERIZED is set, since a local environment is set up once and shared across checkouts. AGENTS.md and CONTRIBUTING.md say so and name what local contributors do instead; the functional spec, architecture and phase plans describe the implemented behavior. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GbnWHq5aFRM3gviE2UBPiE * Give node_modules staging a per-PID name again, and sweep by age Fixes a regression from 396ed2b. That commit moved setup_startup.sh's seed staging to one fixed name so a killed run's leftover could be cleared unconditionally — the cleanup had been living inside a block that is skipped as soon as node_modules exists, so a leaked 601 MB directory was never reclaimed. The fixed name solved that and introduced a worse problem: two runs racing on one staging path corrupt node_modules in 57 of 60 races, silently, each reporting a successful seed and exiting 0. One run's rm -rf unlinks entries out of the other's tree mid-cp, and the survivors get renamed into place; or that rm -rf fails against a directory still being created and the second cp -al copies into it, nesting a whole tree. Reproduced here at 10 of 10. Staging is per-PID again, so no run touches a path that is not its own, and the leak that reintroduces is reclaimed by an age-based sweep of .node_modules.warm.* older than an hour — old enough to be certain nothing is in flight, since a seed takes about half a second. The sweep runs unconditionally, which was the property worth keeping. Verified: 30 races against a 52,037-entry warm tree, correct every time, no staging left behind; a two-hour-old staging directory reclaimed by a run that skipped seeding entirely; a fresh one left alone. The comment and architecture.md both justified the fixed name by saying a glob cleanup would delete a concurrent run's staging anyway. That was false about the code it replaced, which only ever removed its own $$ path. Both now state the real trade-off and the measurement behind it. Also: count seed diagnostics with grep -c '' rather than wc -l, so a single unterminated error line is not counted as zero and dropped by the function whose job is to not be silent; and stop calling mv -T's rename "unconditional" — it refuses a non-empty target, which is the point. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GbnWHq5aFRM3gviE2UBPiE * Sweep legacy staging too, and state what actually makes it safe Two fixes to the staging sweep in setup_startup.sh. The -name pattern had a dot before the wildcard, so it matched .node_modules.warm.<pid> but not the fixed .node_modules.warm name 396ed2b briefly used. A run of that revision killed mid-seed leaves up to 601 MB orphaned permanently, and invisibly, since the name is gitignored. Dropping the dot matches both; the leading dot is what keeps the pattern off node_modules itself, verified both ways. The comment and architecture.md both justified the sweep with "anything older than an hour cannot be in flight, since a seed takes about half a second". That is false. cp -al implies --preserve=all, which stamps the warm tree's mtime onto the staging directory the moment the copy completes: measured going from "now" during the fill to five days old the instant it finished, and on a snapshot-restored VM that inherited date is the image build time. So a complete, live staging directory can read as sweepable. What actually makes deleting a foreign staging path safe is that in every supported configuration there is no second process — the sweep runs before this run creates its own staging, and a concurrent run is already unsupported and already destructive at the npm install. The age filter's real job is the narrower one it can do: not deleting a directory another run is still filling, whose mtime does bump per top-level entry. Both places now say that. Behavior verified: a two-hour-old .node_modules.warm and .node_modules.warm.777 are both reclaimed, a fresh sibling is left alone, node_modules is untouched at 52,037 entries, 10 concurrent races stay clean, and a single seed still shares inodes with the warm tree. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GbnWHq5aFRM3gviE2UBPiE * Phase 6: run setup_startup.sh from a SessionStart hook Closes the project's last known limitation: nothing told an unattended cloud agent to run setup_startup.sh, because the instruction lives in AGENTS.md -> CLAUDE.md and CLAUDE.md is written by that script. setup_env.sh gains CREATE_STARTUP_SCRIPT / --create-startup-script (default false, so a local run never edits a contributor's Claude Code settings). It writes a shim into $VM_SETUP_DIR and merges a SessionStart entry for it into the user-level ~/.claude/settings.json, so no repo file and no operator prompt are involved. This retires both the operator-prompt checklist item and the open decision about a tracked .claude/settings.json. The shim finds a Kiln checkout from CLAUDE_PROJECT_DIR upward and exits 0 silently when there is none, since a user-level hook fires for every repo on a shared VM. It re-emits both streams on stdout (only stdout becomes session context, and setup_startup.sh reports failures on stderr), caps that at ~4 KB keeping head and tail, and always exits 0. The merge is idempotent, keyed on the shim's repo-specific basename, refuses to touch a settings.json that is malformed or wrongly typed, preserves unrelated entries and non-ASCII content, and writes atomically. The entry is scoped to startup|resume|fork so compaction does not re-inject its output. The marker's new session_start_hook field reports the registration, not the shim file, so a refused merge reads registration_failed. Verified that this does not displace the environment's own SessionStart hook, whose loss would break commit signing: a live two-hook session ran both, and Claude Code merges hook arrays across settings sources rather than overriding. Also corrects Phase 5's hot-cache figures wherever they appeared. On a snapshot-cold session the first setup_startup.sh costs 22.9 s with the warm node_modules tree and 32.3 s without it, so the tree is worth 9.4 s, not the ~20 s the per-command numbers implied. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GbnWHq5aFRM3gviE2UBPiE * Stop documenting the setup scripts in AGENTS.md The SessionStart hook runs setup_startup.sh before an agent's first turn, so the instruction to run it is dead weight — and AGENTS.md is copied to CLAUDE.md, which loads into every agent context of every session. Twenty-seven lines of setup reference, for a case that is now automated where it matters and self-signposting where it is not: the scripts print their own repair commands, --help carries the flags, and a failed sync names uv lock itself. One line survives, moved to General Agent Guidance, because nothing else says it: CLAUDE.md is generated and overwritten, so edits belong in AGENTS.md. CONTRIBUTING.md goes back to what it was, plus a pointer to the setup script and its --help. The uv floor, the Python pin and the pyenv interaction were written for a reader who has not hit them yet; the tools explain themselves at the point of failure. Three script comments cited AGENTS.md for things it no longer says, and are rewritten to stand on their own. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GbnWHq5aFRM3gviE2UBPiE * Reword the two spec lines that tripped the debug detector The debug detector greps the whole repo for the task-marker keyword, with only AGENTS.md and code_review_guidelines.md excluded. The two lines it flagged were the ones documenting that this detector fires on prose in spec markdown — the only occurrences in the repo were the sentences describing the problem. Reworded to name the behavior without using the keyword, and the functional spec now says explicitly that writing it down is itself enough to fail the check, so the next person to document it does not reintroduce the failure. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GbnWHq5aFRM3gviE2UBPiE * Drop the out-of-scope notes that tripped the debug detector Reworded in baedbb9; removing them outright instead. They recorded two things deliberately left alone, which is history rather than design, and the specs are a point-in-time record — not worth carrying a phrasing constraint in perpetuity so that a note about a CI check does not fail that check. Decision 7 goes entirely and the rest renumber; nothing referenced them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GbnWHq5aFRM3gviE2UBPiE * Make Playwright work in cloud sandboxes: e2e browser and playwright-cli The e2e suite could not launch a browser in a Claude Code cloud sandbox. The image ships a Chromium, but not the revision app/web_ui's @playwright/test pins, and Playwright reports that as a browser that was never installed rather than as the version mismatch it is. Rather than override the browser path, install the revision Playwright already looks for, so sandbox results match CI and nothing has to be re-pointed when @playwright/test is bumped. No revision is hardcoded anywhere: it is read from the checkout's own playwright. - setup_env.sh gains --add-playwright (and a matching ADD_PLAYWRIGHT entry in the CONFIGURATION block) to install that browser, plus playwright-cli and its agent skill. Off by default: together they are ~800 MB. Downloads route through `fail`, so --best-effort still leaves a VM that boots. --with-deps falls back to a plain install when apt is blocked. - setup_startup.sh checks per session that the pinned browser is present and names the revision. Pure bash, so it adds no measurable time. - .agents/scripts/dev_server.sh starts a server for playwright-cli to drive, on ports that do not collide with the suite's, with an isolated HOME. - .playwright/cli.config.json points playwright-cli at the same Chromium the suite uses, on every platform, instead of a branded Chrome. - .agents/USING_PLAYWRIGHT.md covers running the suite and driving the UI, and what the cloud environment needs. Verified in a sandbox: 197 passed, 3 skipped (pre-existing test.fixme), and one pre-existing cold-compile flake in act/discover/data-generation.spec.ts. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G6zhzLXPYXbAjLfTLKXhW5 * Generate playwright-cli's config instead of checking it in Which browser playwright-cli launches is a property of the device it runs on, not of the repo, so the config does not belong in git — and it was also a new top-level folder in the tree. .playwright/ is now gitignored, alongside the .playwright-cli/ artifacts it sits next to. setup_env.sh --add-playwright writes the file when it has a checkout, and setup_startup.sh writes it per session, which is the one that matters in a cloud sandbox: setup_env.sh runs once at VM build with no checkout in reach, and every session starts from a fresh one. Both write only when the file is absent, so a contributor's own edits to it survive. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G6zhzLXPYXbAjLfTLKXhW5 * Add setup_cloud.sh, so the cloud setup script stops being a pasted copy The environment's setup script field held the whole contents of setup_env.sh, which meant every change to that file needed a manual re-paste and went stale silently until someone noticed a VM missing something. setup_cloud.sh replaces it: it fetches setup_env.sh from main and runs it with the cloud profile, so it is small, its contents almost never change, and pasting it once is the last manual step. KILN_SETUP_REF overrides the ref to try a branch's setup_env.sh without editing the file. It never exits non-zero — a cloud setup script that fails stops the session from starting — and it checks that what came back is a shell script, so a proxy error page produces one clear line instead of pages of syntax errors. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G6zhzLXPYXbAjLfTLKXhW5 * Use playwright-cli's global config, and document the blank-screenshot trap An agent driving playwright-cli in a sandbox hit the Chrome-not-found error this was supposed to have fixed. The cause was where the config lived: a per-checkout .playwright/cli.config.json can only be written by something running inside a session, so a session on a branch without that code got playwright-cli with no config at all. It was also resolved against the process's cwd, so it was missed from app/web_ui. playwright-cli reads ~/.playwright/cli.config.json as a global config, which fixes both: it needs no checkout, so setup_env.sh writes it at VM-build time and it lands in the environment snapshot, and it applies from any directory. That also matches what the file is — a record of which browser is installed on this machine. Docs, from the same session's feedback: - A screenshot taken right after open/goto is very often blank. The DOM is complete and readyState is "complete" at that point, so a snapshot is not a usable gate; Chromium simply has not painted. That matters because a blank frame is the usual signal an app failed to start, so it leads to reporting a working app as broken. Documented waitForLoadState('networkidle'), which also covers the API fetches Kiln's pages make after hydrating. - Use find to discover an element's real role before writing a role locator. Kiln styles links as buttons, so getByRole("button") misses them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G6zhzLXPYXbAjLfTLKXhW5 * Rename both scripts to match what they actually do, trim install docs dev_server.sh -> playwright_server.sh: it is not a general dev server. It runs on ports the e2e suite does not use, with its own isolated HOME, for the single purpose of giving playwright-cli something to drive. setup_cloud.sh -> claude_code_vm_setup.sh: it is not generic cloud setup. It targets a Claude Code cloud environment's setup script field specifically. USING_PLAYWRIGHT.md loses the install and cloud-environment sections. This file is read to drive a browser, and installation is not what a reader needs 99.9% of the time. What is left is a pointer to setup_env.sh --add-playwright. The material moved to where each reader already is: the egress allowlist is now in claude_code_vm_setup.sh's header, next to the fetch it governs, and the rest is in setup_env.sh's --help, which no longer points back at the doc that points at it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G6zhzLXPYXbAjLfTLKXhW5 * Stop setup_startup.sh reporting an installed browser as missing The session-start check told an agent chromium-1223 was missing when it was sitting in /opt/pw-browsers, and named the install command as the fix. The agent believed it and went off to reinstall. A check that is wrong this way is worse than no check: it costs more than the problem it was added to catch. Two causes, both now handled. PLAYWRIGHT_BROWSERS_PATH comes from the image environment and a hook process does not always inherit it. Without it the check read the default cache path instead of the real one. So the browser half is skipped entirely unless its directory can actually be seen from here, and the complaint is left to Playwright at the point of use. The playwright-cli half still catches "nothing was installed", and depends only on PATH. Testing that fix showed the obvious guard is not enough: playwright-cli creates ~/.cache/ms-playwright for its own session state, so the default path exists on a machine whose browsers are in /opt, and the check still fired. A directory now counts as the browsers root only if it already holds a chromium build — which is also exactly the state the check exists to catch, an image shipping one revision where the repo pins another. Verified: correct browser silent; missing env with a stateful-but-browserless cache silent; a real root holding only 1194 still reports 1223; a missing playwright-cli still reported. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G6zhzLXPYXbAjLfTLKXhW5 * Install the e2e browser where Playwright will not garbage-collect it --add-playwright left a cloud VM with playwright-cli's browser and not the one the e2e suite needs, so the session-start check correctly reported the pinned revision missing and the suite could not run. The install was working and then undoing itself. `playwright install` records the installation that asked for a browser in a .links registry beside the browsers, and every later install deletes browsers no surviving registration references. The repo browser was installed from the throwaway clone, whose node_modules is then moved and whose directory is deleted, which invalidates that registration — and the very next install in the same run, playwright-cli's for its own browser, collected the revision that had just been downloaded. The build log says so plainly once you look for it: "Removing unused browser at .../chromium-1223" a few lines after downloading it. Installing from the warm tree after the move instead registers the tree's own path, which lives outside any checkout and survives for the life of the machine. Verified against an empty browsers directory: the run now ends with both revisions and collects nothing. install_playwright_repo_browser's argument is no longer specifically an app/web_ui, so it is named for what it needs and documents why the caller's choice of directory is load-bearing rather than incidental. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G6zhzLXPYXbAjLfTLKXhW5 * Document setting up a Claude Code cloud sandbox for this repo The setup was three artifacts and no instructions: claude_code_vm_setup.sh says paste me, and nothing said what else the environment needs around it. The allowlist and the UV_SYSTEM_CERTS variable lived in a script header and a spec file respectively, neither of which is where someone creating an environment is looking. What the guide adds beyond the fields: what a working first turn looks like, so a broken environment is recognizable rather than just slow; that npm install can leave package-lock.json modified; and that KILN_SETUP_REF belongs in the pasted script, not the environment variables, since only editing the script rebuilds the snapshot. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UZiY1pWyUBjTi5m9kLFDH * Add Qwen 3.8 2.4T A95B to model list (openrouter, fireworks_ai, together_ai) (#1670) Co-authored-by: Claude <noreply@anthropic.com> * Add Gemini 3.7 Flash to model list (#1671) * Add Gemini 3.7 Flash to model list (openrouter, gemini_api, vertex) * Set multimodal_requires_pdf_as_image on Gemini 3.7 Flash OpenRouter provider * Exclude MP3/OGG from Gemini 3.7 Flash vertex provider Vertex's content filter stochastically blocks this model's transcriptions of compressed audio (finish_reason content_filter, no text returned): mp3 ~90% and ogg ~50% of requests in repeated runs. WAV is unaffected, as are all audio types on gemini_api, and Gemini 3.6 Flash on vertex. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Leonard Q. Marcq <marcqleonard@gmail.com> Co-authored-by: Leonard Q. Marcq <leonard@getkiln.ai> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: scosman <scosman@users.noreply.github.com> Co-authored-by: chiang-daniel <16633065+chiang-daniel@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Adds five new Qwen models on OpenRouter and repoints DeepSeek V4 Flash at its new
0731checkpoint. Each was verified against real inference.json_instruction_and_object.json_instructions.reasoning_capableleftFalse(see below).deepseek/deepseek-v4-flash→deepseek/deepseek-v4-flash-0731, keepingjson_schemaand the existing thinking levels. Fireworks provider unchanged.Test Results
Every provider slug was verified with an actual inference call, not just the catalog listing, and the catalogs overstated availability. That's why the new Qwen models ship OpenRouter-only: Fireworks lists the Qwen slugs but the API 404s (not deployed — the known Fireworks-lag quirk); Together hosts the Plus variants as streaming-only (unusable in Kiln, same as the existing Qwen 3.7 Max note) and the 35B FP8 as dedicated-only; Featherless serves the small Qwen deployments but they ignore JSON-instruction prompting and return reasoning prose (
Qwen/Qwen3.5-9Bfailed 8/15 structured tests,Qwen/Qwen3.6-*returned empty), so all were dropped. Qwen 3.6 27B was dropped entirely — it returns empty/degenerate output on every provider.Two config notes. Qwen 3 Max Thinking first failed
RuntimeError: Reasoning is required for this model, but no reasoning was returned— it uses adaptive reasoning and does not always emit a reasoning block, soreasoning_capable=False(reasoning is still parsed/displayed when present); it passed cleanly afterward. Qwen 3.5 9B on OpenRouter flaked once ontest_data_gen_sample_all_models_providersunder parallel load (appended extra keys to structured output) but passed that test on three isolated reruns — a content-quality flake, not a config issue. DeepSeek V4 Flash 0731 had no live OpenRouter endpoints at the start of this work and came online partway through; its first parallel test run hit transient HTTP 429 "temporarily rate-limited upstream" on the just-launched shared pool, but a serial re-run once capacity ramped passed all 11 OpenRouter tests cleanly, so it ships green.Qwen 3.7 Flash (openrouter): 19 passed, 0 failed
Qwen 3.6 Flash (openrouter): 19 passed, 0 failed
Qwen 3.6 35B A3B (openrouter): 19 passed, 0 failed
Qwen 3.5 9B (openrouter): 18 passed, 1 flake (data-gen sample; passed on retry)
Qwen 3 Max Thinking (openrouter): 11 passed, 0 failed (text-only)
DeepSeek V4 Flash (fireworks_ai): full suite passed
DeepSeek V4 Flash (openrouter, 0731): 11 passed, 0 failed (initial parallel run hit transient upstream 429s; all passed on a serial re-run)
Qwen 3.7 Flash (openrouter):
✅ test_data_gen_all_models_providers[qwen_3p7_flash-openrouter]
✅ test_data_gen_sample_all_models_providers[qwen_3p7_flash-openrouter]
✅ test_data_gen_sample_all_models_providers_with_structured_output[qwen_3p7_flash-openrouter]
✅ test_all_built_in_models_llm_as_judge[qwen_3p7_flash-openrouter]
✅ test_all_built_in_models_structured_output[qwen_3p7_flash-openrouter]
✅ test_all_built_in_models_structured_input[qwen_3p7_flash-openrouter]
✅ test_structured_output_cot_prompt_builder[qwen_3p7_flash-openrouter]
✅ test_structured_input_cot_prompt_builder[qwen_3p7_flash-openrouter]
✅ test_all_models_providers_plaintext[qwen_3p7_flash-openrouter]
✅ test_cot_prompt_builder[qwen_3p7_flash-openrouter]
✅ test_tools_all_built_in_models[qwen_3p7_flash-openrouter]
✅ test_supports_vision_is_coherent[qwen_3p7_flash-openrouter]
✅ test_provider_bad_request[qwen_3p7_flash-openrouter]
✅ test_extract_document_success[...-qwen_3p7_flash-openrouter] (6/6 MIME probes)
Qwen 3.6 Flash (openrouter):
✅ test_data_gen_all_models_providers[qwen_3p6_flash-openrouter]
✅ test_data_gen_sample_all_models_providers[qwen_3p6_flash-openrouter]
✅ test_data_gen_sample_all_models_providers_with_structured_output[qwen_3p6_flash-openrouter]
✅ test_all_built_in_models_llm_as_judge[qwen_3p6_flash-openrouter]
✅ test_all_built_in_models_structured_output[qwen_3p6_flash-openrouter]
✅ test_all_built_in_models_structured_input[qwen_3p6_flash-openrouter]
✅ test_structured_output_cot_prompt_builder[qwen_3p6_flash-openrouter]
✅ test_structured_input_cot_prompt_builder[qwen_3p6_flash-openrouter]
✅ test_all_models_providers_plaintext[qwen_3p6_flash-openrouter]
✅ test_cot_prompt_builder[qwen_3p6_flash-openrouter]
✅ test_tools_all_built_in_models[qwen_3p6_flash-openrouter]
✅ test_supports_vision_is_coherent[qwen_3p6_flash-openrouter]
✅ test_provider_bad_request[qwen_3p6_flash-openrouter]
✅ test_extract_document_success[...-qwen_3p6_flash-openrouter] (6/6 MIME probes)
Qwen 3.6 35B A3B (openrouter):
✅ test_data_gen_all_models_providers[qwen_3p6_35b_a3b-openrouter]
✅ test_data_gen_sample_all_models_providers[qwen_3p6_35b_a3b-openrouter]
✅ test_data_gen_sample_all_models_providers_with_structured_output[qwen_3p6_35b_a3b-openrouter]
✅ test_all_built_in_models_llm_as_judge[qwen_3p6_35b_a3b-openrouter]
✅ test_all_built_in_models_structured_output[qwen_3p6_35b_a3b-openrouter]
✅ test_all_built_in_models_structured_input[qwen_3p6_35b_a3b-openrouter]
✅ test_structured_output_cot_prompt_builder[qwen_3p6_35b_a3b-openrouter]
✅ test_structured_input_cot_prompt_builder[qwen_3p6_35b_a3b-openrouter]
✅ test_all_models_providers_plaintext[qwen_3p6_35b_a3b-openrouter]
✅ test_cot_prompt_builder[qwen_3p6_35b_a3b-openrouter]
✅ test_tools_all_built_in_models[qwen_3p6_35b_a3b-openrouter]
✅ test_supports_vision_is_coherent[qwen_3p6_35b_a3b-openrouter]
✅ test_provider_bad_request[qwen_3p6_35b_a3b-openrouter]
✅ test_extract_document_success[...-qwen_3p6_35b_a3b-openrouter] (6/6 MIME probes)
Qwen 3.5 9B (openrouter):
⚠️ test_data_gen_sample_all_models_providers[qwen_3p5_9b-openrouter] — small model added extra keys under parallel load; passed on 3 isolated reruns
✅ test_data_gen_all_models_providers[qwen_3p5_9b-openrouter]
✅ test_data_gen_sample_all_models_providers_with_structured_output[qwen_3p5_9b-openrouter]
✅ test_all_built_in_models_llm_as_judge[qwen_3p5_9b-openrouter]
✅ test_all_built_in_models_structured_output[qwen_3p5_9b-openrouter]
✅ test_all_built_in_models_structured_input[qwen_3p5_9b-openrouter]
✅ test_structured_output_cot_prompt_builder[qwen_3p5_9b-openrouter]
✅ test_structured_input_cot_prompt_builder[qwen_3p5_9b-openrouter]
✅ test_all_models_providers_plaintext[qwen_3p5_9b-openrouter]
✅ test_cot_prompt_builder[qwen_3p5_9b-openrouter]
✅ test_tools_all_built_in_models[qwen_3p5_9b-openrouter]
✅ test_supports_vision_is_coherent[qwen_3p5_9b-openrouter]
✅ test_provider_bad_request[qwen_3p5_9b-openrouter]
✅ test_extract_document_success[...-qwen_3p5_9b-openrouter] (6/6 MIME probes)
Qwen 3 Max Thinking (openrouter):
✅ test_data_gen_all_models_providers[qwen_3_max_thinking-openrouter]
✅ test_data_gen_sample_all_models_providers[qwen_3_max_thinking-openrouter]
✅ test_data_gen_sample_all_models_providers_with_structured_output[qwen_3_max_thinking-openrouter]
✅ test_all_built_in_models_llm_as_judge[qwen_3_max_thinking-openrouter]
✅ test_all_built_in_models_structured_output[qwen_3_max_thinking-openrouter]
✅ test_all_built_in_models_structured_input[qwen_3_max_thinking-openrouter]
✅ test_structured_output_cot_prompt_builder[qwen_3_max_thinking-openrouter]
✅ test_structured_input_cot_prompt_builder[qwen_3_max_thinking-openrouter]
✅ test_all_models_providers_plaintext[qwen_3_max_thinking-openrouter]
✅ test_cot_prompt_builder[qwen_3_max_thinking-openrouter]
✅ test_tools_all_built_in_models[qwen_3_max_thinking-openrouter]
DeepSeek V4 Flash (fireworks_ai — unchanged, re-confirmed):
✅ test_data_gen_all_models_providers[deepseek_4_flash-fireworks_ai]
✅ test_data_gen_sample_all_models_providers[deepseek_4_flash-fireworks_ai]
✅ test_data_gen_sample_all_models_providers_with_structured_output[deepseek_4_flash-fireworks_ai]
✅ test_all_built_in_models_llm_as_judge[deepseek_4_flash-fireworks_ai]
✅ test_all_built_in_models_structured_output[deepseek_4_flash-fireworks_ai]
✅ test_all_built_in_models_structured_input[deepseek_4_flash-fireworks_ai]
✅ test_structured_output_cot_prompt_builder[deepseek_4_flash-fireworks_ai]
✅ test_structured_input_cot_prompt_builder[deepseek_4_flash-fireworks_ai]
✅ test_tools_all_built_in_models[deepseek_4_flash-fireworks_ai]
DeepSeek V4 Flash (openrouter — 0731 checkpoint, serial re-run):
✅ test_data_gen_all_models_providers[deepseek_4_flash-openrouter]
✅ test_data_gen_sample_all_models_providers[deepseek_4_flash-openrouter]
✅ test_data_gen_sample_all_models_providers_with_structured_output[deepseek_4_flash-openrouter]
✅ test_all_built_in_models_llm_as_judge[deepseek_4_flash-openrouter]
✅ test_all_built_in_models_structured_output[deepseek_4_flash-openrouter]
✅ test_all_built_in_models_structured_input[deepseek_4_flash-openrouter]
✅ test_structured_output_cot_prompt_builder[deepseek_4_flash-openrouter]
✅ test_structured_input_cot_prompt_builder[deepseek_4_flash-openrouter]
✅ test_all_models_providers_plaintext[deepseek_4_flash-openrouter]
✅ test_cot_prompt_builder[deepseek_4_flash-openrouter]
✅ test_tools_all_built_in_models[deepseek_4_flash-openrouter]
Related Issues
N/A
Contributor License Agreement
I, @tawnymanticore, confirm that I have read and agree to the Contributors License Agreement.
Checklists
🤖 Generated with Claude Code