Repository navigation
thinking: note thinking models and add a profile switch - #62
Merged
Merged
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved thinking-setting bugs can cause a panic on null kwargs, mishandle repeated kwargs flags, and show controls or state that do not match the selected or active profile.
6 open issues
What changed in this PR
This PR addresses #24 by making reasoning-related latency visible before a model is pulled and adding a default thinking setting for new profiles.
Changes:
- Adds catalogue-driven thinking detection and an engine capability for setting chat-template defaults.
- Exposes thinking information through CLI commands, profile metadata, JSON output, and the panel’s Add model flow.
- Adds validation and regression tests, updates documentation, and records the feature in the changelog.
| File | Description |
|---|---|
internal/web/static/models.js |
Renders thinking controls and submits the selected setting. |
internal/web/static/models.html |
Adds the Thinking control markup. |
internal/web/static/app.css |
Styles the thinking notice. |
internal/web/server_internal_test.go |
Tests invalid settings and existing-profile conflicts. |
internal/web/models.go |
Exposes switch availability and handles thinking requests. |
internal/vllm/validate.go |
Removes quotation marks in a parser comment; no behavior change. |
internal/resolve/thinking.go |
Detects thinking support and applies profile overrides. |
internal/resolve/thinking_internal_test.go |
Tests detection, overrides, validation, and argument persistence. |
internal/resolve/resolve.go |
Integrates thinking into inspection and profile preparation. |
internal/profile/profile.go |
Reads and writes THINKING metadata. |
internal/profile/profile_test.go |
Tests thinking metadata persistence. |
internal/modeltype/modeltype.go |
Defines and validates catalogue thinking entries. |
internal/modeltype/modeltype_internal_test.go |
Tests thinking schema and shipped entries. |
internal/hf/hf.go |
Reads chat templates from template files or tokenizer configuration. |
internal/hf/hf_internal_test.go |
Tests chat-template retrieval fallbacks. |
internal/engine/engine.go |
Adds and validates the chat-template kwargs capability. |
internal/engine/engine_test.go |
Tests capability flag validation. |
internal/cli/search.go |
Displays thinking defaults during inspection. |
internal/cli/run_test.go |
Tests CLI reporting and invalid thinking values. |
internal/cli/profiles.go |
Adds new --thinking and list/status reporting. |
internal/cli/modeltypescmd.go |
Displays thinking rules and handles thinking-only entries. |
internal/cli/editset.go |
Recognizes THINKING as a standard profile key. |
docs/reference/cli/new.md |
Documents the new CLI flag. |
docs/providers.md |
Documents the engine kwargs capability. |
docs/profiles.md |
Explains thinking metadata and manual profile updates. |
docs/panel.md |
Describes the panel’s Thinking control. |
docs/models.md |
Explains thinking latency and disabling it. |
docs/model-types.md |
Documents thinking schema, detection, and overrides. |
CHANGELOG.md |
Announces thinking detection and profile controls. |
catalogue/model-types/smollm3.json |
Adds SmolLM3 thinking metadata. |
catalogue/model-types/README.md |
Introduces thinking-only catalogue entries. |
catalogue/model-types/qwen3.json |
Adds Qwen3 thinking metadata. |
catalogue/model-types/qwen3_moe.json |
Adds Qwen3 MoE thinking metadata. |
catalogue/model-types/qwen3_5.json |
Adds Qwen3.5 thinking metadata. |
catalogue/model-types/qwen3_5_moe.json |
Adds Qwen3.5 MoE thinking metadata. |
catalogue/model-types/glm4v_moe.json |
Adds GLM-4.5V thinking metadata. |
catalogue/model-types/glm4_moe.json |
Adds GLM-4.5–4.7 thinking metadata. |
catalogue/model-types/glm4_moe_lite.json |
Adds GLM-4.7-Flash thinking metadata. |
catalogue/model-types/glm_moe_dsa.json |
Adds GLM-5 thinking metadata. |
catalogue/model-types/deepseek_v4.json |
Adds always-detected, default-on thinking metadata. |
catalogue/model-types/deepseek_v3.json |
Adds template-detected, default-off thinking metadata. |
catalogue/engines/vllm.json |
Declares vLLM’s default chat-template kwargs flag. |
catalogue/engines/README.md |
Documents the new engine capability. |
Review effort: Balanced
Comment on lines
+448
to
+449
| if p, err := profile.Load(cfg.Dirs, r.Profile); err == nil { | ||
| r.Thinking = p.Thinking() |
Comment on lines
+345
to
+349
| if flag != "" { | ||
| arch, _ := hf.Config(ctx, cfgRepo) | ||
| p.Thinking = detectThinking(ctx, arch.ModelType, repo, cfgRepo) | ||
| p.noteThinking(flag) | ||
| } |
Comment on lines
+69
to
+73
| var m map[string]any | ||
| if json.Unmarshal([]byte(val), &m) == nil { | ||
| if b, isBool := m[kwarg].(bool); isBool { | ||
| def = b | ||
| } |
Comment on lines
+99
to
+101
| if err := json.Unmarshal([]byte(val), &kwargs); err != nil { | ||
| return "", fmt.Errorf("%s %s is not a JSON object", flag, val) | ||
| } |
Comment on lines
+406
to
+407
| const offer = d.thinking_switch && !d.existing_profile; | ||
| el.thinkingSelect.hidden = !offer; |
Comment on lines
+402
to
+410
| const def = d.thinking.default ? "on" : "off"; | ||
| el.thinkingNote.textContent = d.thinking.default | ||
| ? `Thinking model: replies begin after a reasoning pass. ${d.thinking.why}` | ||
| : `This model can reason before it answers, but does not unless asked. ${d.thinking.why}`; | ||
| const offer = d.thinking_switch && !d.existing_profile; | ||
| el.thinkingSelect.hidden = !offer; | ||
| if (!offer) return; | ||
| for (const [value, label] of [ | ||
| ["", `Model default (${def})`], |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #24
Thinking models now say so before they are pulled. A profile can also turn thinking off by default.
What changed
thinkingblock withkwarg,default,whyand an optionaldetect(templatechecks the chat template for the kwarg,alwaysskips the check). A file needsengines,thinking, or both. Entries cover:qwen3,qwen3_moe,qwen3_5,qwen3_5_moeglm4_moe,glm4_moe_lite,glm_moe_dsa,glm4v_moesmollm3deepseek_v3(off by default)deepseek_v4(detect: always, because the tokenizer mode starts the thinking block)chat_template_kwargs_flagcapability, set to--default-chat-template-kwargsinvllm.json. No Go code names an engine or model.inspectprintsthinking: on by default …oroff by default ….new --thinking on|offwrites the kwarg through the engine flag and recordsTHINKING=in the profile. A bad value is refused before anything is resolved.listshows[thinking], andstatusgets a thinking line. Both are in the JSON output too.model-types listandshowprint the thinking entry.?thinking=on pull returns 400 on a bad value and 409 when the profile already exists.Existing profiles have no
THINKINGkey until they are re-created. Always-thinking models with no switch (R1, QwQ, Kimi-K2-Thinking) are not flagged.Checks
go test -race ./...: all pass exceptinternal/detect TestProbeReportsThisMachine, which fails on a 16 GB workspace and also fails onmain.go vet ./...,gofmt -l .and the recipediff vet and tests pass.prettier --checkpasses.inspect Qwen/Qwen3-0.6Bandinspect zai-org/GLM-4.5-Airboth reportthinking: on by default.