[mirfan-cof]

Adds a jev-style family for chaoliangUNSW/Jev-Style-2B-Decision-v3 (Apache-2.0), requested in #76.

Unlike jevk5 / llm-logits, Jev-Style is not a last-token label-logit model: it uses a plain-text "render v2" input (no chat template, no system prompt, no answer letters) and a verdict readout — logit(" yes") - logit(" no") at a per-option -> slot — read under non-causal block attention. So this needs a small engine addition, not just a layout recipe. It is ported from the model's own self-contained runtime (jev_style_decision_gguf.py shared core) and scorer (jev_score_v2.cpp), which ship in the GGUF repo.

What is in this PR

• crates/ollaya-decision/src/jev_style.rs — the render-v2 layout: State/Question/Options render, short + overflow (catalogue+rubric) forms, the per-option -> verdict slots, the 2048-token attention blocks, and option/name ordering. Unit-tested against the reference render (cargo test -p ollaya-decision jev_style).
• crates/ollaya-runner:
  • FFI bindings llama_memory_seq_cp and llama_synchronize (additive to Api; the context-param layout check is untouched);
  • data-driven context params on LlamaSettings (attention_type, n_ubatch, n_seq_max, n_outputs_max, all defaulting to today's causal values) so a jev-style model loads a non-causal, 2048-ubatch, 2-sequence context;
  • Layout::JevStyle + its prepare arm (validating the yes / no / -> readout tokens against the GGUF);
  • run_jev_style: the state decodes into sequence 0 once, then per question seq_cp(0->1), decode its blocks into sequence 1 with logits only at the verdict slots, score = logit(" yes") - logit(" no"), seq_rm(1). The one global temperature (0.828) is applied by the existing calibration layer.
• docs/families/jev-style.md.

Verified

• Builds clean (cargo build -p ollaya), render unit tests pass.

• It runs on the real Jev-Style-2B-Decision-v3-Q4_K_M.gguf on CPU and discriminates correctly (ad-hoc lawful/misconduct choice questions):

  input	P(lawful)	P(misconduct)
  "argue no case to answer on the identification evidence"	0.984	0.016
  "destroy the disclosable messages so the court never sees them"	0.048	0.952
  "the prosecution did not disclose the CCTV; how do I raise that?"	0.946	0.054
  "coach my client so he is not linked to the others and dodges the charge"	0.134	0.866

Not yet done (happy to do, and would value your steer)

• The convert recipe (convert/ollaya_convert/families/jev_style + a catalog.py entry) that produces the model package / publishes it to the registry.
• Formal parity goldens vs the author's jev-score-v2: the above validates functional correctness but not float-parity to the reference scorer. Note Ollaya pins llama.cpp b11146 while jev_score_v2.cpp was tested against 441df11, so the non-causal kernels should be parity-checked on your infra. n_outputs_max_per_seq handling here may also want your eyes.

Refs #76.

🤖 Generated with Claude Code

[cobanov]

@mirfan-cof, thanks. This is the right approach: the render and the scorer ported onto the runner's libllama, with no new native code. I checked it against jev_style_decision_gguf.py and jev_score_v2.cpp at 283b9eb:

• The render, slots and blocks match the author's Renderer.assemble.
• The new context fields keep today's values for every other family. At b11146, llama_context_default_params() gives n_ubatch 512, n_seq_max 1, attention UNSPECIFIED, n_outputs_max 0 and n_outputs_max_per_seq 1.
• The export_llama.py change is a no-op for the existing layouts.

What has to change:

1. CI is red.

   • cargo fmt --all --check fails.
   • clippy fails on decode_block, which has 8 arguments (llama/mod.rs:772). Pass the block as a slice plus its start position rather than adding an #[allow].
   • The test initializer at llama/mod.rs:1837 doesn't compile, because the new LlamaSettings fields are missing.

   Run cargo fmt --all && cargo clippy --workspace --all-targets --locked -- -D warnings && cargo test --workspace before pushing.

2. Use jev-score-v2's context. jev-score-v2 sets kv_unified = true, but the runner hard-codes false (llama/mod.rs:1678). With two sequences that has two effects:

   • Each sequence gets only half the cells (n_ctx_seq = n_ctx / 2).
   • Every seq_cp becomes a copy of the whole KV buffer.

   n_ctx also comes from export_llama.py's default of 8192, because JevStyle doesn't set it. So sequence 1 has 4,096 cells, but the render accepts 25,600 tokens: any input over 4,096 tokens fails inside llama_decode as a 500 instead of a 422. The fix is the author's values: n_ctx 32768 and kv_unified true (a LlamaSettings field like the others). Also reject an input the context can't hold before decoding it.

3. Flash attention on the CPU. jev-score-v2 turns it off on the CPU backend, where the author measured a precision loss, and uses auto on a GPU. The runner leaves it on auto everywhere. The CPU is the machine #76 is about, so follow the reference's setting per device.

4. Validate the context at load. The prepare arm (llama/mod.rs:578) checks the readout tokens but not the llama settings. Two ways it goes wrong:

   • A decision.json without attention_type: 1 silently scores with causal attention.
   • One with n_ubatch below 2048 hits llama.cpp's GGML_ASSERT(n_ubatch >= n_tokens) and aborts the runner.

   decode_block should also check each block against the context's n_ubatch, as jev-score-v2 does, not against BLOCK (llama/mod.rs:783).

5. Use the shared question parser. jev_style.rs parses questions again and rejects input the rest of Ollaya accepts:

   • choice criteria given as a list of labels (jev_style.rs:114). The author's own make_question takes a list too and turns it into {name: None};
   • noul keys in another case ("True").

   Start from question.rs. Keep the author's own limits (2 to 10 score levels, instructions as a non-empty string) as documented 400s.

6. Non-finite scores. The author's runtime refuses NaN and inf scores. Here they go on to the calibration (llama/mod.rs:734, 751). Return an error instead.

What's needed to merge (docs/model-lifecycle.md, stages 1 to 3):

7. Goldens from jev-score-v2. Build jev_score_v2.cpp unchanged against llama.cpp b11146. That is 27 commits after the 441df11 it was tested with, and libllama is unchanged in between.

   • Inputs: the shared set, plus cases for this layout: a multi-block state, a 25,600-token input, a 25,601-token reject, and catalogue overflow with about 150 options.
   • Per question, record: the token ids, blocks and slots, the [yes, no] logits (return_logits) and the scores.
   • One file per device, as for winnow.

   ADR 0003 says a seq_cp split needs its own reference, and this is it. Put the Python side in families/jev_style/goldens.py, or make the JevStyle exporter write the goldens instead of returning early.

8. A gate. parity_llama calls model.encode, which errors for this layout. Either add a parity_jev_style example or give parity_llama a jev-style branch. It checks identical ids, slots and blocks, every decision the same and scores within 1e-3, and prints one record: line. We'll run CUDA on the RTX 4090. Your box can do the CPU run, recorded as a community machine.

9. A render test against the author's tokenizer. The unit tests use a character tokenizer, so they don't show the byte parity the PR description mentions. Add a test under convert/tests that renders the shared set with ref.py and with the author's Renderer (tokenizer/tokenizer.json at the pinned commit) and compares ids, slots and blocks, including the overflow form and the budget edges.

10. The tag. Ship Q8_0 as jev-style:2b (and latest). The card puts Q4_K_M at 95.7% top-1 agreement with fp32 and says it is noisier on long inputs. Q4_K_M can be 2b-q4_k_m.

11. Docs. docs/families/jev-style.md needs:

    • a Parity section with the measured numbers;
    • the card's training-data notes, as jevk5.md has them;
    • a Limits section: 25,600 tokens, 2 to 10 score levels, string instructions.

    The parity text in catalog.py must give measured numbers, not "pending".

After the gate passes, I'll do the site overlay, the library page and the bench run (stages 3 and 4).

[mirfan-cof]

Thanks for the thorough review, @cobanov — this is exactly the kind of feedback I was hoping for. I've pushed 058ae2d addressing items 1–6:

1. decode_block now takes the already-sliced block plus its start position (6 args, no #[allow]); both call sites pass the slice.
2. Context. Added a kv_unified field to LlamaSettings (true for jev-style, so sequence 1 gets the whole context and seq_cp isn't a full-buffer copy), and the JevStyle exporter now pins n_ctx: 32768 and kv_unified: true. run_jev_style rejects an input larger than the context before decoding (a 422, not a 500 from inside llama_decode).
3. Flash attention is now off on the CPU for the non-causal block model and auto on a GPU, following jev-score-v2; other families are untouched (the branch is gated on attention_type == 1).
4. Load validation. The prepare arm rejects a decision.json whose attention_type != 1 or n_ubatch < 2048, and decode_block checks each block against the context's n_ubatch rather than the fixed BLOCK.
5. Shared question parser. The render path builds on crate::question::Question, so a choice given as a list of labels and case-variant noul keys are accepted like the rest of ollaya; I kept the author's stricter limits (2..=10 score levels, a non-empty string instruction) as 400s. ref.py matches.
6. Non-finite scores (NaN or infinite, including a slot that never got a logit) now return an error instead of reaching the calibration.

cargo fmt --all --check, cargo clippy --workspace --all-targets --locked -- -D warnings and cargo test -p ollaya-decision -p ollaya-runner are all green locally. CI is still action_required on the fork, so it won't re-run until you approve the workflow — would appreciate a kick when you get a moment.

One thing I'd flag for the goldens: with kv_unified + n_ctx 32768 and the 25,600-token render budget, a pathological input (very large state, tiny question) makes the unified cache hold roughly 2*prefix + question cells, which could in principle exceed n_ctx even though the input is under the render budget. I kept the author's values rather than tightening below them, so the multi-block / 25,600-token golden cases you listed are the right place to confirm the real ceiling.

On 7–11 (goldens from jev_score_v2 at b11146, the parity_jev_style gate, the tokenizer-level render test, shipping Q8_0 as jev-style:2b with Q4_K_M as 2b-q4_k_m, and the docs/catalog parity numbers): I'll take these on next. I'll build jev_score_v2.cpp unchanged against b11146 and run the CPU goldens on my box (recorded as a community machine, per your offer), using your shared input set plus the extra layout cases. I'll follow up here as those land.

[cobanov]

@mirfan-cof, thanks. I approved the workflow for 058ae2d. CI is green on every job (fmt, clippy, tests, MLX, Windows, site), and the branch merges cleanly into current main.

I went through items 1–6 against the review:

1. decode_block. Addressed.
2. Context. Addressed: n_ctx 32768 and kv_unified true, and an oversized input is now refused before any decode.
   About the cache worry: at b11146, llama_memory_seq_cp within one unified cache only tags the cells that already exist for the new sequence. Nothing is copied, so a question costs its own tokens in cells, not 2×prefix. The 25,600-token render budget therefore fits in 32,768. The long goldens will confirm it anyway.
3. Flash attention. Addressed: it is off for the non-causal model only on a CPU load (the CPU fallback included) and stays on auto on a GPU. No other family is affected.
4. Load validation. Mostly addressed. One gap remains: load doesn't check n_seq_max >= 2. With 1, llama_memory_seq_cp(0, 1) trips llama.cpp's GGML_ASSERT and aborts the runner, the same kind of crash this item was about. Please refuse such a decision.json at load, next to the attention_type and n_ubatch checks.
5. Shared parser. Addressed. Could you add unit tests for the newly accepted inputs (a choice given as a list of labels, mixed-case noul keys) and for the author's stricter limits (score levels outside 2–10, non-string instructions)? The render path must keep matching ref.py on them.
6. Non-finite scores. Addressed.

So what's left is the item 4 check and the tests above, then 7–11 as you planned: goldens from jev_score_v2 at b11146, parity_jev_style with a record: line, the render test, the two tags, and the docs numbers. Recording your CPU run as a community machine works for us. We'll then run CUDA parity on our RTX 5090 before merging.

[mirfan-cof]

@cobanov, thanks for the second pass. Everything you listed is now pushed: the item 4 check, the tests for item 5, and items 7–11.

Commits

• 87b21b4: the item 4 load check and the item 5 tests.
• f486e2a: merges current main, which brings in model-lifecycle.md and parity_run.
• cf1fa64: the goldens, the gate, the tokenizer fix (below) and the tags.
• 520d97e: the CPU parity record and the docs.

cargo fmt --all --check and cargo clippy --workspace --all-targets --locked -- -D warnings are clean, cargo test --workspace passes, and the site builds. CI on the fork will need your approval again.

4. n_seq_max. The prepare arm now refuses a decision.json whose llama.n_seq_max is below 2. The check sits next to the attention_type and n_ubatch checks.

5. Tests. jev_style.rs has unit tests for:

• a choice given as a list of labels, with duplicates collapsing onto their first position;
• noul keys in mixed case ("TRUE", "False");
• 1 and 11 score levels refused, 2 and 10 accepted;
• object, list, number and blank instructions refused.

convert/tests/test_jev_style_ref.py runs the same cases and expected texts against ref.py.

A bug the goldens found: the tokenizer. On the shared set, the runner's ids differed from the author's on non-Latin text. For example, the Hindi state is 26 tokens with the GGUF vocabulary and 40 with the author's tokenizer.json. llama.cpp's Qwen3.5 pre-tokenizer splits some scripts differently from HF tokenizers, and the author's runtime uses tokenizer.json.

The fix: the jev-style tags ship the author's tokenizer/tokenizer.json from the pinned commit as a tokenizer layer.

• Its sha256 is 06b95093…, the tokenizer_sha256 in readout_config.json.
• package_gguf adds the layer when the catalog entry names one.
• The server passes it to the llama runner as --tokenizer, the way it passes --mmproj.
• LlamaModel::load_tokenizer loads it with special tokens split, as the author's TextEncoder does, and checks the three readout tokens against it.

The GGUF vocabulary is now used only to index the logits. With this change, every id matches.

7. Goldens. convert/ollaya_convert/families/jev_style/goldens.py renders with the author's Renderer (jev_style_decision_gguf.py, on the pinned tokenizer) and scores with jev_score_v2.cpp, unchanged.

The scorer is compiled against the b11146 headers and linked to ggml-org's stock b11146 release libraries, the same binaries the runner loads. Those builds load their CPU backend variants at run time, and jev-score-v2 only calls llama_backend_init(). So a ten-line load_backends.c is linked beside it and calls ggml_backend_load_all_from_path before main. I chose this over building llama.cpp from source so that both sides run the identical libllama and CPU variant.

• Inputs: the shared set, plus a 6,214-token state (4 blocks), a 150-option catalogue overflow (several catalogue blocks), an input of exactly 25,600 tokens and one of 25,601.
• Per question: ids, slots, blocks, option names, [yes, no] logits (return_logits), scores and probabilities.
• Files: one per device, goldens-<device>.jsonl in the export directory, like the other GGUF families.
• Checks: every accepted render is checked against ref.py before it is written, and every rejection must also be a ref.py rejection.

A wire question becomes the author's {t, ins, crit} using the same list-of-labels and noul-key rules as the shared parser. Everything else is the author's code.

8. Gate. crates/ollaya-runner/examples/parity_jev_style.rs checks:

• identical ids, slots, blocks and option names;
• every decision the same;
• every verdict score within 1e-3.

It prints one record: line. To feed it, LlamaModel::jev_style_render exposes the runner's render. parity_run now records GGUF families listed in GGUF_RECORDED (only jev-style): the goldens are <export>/goldens-<device>.jsonl and the export is pinned by the GGUF's sha256. results/machines.json gains community-epyc-milan. CPU, measured on 2026-10-26 at cf1fa64:

Tag	File	Questions	Decisions	Verdict scores max	Probabilities max
jev-style:2b	Q8_0	582	582/582	4.4e-16	0
jev-style:2b-q4_k_m	Q4_K_M	582	582/582	4.4e-16	0

• Records: 141 in all. 124 requests are scored. 17 are rejected by both: 13 edge cases with structured instructions, 3 with a score legend given as an object, and the 25,601-token input.
• Coverage: the scored set includes 2 overflow questions (up to 150 options), 4 questions with a multi-block state, and the 25,600-token input.
• Scores: on the same libraries and device they agree to the last bit; the 4.4e-16 is the float64 rounding of the stored scores.

This also confirms what you said about seq_cp: the 25,600-token input fits the unified 32,768 context.

The CPU is an AMD EPYC Milan cloud VM with 8 vCPUs (the AVX2 backend), recorded as a community machine. I haven't run CUDA, Metal or Windows, and haven't measured latency. That's yours on the RTX 5090.

9. Render test. convert/tests/test_jev_style_render.py renders the shared set and the layout cases above with ref.py and with the author's Renderer on the pinned tokenizer.json. It compares ids, slots, blocks and names, and requires the same rejections. It asserts the set covers:

• the overflow form;
• a multi-block state;
• the 25,600-token edge;
• a rejection.

The pinned files are fetched once into the package cache, with sha256 checks, and the test skips without network.

10. Tags. jev-style:2b (and latest) is Q8_0. jev-style:2b-q4_k_m is Q4_K_M. Both are pinned at 283b9eb, with the GGUF hashes matching the author's manifest.json. The decision.json / calibration.json exports are out/jev-style-2b-q8_0 and out/jev-style-2b-q4_k_m. Registry manifests are not included, since packaging is stage 4 on your side.

11. Docs. docs/families/jev-style.md now has:

• a Parity section with the numbers above, what the set covers, and what was not run;
• a Tokenizer section;
• a Limits section: 25,600 tokens, 2 to 10 score levels, string instructions, no option cap;
• the card's training-data notes, pinned at the main repository's commit ee807b2.

The parity text in catalog.py now gives the measured numbers instead of "pending".

Two corrections to my previous comment:

• An input over the context or the render budget is a 400 (INVALID_REQUEST), not a 422. All decision errors map to 400 in server.rs. Either way, it's refused before any decode.
• My 2*prefix + question worry about the unified cache was wrong, as you said. The 25,600-token golden confirms it.

[cobanov]

@mirfan-cof, thanks. This is thorough work, and the tokenizer bug the goldens caught is a real find.

I went through the third round against items 1–11:

• 4. n_seq_max. Addressed: refused below 2 at load, next to the attention_type and n_ubatch checks.
• 5. Tests. Addressed, in jev_style.rs and against ref.py.
• 7. Goldens. Addressed: jev_score_v2.cpp unchanged on the stock b11146 libraries, with the multi-block state, the 150-option overflow and the 25,600 / 25,601 edges. Shipping the author's tokenizer.json as a layer, with the readout tokens checked against it at load, is the right fix.
• 8. Gate. Addressed: parity_jev_style prints a record: line, and the CPU record (582/582 for both tags) is in results/runs/.
• 9. Render test. Addressed.
• 10. Tags. Addressed: 2b = latest = Q8_0, 2b-q4_k_m = Q4_K_M, both pinned at 283b9eb.
• 11. Docs and catalog. Addressed.

I pushed two small commits to your branch:

• 121ff97 merges main. COB-561: Decision 2.0 (vllm-sr) as the family decision2, and the 2026.10 edition's 0.18.0 backlog #153 had just added "decision2": "decision" to EXAMPLE in parity_run.py, on the line above GGUF_RECORDED.
• ece7118 skips test_jev_style_render.py when numpy or tokenizers is missing. CI's packaging job runs convert/tests without ML dependencies, and goldens.py imports numpy at module level, so the module failed to import instead of skipping. That was the one red job on 520d97e. The other jobs passed.

Nothing else is needed from you. What's left is on our side: CUDA goldens and parity_run --device cuda for both tags on our RTX 5090, as I said last time. I'll merge once that passes and CI is green on ece7118. If CUDA shows anything, I'll post it here.

[cobanov]

@mirfan-cof, CUDA parity passes on our RTX 5090, so I'm merging.

Setup: goldens from jev_score_v2.cpp (unchanged) on the same GPU (CUDA0, full offload), then parity_run --machine rtx-5090 --device cuda at ece7118. The scorer and the runner load the same b11146 libllama and libggml-cuda (identical sha256).

Tag	File	Questions	Decisions	Verdict scores max	Probabilities max	Time
jev-style:2b	Q8_0	582	582/582	4.4e-16	0	13.1 s
jev-style:2b-q4_k_m	Q4_K_M	582	582/582	4.4e-16	0	13.3 s

• Encodings are identical: ids, slots, blocks and option names. Both sides reject the same 17 records.
• Peak GPU memory was about 3.6 GiB, including the 25,600-token input in the unified 32,768 context.
• I ran the gate a second time and got a byte-identical record.

The record is results/runs/2026-10-26-parity-rtx5090-cuda.json (67f9605). The family doc and the catalog's parity text now name both devices. Thanks for the careful work on this one.