Skip to content

README refresh: 50+ tok/s headline, recipe, new charts, project-scale card - #3

Merged
edagher92-coder merged 5 commits into
mainfrom
readme-refresh
Oct 17, 2026
Merged

edagher92-coder merged 5 commits into
mainfrom
readme-refresh

Conversation

@edagher92-coder

Copy link
Copy Markdown
Owner

Front-page refresh to make the repo easier to find and act on for Intel Arc owners. No new measurements: every figure comes from existing logs.

  • Hero banner and headline: 53 tok/s mean per-request decode (47-58 across 7 coding tasks, data/raw/tune.txt) with a 64K KV window allocated. Fine print up front: ~42 tok/s at 30-37K real context; 22 tok/s at 53K under contention on 17 Oct; 50-60K not yet qualified.
  • New "recipe" section: the four changes that matter, plus a copy-paste llama-server line.
  • New charts (light + dark): per-task decode, GPU contention (17 Oct), project by the numbers. Refreshed social preview.
  • tools/make_hero.py regenerates them from data/decode-per-request.csv and data/gpu-contention-20261017.csv.
  • Checked locally: check_links (0 broken), build_tables --check, build_site (no docs diff besides the synced social preview), validate_repo (0 failed).

Generated with Claude (Cowork).

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-17T09:55:03.797672Z c86bf48 PR opened
🔒 Security Review ✅ Completed 2026-10-17T09:53:27.031162Z c86bf48 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c86bf48560

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread README.md

| Step | What it buys | Evidence |
|---|---|---|
| **1. DFlash2 draft model**, 7 tokens per round | Decode **16 -> 53 tok/s** mean (47-58), coding still 7/7 correct | [tune.txt](data/raw/tune.txt), [compare-f16.txt](data/raw/compare-f16.txt) |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid presenting unlike decode workloads as an A/B

The 16 → 53 tok/s arrow implies a measured DFlash2 speedup on the same workload, but the new CSV shows that 53 is the mean of seven server-side coding requests while 16.05 comes from llama-bench tg32 at empty context. Because speculative-decoding acceptance and throughput depend on the generated workload, these figures cannot establish that before/after gain; either rerun the seven tasks without the drafter or label the values as separate measurements without the arrow.

Useful? React with 👍 / 👎.

Comment thread tools/make_hero.py
Comment on lines +103 to +104
fig.text(0.04, 0.93, f"Every coding task decodes at 47-58 tok/s; {sum(v >= 50 for v in drafted)} of {len(drafted)} clear 50",
color=t["ink"], fontsize=16.5, fontweight="bold")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Derive the decode headline from the input rows

When data/decode-per-request.csv is updated with another task or new measurements, the bars and mean will regenerate from the CSV but this headline will continue to claim the fixed range 47-58. That silently produces a self-contradictory published chart despite running the documented generator; compute the displayed minimum and maximum from drafted just as the mean is computed.

Useful? React with 👍 / 👎.

Comment thread README.md

```powershell
$env:GGML_VK_FORCE_MMVQ = "1"
llama-server -m Qwen3.8-27B-UD-Q4_K_M.gguf -ngl 99 -fa on -c 65536 -ctk f16 -ctv f16 `

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Use the paths created by the documented setup

For a user following the linked Quick start and docs/SETUP.md, this copy-paste command cannot start: the build writes the executable to C:\AI\llama.cpp\vulkan-intelfa\llama-server.exe, while the model and drafter are placed under C:\AI\models\qwen3.8-27b\; neither is added to PATH or copied into the repository working directory. Invoke the built executable and pass the documented absolute model paths so the advertised recipe works from the location where users run the README commands.

Useful? React with 👍 / 👎.

@edagher92-coder
edagher92-coder merged commit abe2f60 into main Oct 17, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant