Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
name: Bug report
about: Something broke on your AMD GPU
title: "bug: "
labels: bug
---
**Hardware / stack** (paste `scripts/00-check-env.sh` output or
docs/results/environment.json):
**Exact command:**
**Full log** (pastebin/gist if long):
**What you expected vs what happened:**
11 changes: 11 additions & 0 deletions .github/ISSUE_TEMPLATE/validation_report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
name: Validation report
about: Share results from YOUR hardware (successes AND failures)
title: "validation: <GPU> / ROCm <version>"
labels: validation
---
**GPU / ROCm / torch:**
**`00-check-env.sh` output:**
**Summary table** (`python3 scripts/summarize_results.py`):
**Receipts** (docs/results/… if you can share):
**Anything that failed, with the log:**
10 changes: 10 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
**What & why** (one paragraph; link the spec/design doc if any):

**Evidence** (receipts under docs/results/, or `verified on: <gfx>` +
link for ports from sibling repos — see CONTRIBUTING):

**Gates** (must both be green):
- [ ] `python3 -m pytest tests/ -q`
- [ ] `shellcheck scripts/*.sh scripts/lib/common.sh docs/hardware/strix-halo/hsa_fix.sh`

**If ported from SenseNova-U1.5-ROCm-8060S:** source commit SHA:
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,4 @@ jobs:
- name: ShellCheck the pipeline scripts
run: |
sudo apt-get install -y shellcheck -q
shellcheck scripts/*.sh scripts/lib/common.sh
shellcheck scripts/*.sh scripts/lib/common.sh docs/hardware/strix-halo/hsa_fix.sh
22 changes: 22 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,28 @@ and the project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.
to PR head e2f2c865; V1–V4 on-model verification (list input fixed,
tuple output byte-identical) — receipts
docs/results/validation/interleave-image-size/.
- Patch 0003: optional torch.compile/cudagraph safety (env-gated
`SENSENOVA_COMPILE*`); ported from the Strix Halo sibling repo —
verified on gfx1151 (V1–V8 + quality bench there); gfx1100 receipt
pending, compile stays opt-in.
- `scripts/gpu_monitor.py`: sample VRAM + GTT while a workload runs
(JSONL + peak summary) — GTT visibility is what evidences APU
GTT-spill runs.
- `scripts/rocm_check.py`: env sanity check (HIP build, wheel gfx
coverage, bf16 matmul + SDPA smoke, optional GTT-spill alloc test).
- Hardware profile `docs/hardware/strix-halo/` (EN+CN): GTT-spill
memory model, HSA runtime shim (`hsa_fix.sh`, now CI-shellchecked),
gfx1151 wheel guidance, AOTriton hd72 caveat — the umbrella's first
APU profile, ported from the 8060S repo.
- `docs/porting.md` + `docs/porting_CN.md`: generic ROCm porting notes,
split from the 8060S repo's porting doc (APU-specific depth moved to
the strix-halo profile).
- `docs/upstream/aotriton-54.md`: upstream tracking entry for the
AOTriton hd=72 silent-wrong-output bug (issue #54, fix unmerged);
verified on gfx1151, evidence in the 8060S repo.
- Governance: Code of Conduct, `SECURITY.md`, bug/validation issue
templates, PR template (evidence gates), CONTRIBUTING cross-repo
section (graduation model), README/README_CN hardware-coverage table.

### Changed
- pytorch#194498 root cause verified on-host: SDPA fused-backend launch
Expand Down
22 changes: 22 additions & 0 deletions CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Code of Conduct

## Our pledge
Be evidence-first about claims and kind to people. We welcome
contributors of every background and skill level.

## Standards
**Encouraged:** citing receipts for claims, asking questions, reporting
both successes and failures, reviewing others' work respectfully.
**Unacceptable:** harassment or discriminatory language, hostile
responses to newcomers, presenting unverified numbers as measured,
doxxing, spam.

## Enforcement
Report violations by opening a private GitHub security advisory
("Report a vulnerability" → select "not applicable") or contacting a
maintainer via GitHub. Maintainers will respond within 7 days and may
remove comments/commits/ban repeat offenders.

## Attribution
Adapted in spirit from the Contributor Covenant 2.1.
本行为准则同样适用于中文社区讨论;违反行为可同样通过上述渠道举报。
22 changes: 22 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,3 +51,25 @@ re-runs a single block for a minimal reproduction.
- Scripts: `set -euo pipefail`, a `usage()` with `--help`, clear `[project]`
log prefixes — follow the existing scripts.
- Numbers in docs: cite the receipt file they came from.

## Contributing between the two repos

This repo is the **umbrella** (hardware-generic assets + hardware
profiles). [SenseNova-U1.5-ROCm-8060S](https://github.com/AIwork4me/SenseNova-U1.5-ROCm-8060S) is the **Strix Halo lab**:
APU-first experiments live there; verified work graduates here.

**8060S → umbrella (graduation).** A change graduates when it is
① hardware-independent or absorbed as a hardware profile, ② verified on
gfx1151 with evidence in the 8060S repo, ③ passes this repo's offline
CI, ④ documented EN+CN, ⑤ not a duplicate of an existing tool here.
Open a PR labelled `backport`; the commit carries
`Ported from …@<sha>` and the claim is labelled
`verified on: gfx1151` until a gfx1100 receipt lands here.

**Umbrella → 8060S (flow-down).** Changes to shared surfaces
(`patches/`, `scripts/`) are cherry-picked by the 8060S repo as
needed; that repo records `Merged-upstream: <sha>`.

**Evidence across repos.** `receipt.py` records `hardware=<gfx>` on
every run; cross-hardware claims must link the producing repo's
evidence.
11 changes: 11 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,17 @@ bash scripts/quickstart.sh # t2i | vqa | edit | interleave
[Troubleshooting](docs/getting-started.md#troubleshooting) ·
[Contributing](CONTRIBUTING.md)

## Hardware coverage

| hardware | status | where |
|---|---|---|
| gfx1100 (RDNA3 dGPU) | fully validated — receipts in [docs/results/](docs/results/) | this repo |
| gfx1151 (Strix Halo APU) | fully validated — evidence in the [8060S repo](https://github.com/AIwork4me/SenseNova-U1.5-ROCm-8060S); profile: [docs/hardware/strix-halo/](docs/hardware/strix-halo/) | this repo + Strix Halo lab |

Other AMD GPUs: the pipeline is hardware-generic (see
[Porting Notes](docs/porting.md)) — validation reports welcome
(issue template `validation_report`).

## Showcase: movie-poster style

Generated on the reference gfx1100 host with this repo's pipeline
Expand Down
14 changes: 14 additions & 0 deletions README_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,25 @@
[SenseNova-U1#260](https://github.com/OpenSenseNova/SenseNova-U1/pull/260))与
ROCm SDPA 融合后端崩溃(补丁 0002,[pytorch#194498](https://github.com/pytorch/pytorch/issues/194498)、
[SenseNova-U1#261](https://github.com/OpenSenseNova/SenseNova-U1/issues/261))
- 🧪 **gfx1151(Strix Halo APU)同样全量验证**:证据见姊妹仓库
[SenseNova-U1.5-ROCm-8060S](https://github.com/AIwork4me/SenseNova-U1.5-ROCm-8060S)
(Strix Halo 实验室),硬件配置在本仓库
[docs/hardware/strix-halo/](docs/hardware/strix-halo/)

```bash
bash scripts/quickstart.sh # 可选任务: t2i | vqa | edit | interleave
```

## 硬件覆盖

| 硬件 | 状态 | 位置 |
|---|---|---|
| gfx1100(RDNA3 独显) | 完全验证 —— 收据见 [docs/results/](docs/results/) | 本仓库 |
| gfx1151(Strix Halo APU) | 完全验证 —— 证据见 [8060S 仓库](https://github.com/AIwork4me/SenseNova-U1.5-ROCm-8060S);配置:[docs/hardware/strix-halo/](docs/hardware/strix-halo/) | 本仓库 + Strix Halo 实验室 |

其他 AMD GPU:本管线与硬件无关(见[移植指南](docs/porting.md))——
欢迎提交验证报告(issue 模板 `validation_report`)。

## 效果展示:电影海报风格

以下海报由本仓库管线在 gfx1100 参考机上生成
Expand Down
17 changes: 17 additions & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Security Policy

## Supported versions
The tip of `main` only.

## Reporting a vulnerability
Please do NOT open a public issue. Use GitHub's private vulnerability
reporting ("Security" tab → "Report a vulnerability"). Include
reproduction steps and affected components (scripts/, patches/, docs/
pipeline). Expect a response within 7 days.

## Scope
This repo ships shell/python scripts and upstream patches; it does not
serve anything to the network. Model weights come from upstream
ModelScope — weight integrity is verified by
configs/artifact-manifest.json SHA256s (any tampering report is
welcome but belongs upstream).
3 changes: 3 additions & 0 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@

From a bare Linux host with an AMD GPU to your first SenseNova-U1.5 output.

For how this port works (wheels, attention backend, memory tiers):
[Porting Notes](porting.md) · [移植笔记](porting_CN.md)

## What you need

| Requirement | Why | Reference host |
Expand Down
93 changes: 93 additions & 0 deletions docs/hardware/strix-halo/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# Hardware Profile: Strix Halo (Radeon 8060S iGPU, gfx1151)

verified on: gfx1151 — Ryzen AI MAX+ PRO 395, ROCm 7.2.1 system +
torch 2.12.0+rocm7.14.0 wheels. Full evidence corpus (V1–V8 speedup,
quality bench, determinism, tier tests):
[SenseNova-U1.5-ROCm-8060S](https://github.com/AIwork4me/SenseNova-U1.5-ROCm-8060S).
gfx1100 receipts live in [docs/results/](../../results/) here.
Generic background: [porting notes](../../porting.md).

## Why full bf16 fits a "32 GB" iGPU — GTT spill

On AMD APUs the GPU shares physical memory with the CPU. The driver
exposes two pools: dedicated **VRAM** (BIOS carve-out, e.g. 32 GB on the
reference machine) and **GTT** (the rest of system memory the GPU may
map, e.g. 80 GB). ROCm's allocator can allocate past dedicated VRAM by
spilling into GTT, so `torch.empty(50 GiB, device="cuda")` succeeds.
Since both pools are the same physical LPDDR5X on package, the penalty
is modest — in the 8060S repo's suite runs the allocator put ~1.2 GiB in
VRAM and up to 41.6 GiB in GTT (112 GiB total GPU-addressable), with
full-mode peaks ≤ 42.9 GiB, so ~45+ GiB addressable works in practice.

`scripts/rocm_check.py --alloc-test 48` verifies exactly this spill
(48 GiB allocation; archived as `evidence/stack714/env_check.json` in
the 8060S repo). This is what makes `--vram_mode full` (entire ~50 GB
checkpoint GPU-resident, no layer offload) possible on a "32 GB" GPU.
Note when choosing tiers on the 7.14 stack: `full` is also the fastest
tier there — do not override to `fast`/`balanced` for speed (8060S
PERFORMANCE.md, 2026-08-25 correction; tiers remain the tool for VRAM
capacity on smaller GPUs).

## HSA runtime pitfall (torch +rocm7.0 wheels, driver ≥ 7.1)

Official `torch 2.10.0+rocm7.0` wheels bundle a ROCm 7.0 HSA runtime
that segfaults on gfx1151 (Strix Halo) when the kernel driver is from
ROCm ≥ 7.1: crash in `GpuAgent::QueueCreate` → `ReleaseQueueMainScratch`
while freezing the first code object — every GPU op dies. Fix: preload
the system ROCm runtime
(`LD_PRELOAD=/opt/rocm/lib/libhsa-runtime64.so`), which always matches
the kernel driver.

This profile ships the shim as [hsa_fix.sh](hsa_fix.sh) — source it
(not execute) and call `hsa_fix_apply "$PY" "$REPO_ROOT"`; it exports
`LD_PRELOAD` only when the system runtime is NEWER than the wheel's
bundled one, so machines where the bundled runtime already works (or is
newer) are untouched. The recommended 7.14 wheels run bare and never
trigger it.

## Wheel guidance for gfx1151

- **Default (recommended):** AMD official multi-arch
`torch 2.12.0+rocm7.14.0` from
`https://repo.amd.com/rocm/whl-multi-arch/` — verified bare-run on
gfx1151 (no HSA preload needed) and faster than the torch-2.10 build
in the 8060S A/B (167.9 s vs 232.1 s for a 1024×1024 t2i in `fast`
mode; both sessions were contended, so treat the magnitude as
indicative). These wheels bundle their own ROCm userspace — a system
ROCm mainly provides the kernel-driver tooling (`rocminfo`,
`rocm-smi`).
- **Fallback:** official PyTorch rocm7.0 index
(`https://download.pytorch.org/whl/rocm7.0`, `torch 2.10.0+rocm7.0`;
covers gfx942/950/1100/1101/1151/1200) — on gfx1151 it needs the HSA
preload above.
- **Last resort:** AMD arch-specific nightlies
(`https://rocm.nightlies.amd.com/v2/<gfx>/`) for arches the official
wheels miss.

In this repo `scripts/01-setup-venv.sh` installs from a single index —
point `PY_INDEX_ROCM` at the source you want; the deeper
wheel-has-kernels-for-your-gfx check is `scripts/rocm_check.py`
(see [porting notes §4](../../porting.md#4-pytorch-wheel-matrix-known-rocm-wheel-sources)).

## AOTriton caveat (hd=72 ViT heads)

On gfx1151 with torch 2.12+rocm7.14, AOTriton flash SDPA is *silently
wrong* for head_dim=72 with seq>1024, **non-causal only** (causal
attention at the same shape is correct) — errors drift run-to-run,
typically 16–730% relative error, dominantly finite garbage with
sporadic NaN in ~1/5 of calls. SenseNova's own attention shapes
(head_dim 64/128) are unaffected, but disable AOTriton for a foreign
model with a 16-head/1152-hidden ViT. Root cause: upstream AOTriton
[issue #54](https://github.com/ROCm/aotriton/issues/54) (missing
`out_dtype=tl.float32` in the fwd kernel); a fix exists (commit
`8232d69672`) but is unmerged in every release so far — tracked in
[docs/upstream/aotriton-54.md](../../upstream/aotriton-54.md).

## Companion repo

The daily-driver run scripts, one-command quickstart, and the full
evidence corpus live in the
[8060S repo](https://github.com/AIwork4me/SenseNova-U1.5-ROCm-8060S) —
the Strix Halo lab where these numbers were measured. Experiments there
graduate into this umbrella via PR when they meet the graduation
criteria ([CONTRIBUTING.md](../../../CONTRIBUTING.md)).
82 changes: 82 additions & 0 deletions docs/hardware/strix-halo/README_CN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# 硬件 Profile:Strix Halo(Radeon 8060S 核显,gfx1151)

verified on: gfx1151 —— Ryzen AI MAX+ PRO 395,系统 ROCm 7.2.1 +
torch 2.12.0+rocm7.14.0 wheel。完整证据集(V1–V8 提速、质量评测、
确定性、档位测试):
[SenseNova-U1.5-ROCm-8060S](https://github.com/AIwork4me/SenseNova-U1.5-ROCm-8060S)。
gfx1100 凭据在本仓库 [docs/results/](../../results/)。
通用背景见[移植笔记](../../porting_CN.md)。

## 为什么"32 GB"核显装得下完整 bf16 —— GTT 溢出

AMD APU 的 GPU 与 CPU 共享物理内存。驱动暴露两个池:专用 **VRAM**
(BIOS 挖出,参考机型为 32 GB)与 **GTT**(GPU 可映射的其余系统内存,
如 80 GB)。ROCm 分配器越过专用 VRAM 时会溢出到 GTT,因此
`torch.empty(50 GiB, device="cuda")` 能成功。两个池都是同一颗封装内
的 LPDDR5X 物理内存,代价不大——8060S 仓库套件运行期间,分配器把
~1.2 GiB 放进 VRAM、最多 41.6 GiB 进 GTT(GPU 可寻址合计 112 GiB),
full 档峰值 ≤ 42.9 GiB,实践上 ~45+ GiB 可寻址即可。

本仓库的 `scripts/rocm_check.py --alloc-test 48` 正是验证这一溢出
(48 GiB 分配;8060S 仓库存证 `evidence/stack714/env_check.json`)。
这就是"32 GB 显存"也能跑 `--vram_mode full`(约 50 GB 权重全驻留
GPU、不做 layer offload)的原因。7.14 栈上选档提示:`full` 同时也是
最快档——不要为提速改选 `fast`/`balanced`(8060S PERFORMANCE.md
2026-08-25 更正;档位仍是小显存 GPU 的容量工具)。

## HSA 运行时的坑(torch +rocm7.0 wheel,驱动 ≥ 7.1)

官方 `torch 2.10.0+rocm7.0` 轮子自带 ROCm 7.0 的 HSA 运行时,在
gfx1151(Strix Halo)+ ROCm ≥ 7.1 内核驱动上冻结首个 code object 时
(`GpuAgent::QueueCreate` → `ReleaseQueueMainScratch`)段错误——任何
GPU 算子都跑不起来。解法:预加载系统 ROCm 运行时
(`LD_PRELOAD=/opt/rocm/lib/libhsa-runtime64.so`),它必然与内核驱动
匹配。

本 profile 以 [hsa_fix.sh](hsa_fix.sh) 携带该 shim——source(而非
执行)本文件后调用 `hsa_fix_apply "$PY" "$REPO_ROOT"`;仅当系统运行时
比 wheel 自带的**更新**时才 export `LD_PRELOAD`,自带运行时本来就
正常(或更新)的机器不受影响。推荐的 7.14 wheel 裸跑,不会触发。

## gfx1151 wheel 指南

- **默认(推荐)**:AMD 官方多架构 `torch 2.12.0+rocm7.14.0`,
索引 `https://repo.amd.com/rocm/whl-multi-arch/`——gfx1151 实测
裸跑(无需 HSA 预加载),且在 8060S A/B 中快于 torch-2.10 构建
(1024×1024 t2i fast 档 167.9 s vs 232.1 s——当年会话存在争用,
幅度仅供参考)。这类 wheel 自带 ROCm 用户态——系统 ROCm 主要提供
内核驱动工具(`rocminfo`、`rocm-smi`)。
- **回退**:官方 PyTorch rocm7.0 索引
(`https://download.pytorch.org/whl/rocm7.0`,`torch 2.10.0+rocm7.0`;
覆盖 gfx942/950/1100/1101/1151/1200)——gfx1151 上需上面的 HSA
预加载。
- **兜底**:AMD 按架构 nightly
(`https://rocm.nightlies.amd.com/v2/<gfx>/`),用于官方 wheel 缺
架构时。

本仓库的 `scripts/01-setup-venv.sh` 从单一索引安装——用
`PY_INDEX_ROCM` 指向你想要的来源;更深入的"wheel 是否带你 gfx 内核"
检查是 `scripts/rocm_check.py`
(见[移植笔记 §4](../../porting_CN.md#4-pytorch-wheel-矩阵已知-rocm-wheel-来源))。

## AOTriton 告警(hd=72 ViT 头)

gfx1151 + torch 2.12+rocm7.14 下,AOTriton flash SDPA 在 head_dim=72
且 seq>1024、**仅非因果注意力**时**静默输出错误值**(同形状因果注意力
正常)——误差逐次漂移,典型相对误差 16~730%,以有限错误值为主、约
1/5 次调用出现 NaN。SenseNova 自身的注意力形状(head_dim 64/128)
不受影响;但若复用环境跑 16 头/hidden 1152 的外部 ViT 模型,请对其
关闭 AOTriton。根因:上游 AOTriton
[issue #54](https://github.com/ROCm/aotriton/issues/54)(fwd 内核缺
`out_dtype=tl.float32`);修复 commit `8232d69672` 已存在但至今未合并
进任何发布——追踪详情见
[docs/upstream/aotriton-54.md](../../upstream/aotriton-54.md)。

## 配套仓库

日常驱动脚本、一条命令 quickstart 与完整证据集都在
[8060S 仓库](https://github.com/AIwork4me/SenseNova-U1.5-ROCm-8060S)
——即 Strix Halo 实验室,上述数字都在那里测得。实验在那里达到毕业
标准(graduation criteria,
[CONTRIBUTING.md](../../../CONTRIBUTING.md))后,通过 PR 毕业进入
本 umbrella 仓库。
Loading
Loading