Skip to content

Add HIP-Basic execution context example - #485

Open
neon60 wants to merge 13 commits into
amd-stagingfrom
execution_context
Open

Add HIP-Basic execution context example#485
neon60 wants to merge 13 commits into
amd-stagingfrom
execution_context

Conversation

@neon60

@neon60 neon60 commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Motivation

Demonstrate partitioning a device's compute units into separate execution contexts, running a long-running background kernel and a latency-sensitive critical kernel on their own dedicated resources. Register the example in the HIP-Basic CMake build and the top-level README.

Technical Details

Adds a new HIP-Basic/execution_context example (HIP's counterpart to CUDA green contexts).

  • Queries the device's compute units with hipDeviceGetDevResource (hipDevResourceTypeSm), splits them with hipDevSmResourceSplit, wraps each group in a descriptor via hipDevResourceGenerateDesc, and creates an execution context per group with hipGreenCtxCreate.
    A stream created on each context with hipExecutionCtxStreamCreate confines its kernels to that context's CUs.
  • Runs a fixed critical workload concurrently with a device-saturating background kernel and measures the critical kernel's runtime. It first records a baseline where both kernels share all CUs, then sweeps several partition sizes (about an eighth, a quarter, and half of the device), printing a table of each configuration's critical- and background-kernel runtimes plus the critical kernel's speedup over the baseline. As the critical partition grows, its runtime drops below the contended baseline while the background kernel (confined to fewer CUs) takes longer.
  • Partitioning is guarded by HIP_PLATFORM_AMD. On the CUDA backend, where the required runtime support may be unavailable, only the shared-CU baseline runs, and the CU-equivalent count is read from hipGetDeviceProperties.
  • Includes a warm-up launch and hipDeviceSynchronize between cases so the device is idle before contexts are created and destroyed.
  • Registered via add_subdirectory(execution_context) in HIP-Basic/CMakeLists.txt and listed in the top-level README.md.

Files added: main.hip, CMakeLists.txt, Makefile, README.md, .gitignore.

Test Plan

  • Build through the HIP-Basic CMake target hip_execution_context and run ./bin/hip_execution_context on an AMD GPU.
  • Verify the baseline row plus one partitioned row per sweep size are printed, that the critical kernel's runtime falls (and
    speedup rises) as the partition grows, and that the program completes cleanly.

Test Result

Run on a 80-CU device (gfx1100): the critical kernel's runtime dropped from the shared baseline (~230 ms) to ~110 ms on larger partitions, roughly a 2x speedup, with the background-kernel runtime rising as its CU share shrank.

Added/Updated documentation?

  • Yes
  • No, does not apply to this PR.

Included Visual Studio files?

  • Yes
  • No, does not apply to this PR.

Submission Checklist

@neon60
neon60 force-pushed the execution_context branch from 7dda124 to 9631fa4 Compare July 24, 2026 10:00
@neon60
neon60 requested review from adeljo-amd and j-stephan July 24, 2026 10:49
@neon60
neon60 marked this pull request as ready for review July 24, 2026 10:49
@neon60
neon60 requested review from a team as code owners July 24, 2026 10:49
@neon60
neon60 force-pushed the execution_context branch from 9631fa4 to e721050 Compare July 24, 2026 10:55

@adeljo-amd adeljo-amd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment thread HIP-Basic/execution_context/main.hip Outdated

// [read-wq-config-start]
hipDevResource wq_config = {};
HIP_CHECK(hipDeviceGetDevResource(current_device, &wq_config, hipDevResourceTypeWorkqueueConfig));

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line fails the test on CI's gf1100 and 1151, does it require driver updates? Any ASIC support gap?

@neon60 neon60 Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zichguan-amd updated the code to not fail the test for this only log. I will ask HIP team about this.

neon60 and others added 12 commits July 31, 2026 08:21
Demonstrate partitioning a device's compute units into separate
execution contexts, running a long-running kernel and a critical kernel
on their own reserved resources. Register the example in the HIP-Basic
CMake build and the top-level README.

Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
Guard the execution-context resource partitioning with
__HIP_PLATFORM_AMD__ so the example still compiles and runs on the CUDA
backend, where that support may be unavailable. On non-HIP backends it
falls back to two ordinary non-blocking streams without CU partitioning
and reads the CU-equivalent count from hipGetDeviceProperties. Update the
README to describe the fallback path and its APIs.

Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
The example previously timed the long and critical kernels separately, so
it never showed the benefit of execution contexts. Restructure it to run
the critical kernel concurrently with a long-running background kernel and
measure the critical kernel's latency in two scenarios: a baseline where
both kernels share all CUs on ordinary streams, and a partitioned run
where the background kernel is confined to a larger execution context and
the critical kernel runs on its own reserved CUs. Report both latencies
and the speedup. Update the README to describe the comparison.

Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
A single baseline-vs-partitioned comparison did not clearly show the
effect of partitioning. Restructure the example to keep the critical
kernel's work fixed while sweeping how many dedicated CUs it gets (about
an eighth, a quarter, and half of the device), each as its own execution
context, and print a results table of latency and speedup over the shared
baseline. As the partition grows, the critical kernel's latency falls
below the contended baseline, making the benefit visible as a trend.

Extract time_partitioned_case to keep each sweep iteration self-contained
and bundle workload sizing into a struct. Update the README to describe
the sweep.

Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
Add the background kernel's runtime alongside the critical kernel's in the
results table by timing both kernels with events and returning them in a
struct. As the critical partition grows, the table now shows the critical
runtime falling while the background runtime rises, since the background
kernel is confined to fewer CUs.

Address the occasional hang by warming up the GPU before the first
measurement and calling hipDeviceSynchronize after the baseline and each
partitioned case, so the device is idle before execution contexts are
created and destroyed. Update the README to describe the runtime columns.

Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
Lower the background grid from 64x to 16x the CU count. This still
oversubscribes the device enough to contend with the critical kernel while
shortening the background kernel's total runtime.

Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
Implement the partitioned sweep on the CUDA backend using the CUDA driver
API green-context calls (cuDevSmResourceSplitByCount, cuDevResourceGenerateDesc,
cuGreenCtxCreate, cuGreenCtxStreamCreate), the direct counterpart of HIP
execution contexts. A single split carves the device SMs into a critical
group and a disjoint remainder so the two green contexts never overlap, and
the green-context stream is returned as a cudaStream_t so the existing kernel
launches and event timing work unchanged. The baseline and sweep logic are
now shared across both backends.

Link the CUDA driver library (libcuda) in the CMake and Make builds, and
update the README to document the CUDA green-context path.

Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
The example failed in CI on a device whose CU count is not a multiple of the
WGP alignment: total_cus / 4 requested an odd critical partition, which the
split API rejected with an "unknown error". Read smCoscheduledAlignment and
snap each requested partition size up to it so both groups are aligned.

Also correct the demonstration: the critical kernel launched a full-device
grid, so on a small partition its blocks queued onto few CUs and it ran
slower than the shared baseline (speedup below 1x). Size the critical kernel
to a couple of thread blocks that fit inside even the smallest partition, so
its runtime reflects waiting for CUs rather than how many CUs it holds. Fix a
related warm-up launch that wrote a full-device grid into the small critical
buffer by targeting the large background buffer instead.

Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
The CUDA CUdevSmResource struct has no smCoscheduledAlignment member, so
reading it broke the nvcc build. Read it only on the HIP backend; on CUDA
use an alignment of 1, since cuDevSmResourceSplitByCount rounds partition
sizes internally.

Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
Wrap the busy kernel, timing helper, partitioned measurement, and baseline
regions in named [<region>-start]/[<region>-end] comments so the HIP docs can
literalinclude them as the single source of truth.

Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
Add demonstrate_reading_resources() with Sphinx marker comments so the
device compute-unit and work queue configuration query snippets shown in
the HIP execution context how-to are sourced from compiled, tested code.

Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
Reading hipDevResourceTypeWorkqueueConfig from a device fails with an error on
runtimes that do not support it, which aborted the example after the CU query
succeeded. Capture the status and print the configuration only when the query
succeeds, so the example runs on all runtimes.

Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
@neon60
neon60 force-pushed the execution_context branch from 61236fc to 8a4d598 Compare July 31, 2026 06:22
The non-fatal work queue configuration query leaves hipErrorUnknown in the
thread's sticky last-error state on runtimes that reject it. The next
hipGetLastError check, after the warmup launch, then reported that stale error
and aborted. Consume the error with hipGetLastError right after handling the
failed query so it is not misattributed to a later, successful launch.

Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>

@zichguan-amd zichguan-amd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants