Skip to content

Document GPU configuration flow across pipelines, modules, and profiles - #4352

Open
mahesh-panchal wants to merge 7 commits into
nf-core:mainfrom
mahesh-panchal:pipeline-gpu-configuration
Open

Document GPU configuration flow across pipelines, modules, and profiles#4352
mahesh-panchal wants to merge 7 commits into
nf-core:mainfrom
mahesh-panchal:pipeline-gpu-configuration

Conversation

@mahesh-panchal

@mahesh-panchal mahesh-panchal commented Aug 14, 2026

Copy link
Copy Markdown
Member

Summary

  • Clarify that accelerator is a map (request/type keys), not a plain integer, and document how a GPU request flows: module -> pipeline config -> institutional profile -> user override.
  • Add a "GPU resource requests" section to the institutional profiles guide, including a note on --gres vs --gpus on SLURM, and update the example gpu profile to read task.accelerator via a closure instead of hardcoding --gres=gpu:1.
  • Add a new running/configuration/gpu-pipelines.md guide aimed at pipeline users.
  • Remove the now-unused ext.use_gpu key from the ext-args reference.

@netlify /docs/running/configuration/gpu-pipelines

Clarify that `accelerator` is a map (`request`/`type`), not a plain
integer, and explain how a GPU request flows from module ->
pipeline config -> institutional profile -> user override. Add a
GPU resource requests section to the institutional profiles guide
(with a `--gres` vs `--gpus` SLURM note), update the example gpu
profile to read `task.accelerator` via a closure instead of
hardcoding `--gres=gpu:1`, and add a new running/configuration guide
for pipeline users. Remove the now-unused `ext.use_gpu` key from the
ext-args reference.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@netlify

netlify Bot commented Aug 14, 2026

Copy link
Copy Markdown

Deploy Preview for nf-core-main-site ready!

Name Link
🔨 Latest commit 826532b
🔍 Latest deploy log https://app.netlify.com/projects/nf-core-main-site/deploys/6a8359268f70360008922a6e
😎 Deploy Preview https://deploy-preview-4352--nf-core-main-site.netlify.app/docs/running/configuration/gpu-pipelines
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@netlify

netlify Bot commented Aug 14, 2026

Copy link
Copy Markdown

Deploy Preview for nf-core-docs ready!

Name Link
🔨 Latest commit 826532b
🔍 Latest deploy log https://app.netlify.com/projects/nf-core-docs/deploys/6a835926627faf000832bad4
😎 Deploy Preview https://deploy-preview-4352--nf-core-docs.netlify.app/docs/running/configuration/gpu-pipelines
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@mahesh-panchal

Copy link
Copy Markdown
Member Author

@nf-core-bot fix linting

Comment thread sites/docs/src/content/docs/developing/components/ext-args.md

@mashehu mashehu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just some nitpicks, can be ignored if you disagree

Comment thread sites/docs/src/content/docs/developing/components/gpu-modules.md Outdated
Comment thread sites/docs/src/content/docs/developing/institutional-profiles/configuration.md Outdated
Comment thread sites/docs/src/content/docs/running/configuration/gpu-pipelines.md Outdated
Comment thread sites/docs/src/content/docs/running/configuration/gpu-pipelines.md Outdated
Comment thread sites/docs/src/content/docs/running/configuration/gpu-pipelines.md Outdated
Comment thread sites/docs/src/content/docs/running/configuration/gpu-pipelines.md Outdated
Comment thread sites/docs/src/content/docs/running/configuration/gpu-pipelines.md Outdated
Comment thread sites/docs/src/content/docs/developing/components/gpu-modules.md Outdated
Comment thread sites/docs/src/content/docs/developing/institutional-profiles/configuration.md Outdated
Comment thread sites/docs/src/content/docs/running/configuration/gpu-pipelines.md
Comment thread sites/docs/src/content/docs/developing/components/gpu-modules.md Outdated
Comment thread sites/docs/src/content/docs/developing/components/gpu-modules.md Outdated
mahesh-panchal and others added 2 commits August 17, 2026 11:39
Co-authored-by: Matthias Hörtenhuber <mashehu@users.noreply.github.com>
Co-authored-by: Pontus Freyhult <pontus_github@soua.net>
Co-authored-by: Cormac Kinsella <27350062+CormacKinsella@users.noreply.github.com>
Reword the module-detection rationale to state the actual mechanical
reason (task.accelerator is readable inside the process body, a label
is not) and decouple it from the separate question of whether GPU use
is opt-in by default. Simplify the institutional profile GPU example
to a single process scope instead of a withLabel block, drop the
profile-order note and duplicated accelerator-map explanation, and
clean up stray blank lines.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@mahesh-panchal

Copy link
Copy Markdown
Member Author

@nf-core-bot fix linting

@pontus pontus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work, thanks!

The distinction is already covered by the "SLURM: --gres vs --gpus" note earlier in the GPU resource requests section.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Comment thread sites/docs/src/content/docs/developing/components/gpu-modules.md Outdated
Co-authored-by: Cormac Kinsella <27350062+CormacKinsella@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants