Repository navigation
Put cray-mpich's libraries on LD_LIBRARY_PATH in Perlmutter templates - #541
Merged
Merged
Conversation
xylar
added
bug
xylar
changed the title
With craype-accel-nvidia80, the Cray wrappers link the GTL into every executable. Spack clears LD_LIBRARY_PATH, so executables run during a build (such as boost's b2) found the GTL through /opt/cray/pe/lib64, which now points at the default cray-mpich 9.1.0 and needs CUDA 13 rather than the 12.9 the template loads. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
/opt/cray/pe/lib64 links the intel and nvidia MPI libraries, and the GTL, to the default cray-mpich 9.1.0, so executables run during a build either failed to load (libifport.so.5 for intel, libcudart.so.13 wherever the GTL is linked) or silently ran against MPI 4.1 (nvidia). The gnu libraries there still point at 9.0.1, but are included too so the templates keep working when that changes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
xylar
force-pushed
the
fix-pm-gpu-ld-library-path
branch
from
October 23, 2026 16:07
fe8546d to
50df93a
Compare
Collaborator
Author
TestingAll on pm-gpu, with templates identical to this branch:
Posted by Claude Code on @xylar's behalf. The testing, analysis and wording above are AI-authored; please check them accordingly. |
xylar
marked this pull request as ready for review
October 23, 2026 16:43
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On Perlmutter,
/opt/cray/pe/lib64now links the intel and nvidia MPI libraries and cray-mpich's GTL to the default cray-mpich 9.1.0, while the templates use 9.0.1. Since Spack clearsLD_LIBRARY_PATH, executables that Spack builds and runs mid-build pick those up: Boost'sb2fails on gnugpu (libcudart.so.13), anything built with intel fails to load (libifport.so.5), and nvidia builds silently run against MPI 4.1. This puts each template's cray-mpich 9.0.1 lib directory, plus CUDA 12.9's for the GPU templates, ahead of libfabric. The gnu libraries in/opt/cray/pe/lib64still point at 9.0.1, but the gnu templates get the same change so they keep working when that moves.Checklist
Testingcomment, if appropriate, in the PR documents testing used to verify the changesPosted by Claude Code on @xylar's behalf. The testing, analysis and wording above are AI-authored; please check them accordingly.