Skip to content

libnss_pwb symlink is deleted by ldconfig during R install, breaking pwb NSS module in all workbench-session images #160

Closed

Description

@ianpittwood

Summary

The libnss_pwb NSS module wiring in workbench-session images is broken: the dangling multiarch symlink created at build time is deleted by ldconfig during the R install step, on every image built since the feature was added (#7b18044). As a result, the "rootless in-session identity resolution" feature has never actually been active in any shipped workbench-session image — it silently fails the same way the intended dormant/fallback state does, so it wasn't caught by functional testing.

This was surfaced by bakery dgoss run --matrix-versions only --latest --image-platform linux/amd64 on the deps/session-image-packages branch, which adds a Standard variant to workbench-session. The existing goss test happened to become meaningful for the first time on that variant.

Root cause

workbench-session/template/Containerfile.ubuntu{2204,2404}.jinja2 contains:

### Enable the pwb NSS module for rootless in-session identity resolution ###
# libnss_pwb.so is delivered at runtime by the workbench-session-init container
# into /usr/lib/rstudio-server/bin/noble/. Create the (build-time dangling)
# multiarch symlink + nsswitch entries so glibc loads it once that volume is
# populated. Dormant-safe: with no workbench-nss.conf the module returns
# Unavail and resolution falls through to `files`.
RUN case "${TARGETARCH}" in \
      amd64) TRIPLET=x86_64-linux-gnu ;; \
      arm64) TRIPLET=aarch64-linux-gnu ;; \
      *) echo "unsupported arch ${TARGETARCH}"; exit 1 ;; \
    esac && \
    ln -sf /usr/lib/rstudio-server/bin/noble/libnss_pwb.so \
           "/usr/lib/${TRIPLET}/libnss_pwb.so.2" && \
    ...

The symlink target (/usr/lib/rstudio-server/bin/noble/libnss_pwb.so) does not exist at build time — it's meant to be populated by a runtime mount from workbench-session-init. Several build steps later, RUN install of R (rstd.io/r-install) installs a .deb via apt-get install ./r-*.deb, which triggers libc-bin's ldconfig postinst trigger. ldconfig deletes symlinks in the multiarch lib directories whose target does not exist at all (as opposed to an existing-but-empty target, which it merely warns about). This silently removes /usr/lib/<triplet>/libnss_pwb.so.2 before the image is even finished building.

I confirmed this two ways:

  1. Extracted the built workbench-session image layers (docker save + tar inspection) and found a .wh.libnss_pwb.so.2 whiteout entry in the same layer as the R-install RUN command, in both the Minimal and Standard variants.
  2. Isolated repro on bare ubuntu:24.04:
    ln -sf /usr/lib/rstudio-server/bin/noble/libnss_pwb.so /usr/lib/x86_64-linux-gnu/libnss_pwb.so.2
    RUN_UNATTENDED=1 R_VERSION=4.6.1 bash -c "$(curl -fsSL https://rstd.io/r-install)"
    ls -la /usr/lib/x86_64-linux-gnu/libnss_pwb.so.2   # No such file or directory
    Creating an empty placeholder file at the target path first (touch /usr/lib/rstudio-server/bin/noble/libnss_pwb.so) before the symlink prevents the deletion — ldconfig then only logs File ... is empty, not checked. and leaves the symlink in place.

Why the goss test didn't catch this until now

workbench-session/matrix/test/goss.yaml has:

"Verify libnss_pwb symlink present current architecture":
  exec: 'dpkg-architecture -qDEB_HOST_MULTIARCH | xargs -I{} test -L "/usr/lib/{}/libnss_pwb.so.2"'

dpkg-architecture is not installed in the Minimal variant, so the left side of the pipe produces no output, xargs never invokes test -L, and the command exits 0 — the test vacuously passes without checking anything. It only became a real assertion on the new Standard variant (which pulls in dpkg-dev transitively via one of the added -dev packages), where it correctly fails.

Suggested fix

In the same RUN block, create a placeholder regular file at the symlink target before creating the symlink, so ldconfig sees an existing (if empty) target instead of a fully dangling one:

    esac && \
    mkdir -p /usr/lib/rstudio-server/bin/noble && \
    touch /usr/lib/rstudio-server/bin/noble/libnss_pwb.so && \
    ln -sf /usr/lib/rstudio-server/bin/noble/libnss_pwb.so \
           "/usr/lib/${TRIPLET}/libnss_pwb.so.2" && \
```//jammy" for the 22.04 template.

Separately, the Minimal-variant goss test should be made to actually assert something regardless of whether `dpkg-architecture` is present (e.g. compute the triplet via `uname -m`/a static case statement instead of relying on `dpkg-dev`), so it doesn't silently mask regressions like this one again.

## Affected files

- `workbench-session/template/Containerfile.ubuntu2204.jinja2`
- `workbench-session/template/Containerfile.ubuntu2404.jinja2`
- (rendered) `workbench-session/matrix/Containerfile.ubuntu{2204,2404}.{min,std}`
- `workbench-session/matrix/test/goss.yaml` (+ its template) for the Minimal-variant test gap

Activity

  1. ianpittwood commented on Jul 9, 2026

    @ianpittwood
    ContributorAuthor

    @ssinnott this bug popped up as part of #113. Thoughts on the suggested fix? A placeholder file does sound like it would ensure the symlink is properly created for session-init to rewrite the original file.

  2. added
    cvp:2Quality of life improvements, or rough edge mitigations that impact one or more user personas
    tdp:2The team notices.
    on Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingcvp:2Quality of life improvements, or rough edge mitigations that impact one or more user personasdockerRelated to container images we producetdp:2The team notices.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions