Skip to content

kata-install: skip osbuilder on peer-pods clusters #2598

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
snir911 merged 1 commit into from
Sep 30, 2026
Merged

kata-install: skip osbuilder on peer-pods clusters #2598

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 35 additions & 4 deletions scripts/kata-install/osc-kata-install.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -280,10 +280,41 @@ install_kata() {
# builds a host-kernel-derived initrd and symlinks the kernel. When
# packages were already present or the RPM postinstall scriptlet failed
# silently inside the DaemonSet chroot, these files may be missing.
if ! chroot /host test -e /var/cache/kata-containers/osbuilder-images/kata.kernel; then
echo "kata VM kernel/initrd missing, running kata-osbuilder.sh"
chroot /host mkdir -p /var/cache/kata-containers/osbuilder-images
chroot /host /usr/libexec/kata-containers/osbuilder/kata-osbuilder.sh
#
# SHORT-TERM FIX: skip osbuilder on IBM Cloud peer-pods deployments.
#
# On IBM Cloud workers, kata-remote (peer-pods) is the only runtime in
# use. The local kata VM initrd is never needed — kata-agent runs inside
# the remote peer pod VSI, not on the worker. However, kata-osbuilder.sh
# fails on IBM Cloud RHCOS workers because the kata-agent binary shipped
# in kata-containers-3.25.0-7.rhaos4.20/21.el9 is a dynamically linked
# PIE executable, and strip(1) corrupts it in the chroot environment.
# This causes permanent CrashLoopBackOff on every pod restart.
#
# KNOWN LIMITATION: this guard is IBM Cloud-specific. It does not protect
# other peer-pods-only cloud deployments (AWS, Azure) that have the same
# strip bug. Those providers are not affected today because they do not
# yet use the DaemonSet install path, but this should be revisited.
#
# LONG-TERM: two upstream changes are needed to remove this guard:
# 1. Fix strip(1) corrupting the dynamically linked kata-agent in
# rootfs.sh (kata-containers/kata-containers). The RPM should either
# ship the binary pre-stripped, or rootfs.sh must handle strip
# failure non-fatally. This same bug also affects
# kata-osbuilder-generate.service at boot — that service will fail
# with the same error on every node reboot until the strip bug is
# fixed, even with this guard in place.
# 2. When enablePeerPods is true and no local kata runtime class is
# needed, the controller should not create the kata/kata-nvidia-gpu
# RuntimeClasses, and the kata-osbuilder-generate.service should not
# be enabled. The local kata VM stack is unnecessary overhead on
# pure peer-pods deployments.
if [[ "${CLOUD_PROVIDER:-}" != "ibmcloud" ]]; then
Comment thread
c3d marked this conversation as resolved.
if ! chroot /host test -e /var/cache/kata-containers/osbuilder-images/kata.kernel; then
echo "kata VM kernel/initrd missing, running kata-osbuilder.sh"
chroot /host mkdir -p /var/cache/kata-containers/osbuilder-images
chroot /host /usr/libexec/kata-containers/osbuilder/kata-osbuilder.sh
fi
Comment on lines +312 to +317

@coderabbitai coderabbitai Bot Jul 29, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Guard on PEER_PODS, not CLOUD_PROVIDER.

controllers/daemonset_reconcile.go sets PEER_PODS=true when peer pods are enabled, but this block ignores it. As written, IBM Cloud non-peer-pods skip required local initrd generation, while peer-pods deployments on other providers still run kata-osbuilder.sh.

Proposed fix 
-if [[ "${CLOUD_PROVIDER:-}" != "ibmcloud" ]]; then
+if [[ "${PEER_PODS:-}" != "true" ]]; then
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if [[ "${CLOUD_PROVIDER:-}" != "ibmcloud" ]]; then
if ! chroot /host test -e /var/cache/kata-containers/osbuilder-images/kata.kernel; then
echo "kata VM kernel/initrd missing, running kata-osbuilder.sh"
chroot /host mkdir -p /var/cache/kata-containers/osbuilder-images
chroot /host /usr/libexec/kata-containers/osbuilder/kata-osbuilder.sh
fi
if [[ "${PEER_PODS:-}" != "true" ]]; then
if ! chroot /host test -e /var/cache/kata-containers/osbuilder-images/kata.kernel; then
echo "kata VM kernel/initrd missing, running kata-osbuilder.sh"
chroot /host mkdir -p /var/cache/kata-containers/osbuilder-images
chroot /host /usr/libexec/kata-containers/osbuilder/kata-osbuilder.sh
fi
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/kata-install/osc-kata-install.sh` around lines 312 - 317, Update the
outer condition around the kata-osbuilder generation block to guard on the
PEER_PODS setting rather than CLOUD_PROVIDER. Ensure non-peer-pods deployments,
including IBM Cloud, run the existing kernel/initrd existence check and
kata-osbuilder.sh flow, while peer-pods deployments skip it.

fi

# Set label before CRI-O restart (restart kills this pod).
Expand Down