Apply package-owned device tree overlays to m1n1 stage 2

.github/workflows/tests.yml

@@ -55,7 +55,7 @@ jobs:
             "$ARCH_IMAGE" sleep infinity
           docker exec arch pacman -Syu --noconfirm --needed \
             git jq lua python python-yaml nodejs npm ripgrep shellcheck imagemagick ffmpeg kitty \
-            gum plocate openssh unzip bc libarchive libxkbcommon desktop-file-utils glib2 sqlite
+            gum plocate openssh unzip bc libarchive libxkbcommon desktop-file-utils glib2 sqlite dtc
           docker exec arch bash -c 'echo "en_US.UTF-8 UTF-8" >/etc/locale.gen && locale-gen'
           docker exec arch useradd --create-home --uid "$(id -u)" tester
 

packages/omarchy-mac/boot/README.md

@@ -8,6 +8,12 @@ While first boot keeps the splash up, a laptop with its lid open shows it on the
 
 The implementation was moved from runtime integration `e82fe3c8850b184dfa22eefd571646ad85446122`, preserving Marcelo's provenance recorded in the runtime source-port ledger. The payload in `files/` comes from the boot recipe paired with #503 (omarchy-mac/omarchy-pkgs-aarch64#65 at `617a907f99a1`) with maralcbr/omarchy-pkgs#202 applied (omarchy-mac-boot 20260921-10: the owner's keyboard layout and dock keyboards at the disk prompt). The owner wizard, recovery-slot operations, journal/retry handling and shared Limine menu/hash operations remain core code. Hardware setup reaches its setup leaves through `omarchy-lifecycle-dispatch setup-boot [image-first-boot]` (`entrypoints/setup-boot`: `setup/grub-console.sh`, then `setup/limine-boot.sh`, which on an image's first boot asks for the deferred rebuild instead of building the UKI); the image builder and the migration engine source them directly. `entrypoints/update-takeover <path>...` answers an update that wants to move aside files no package owns (the step 6 seam asks before it moves anything): read-only, it refuses the whole takeover when any path is the Mac's boot chain, `/usr/lib/initcpio` or pacman configuration, which its image and this package write, or an mkinitcpio file whose own `HOOKS` carry the `asahi` or busybox `encrypt` hook (a legacy Mac's unlock), and lets anything else go, including the memory, USB and mkinitcpio drop-ins omarchy-settings ships on aarch64 too. Owner provisioning, factory reset and `omarchy-drive-password` reach this package only through `omarchy-lifecycle-dispatch`, which runs the `entrypoints/` staged in `/usr/lib/omarchy/mac-boot` and fails on Apple Silicon when they are missing.
 
+## Device tree overlays
+
+A package can add a device tree node that the kernel's device trees do not have yet. Any package may ship one: a new SoC or a new device needs only a new `.dtbo` in that package's file list, never a change to omarchy-mac-boot. The package ships a compiled overlay as `/usr/share/omarchy-platform/dtb-overlays/PREFIX/NAME.dtbo`, the platform root the rest of quattro uses (`/usr/share/omarchy-platform`). `PREFIX` selects the device trees: `t8103` selects every `t8103-*.dtb`, and `t6001-j316c` selects `t6001-j316c.dtb` only. `/etc/default/update-m1n1` sources `lib/dtb-overlays.sh`. When an overlay applies to one of the newest kernel's device trees, it sets `DTBS` to that kernel's device trees, with overlaid copies in `/run/omarchy-dtb-overlays` in place of the originals. The kernel files do not change, so pacman's mtree check still holds. The overlays apply in C order of `PREFIX/NAME`. An overlay whose root node has the string list `omarchy,skip-if-compatible` does not apply to a device tree that already has an available node with one of those compatibles (Linux's `of_device_is_available()`: no `status`, `okay` or `ok`). So a kernel that adds the node takes over from the overlay. An overlay whose root node has the string `omarchy,opt-in` applies only when that string is a line of `/etc/omarchy-platform/dtb-overlays.opt-in`. Use it for hardware whose driver must not start until the owner chooses it. If an overlay fails to apply, or dtc cannot read the result, that device tree stays as the kernel shipped it. With no overlays, `DTBS` keeps update-m1n1's default, and `boot.bin` does not change. Overlays need `dtc` (`dtc`, `fdtoverlay` and `fdtget`). The package that ships an overlay depends on it.
+
+`omarchy-apple-silicon-boot-check` applies the same overlays when it rebuilds `boot.bin` for its comparison. Each overlay must belong to a package, or the check fails. It reads `/etc/default/update-m1n1` with `OMARCHY_DTB_OVERLAYS=0`, so reading the configuration builds nothing. `95-omarchy-mac-dtb-overlays.hook` runs `update-m1n1` when a package adds, changes or removes an overlay, as `95-m1n1-install.hook` does after a kernel update. So the next boot and the boot check read the same device trees.
+
 ## Temporary adaptations
 
 - Deploy ARM64 Limine to U-Boot's `EFI/BOOT/BOOTAA64.EFI` slot until the shared Limine installer supports that target.

packages/omarchy-mac/boot/bin/omarchy-apple-silicon-boot-check

@@ -114,18 +114,47 @@ for package in "${rivals[@]}"; do
   [[ $package == "$kernel" || $package == "$bootloader" ]] || ! is_installed "$package" ||
     fail "$kernel boots with $bootloader, but $package is installed too"
 done
+# True when every size or checksum mismatch pacman reports on a modules.* file
+# of this kernel is what depmod itself writes: a fresh depmod run over the
+# installed modules reproduces the file byte for byte. depmod regenerates
+# modules.dep and its siblings whenever a module is installed or removed, so
+# after any DKMS module they no longer match the package mtree; what it does
+# not regenerate (modules.builtin, modules.builtin.modinfo) never passes.
+depmod_maps_match() {
+  local fresh=$workdir/depmod kver line rel
+  local flagged
+  flagged=$(grep -E '^warning: [^:]+: /usr/lib/modules/[^/]+/modules\.[A-Za-z.]+ \((Size|SHA256 checksum) mismatch\)$' <<<"$1") || return 0
+  kver=${flagged#*/usr/lib/modules/}
+  kver=${kver%%/*}
+  mkdir -p "$fresh"
+  if [[ -n $root ]]; then
+    depmod -b "$root" -o "$fresh" "$kver" 2>/dev/null || return 1
+  else
+    depmod -o "$fresh" "$kver" 2>/dev/null || return 1
+  fi
+  while IFS= read -r line; do
+    rel=${line#warning: *: }
+    rel=${rel%% (*}
+    cmp -s "$fresh/lib/modules/${rel#/usr/lib/modules/}" "$(path "$rel")" || return 1
+  done <<<"$flagged"
+}
+
 # The mtree check, minus the depmod outputs: the kernel package ships
-# modules.* and the depmod hook rewrites them on every install, so their
-# modification times never match. Anything else altered or missing fails.
+# modules.* and the depmod hook rewrites them on every install and after any
+# DKMS module, so their modification times never match and their size and
+# checksum match only while they are depmod's own (depmod_maps_match).
+# Anything else altered or missing fails.
 for package in "$kernel" "$bootloader"; do
   set +e
   mtree_output=$(LC_ALL=C pacman -Qkk "$package" 2>&1)
   mtree_status=$?
   set -e
+  modules_drop='\(Modification time mismatch\)$'
+  depmod_maps_match "$mtree_output" && modules_drop='\((Modification time|Size|SHA256 checksum) mismatch\)$'
   # A fresh image ships no sync databases; pacman warns about each on stderr.
   altered=$(grep -Ev ' altered files$' <<<"$mtree_output" |
     grep -Ev "^warning: database file for '[^']+' does not exist" |
-    grep -Ev '^warning: [^:]+: /usr/lib/modules/[^/]+/modules\.[A-Za-z.]+ \(Modification time mismatch\)$' || true)
+    grep -Ev "^warning: [^:]+: /usr/lib/modules/[^/]+/modules\.[A-Za-z.]+ $modules_drop" || true)
   # --boot-chain holds only the files the boot files are built from against
   # the mtree: the kernel image, its device trees and m1n1. Drift in any other
   # file of the package is left out; everything else pacman reports counts.
@@ -513,6 +542,13 @@ if (( has_crypttab_root || busybox_luks )) && [[ -f $host_vconsole ]]; then
   fi
 fi
 
+# Package-owned device tree overlays, applied the way /etc/default/update-m1n1
+# applies them (dtb-overlays.sh).
+overlay_lib=$(dirname -- "$(realpath -- "$0")")/../lib/omarchy-mac/boot/dtb-overlays.sh
+[[ -r $overlay_lib ]] || overlay_lib=$(dirname -- "$(realpath -- "$0")")/../lib/dtb-overlays.sh
+# shellcheck source=../lib/dtb-overlays.sh
+source "$overlay_lib"
+
 # m1n1 stage 2, built the way this update-m1n1 builds it.
 script=$(path /usr/bin/update-m1n1)
 [[ -r $script ]] || fail "/usr/bin/update-m1n1 is not installed"
@@ -550,7 +586,7 @@ done
 SOURCE="" M1N1="" U_BOOT="" CONFIG="" DTBS="" disabled=""
 config_file=$(path /etc/default/update-m1n1)
 if [[ -e $config_file ]]; then
-  env -i PATH=/usr/bin:/bin bash --noprofile --norc -c '
+  env -i PATH=/usr/bin:/bin OMARCHY_DTB_OVERLAYS=0 bash --noprofile --norc -c '
     unset DTBS SOURCE M1N1 U_BOOT CONFIG M1N1_UPDATE_DISABLED
     . "$1" >/dev/null 2>&1 || exit 1
     for name in DTBS SOURCE M1N1 U_BOOT CONFIG M1N1_UPDATE_DISABLED; do
@@ -650,6 +686,33 @@ for dtb in "${dtbs[@]}"; do
   grep -Fxq -- "$owned" "$workdir/kernel-files" || fail "device tree $dtb is not owned by $kernel"
 done
 
+# The file each device tree is read from: the kernel's, or the copy that
+# carries the package-owned overlays update-m1n1 applies to it.
+paths=()
+for dtb in "${dtbs[@]}"; do
+  paths+=("$root$dtb")
+done
+export OMARCHY_DTB_OVERLAYS_ROOT=$root
+mapfile -t overlays < <(dtb_overlays_list)
+if (( ${#overlays[@]} )); then
+  for overlay in "${overlays[@]}"; do
+    pacman -Qoq "${overlay#"$root"}" >/dev/null 2>&1 ||
+      fail "device tree overlay ${overlay#"$root"} is not owned by a package"
+  done
+  if ! dtb_overlays_supported "$script"; then
+    echo "Apple Silicon boot check: warning: /usr/bin/update-m1n1 has a DTBS default the device tree overlays do not follow, so it leaves them out" >&2
+  elif ! dtb_overlays_tools; then
+    echo "Apple Silicon boot check: warning: device tree overlays are installed, but update-m1n1 leaves them out without dtc 1.7.1 or newer (dtc, fdtoverlay and fdtget); install or update dtc" >&2
+  else
+    mkdir "$workdir/dtbs"
+    mapfile -t paths < <(dtb_overlays_apply "$workdir/dtbs" "${paths[@]}")
+  fi
+fi
+declare -A dtb_file=()
+for i in "${!dtbs[@]}"; do
+  dtb_file[${dtbs[$i]}]=${paths[$i]}
+done
+
 m1n1_options() {
   local line
   [[ -e $root$CONFIG ]] || return 0
@@ -667,7 +730,7 @@ build_image() {
   local image=$1 dtb paths=()
   shift
   for dtb; do
-    paths+=("$root$dtb")
+    paths+=("${dtb_file[$dtb]}")
   done
   {
     cat "$root$M1N1" "${paths[@]}"
@@ -686,7 +749,7 @@ dtb_order() {
   local image=$1 offset=$2 end magic size hash dtb
   local -A by_hash=() left=()
   for dtb in "${dtbs[@]}"; do
-    hash=$(sha256sum <"$root$dtb")
+    hash=$(sha256sum <"${dtb_file[$dtb]}")
     hash=${hash%% *}
     by_hash[$hash]=$dtb
     left[$hash]=$(( ${left[$hash]:-0} + 1 ))

packages/omarchy-mac/boot/files/etc/default/update-m1n1

@@ -5,3 +5,10 @@
 # hook inherits the caller's. The C locale gives the same boot.bin however it
 # is rebuilt, which the boot check can then reproduce byte for byte.
 export LC_ALL=C
+
+# Package-owned device tree overlays under /usr/share/omarchy-platform/dtb-overlays
+# (dtb-overlays.sh). With none installed, DTBS keeps update-m1n1's default.
+if [ -r /usr/lib/omarchy-mac/boot/dtb-overlays.sh ]; then
+  . /usr/lib/omarchy-mac/boot/dtb-overlays.sh
+  dtb_overlays_update_m1n1 || :
+fi

packages/omarchy-mac/boot/files/usr/share/libalpm/hooks/95-omarchy-mac-dtb-overlays.hook

@@ -0,0 +1,14 @@
+# A package added, changed or removed a device tree overlay: rebuild m1n1
+# stage 2 the way 95-m1n1-install.hook does after a kernel update, so the
+# next boot and the boot check read the same device trees.
+[Trigger]
+Type = Path
+Operation = Install
+Operation = Upgrade
+Operation = Remove
+Target = usr/share/omarchy-platform/dtb-overlays/*
+
+[Action]
+Description = Updating m1n1 image for device tree overlays...
+When = PostTransaction
+Exec = /usr/bin/update-m1n1

packages/omarchy-mac/boot/lib/dtb-overlays.sh

@@ -0,0 +1,174 @@
+# shellcheck shell=dash
+# Package-owned device tree overlays for m1n1 stage 2 (sh with local).
+#
+# A package that adds hardware support the kernel's device trees lack ships a
+# compiled overlay as /usr/share/omarchy-platform/dtb-overlays/PREFIX/NAME.dtbo.
+# It applies to every device tree whose file name is PREFIX.dtb or starts with
+# PREFIX- ("t8103" covers every M1 board, "t6001-j316c" one board). A device
+# tree takes its overlays in C order of PREFIX/NAME. An overlay whose root node
+# has the string list "omarchy,skip-if-compatible" is left out of a device tree
+# that already has an available node with one of those compatibles (Linux's
+# of_device_is_available(): no status, "okay" or "ok"), so a kernel that gains
+# the node wins. An overlay whose root node has the string "omarchy,opt-in"
+# applies only when that string is a line of
+# /etc/omarchy-platform/dtb-overlays.opt-in: the owner's choice for hardware
+# whose driver must not start by default. When an overlay does not apply, or
+# dtc cannot read the result, that device tree stays as the kernel shipped it.
+# With no overlays, nothing changes.
+#
+# /etc/default/update-m1n1 calls dtb_overlays_update_m1n1 to set DTBS, and
+# omarchy-apple-silicon-boot-check calls dtb_overlays_apply to rebuild the same
+# image. OMARCHY_DTB_OVERLAYS=0 turns the update-m1n1 side off, so the boot
+# check can read the configuration without building anything.
+# OMARCHY_DTB_OVERLAYS_ROOT prefixes every path read or written (tests, and the
+# boot check's root).
+
+dtb_overlays_dir() {
+  printf '%s\n' "${OMARCHY_DTB_OVERLAYS_ROOT:-}/usr/share/omarchy-platform/dtb-overlays"
+}
+
+# The overlays, one path per line, in the order they apply.
+dtb_overlays_list() {
+  (
+    LC_ALL=C
+    for overlay in "$(dtb_overlays_dir)"/*/*.dtbo; do
+      if [ -f "$overlay" ]; then
+        printf '%s\n' "$overlay"
+      fi
+    done
+  )
+}
+
+# dtc, fdtoverlay and fdtget, from dtc 1.7.1 or newer: older fdtoverlay gives
+# an existing node a new phandle when an overlay labels it, and every
+# reference to the old one then points nowhere.
+dtb_overlays_tools() {
+  local version
+  command -v dtc >/dev/null 2>&1 && command -v fdtoverlay >/dev/null 2>&1 &&
+    command -v fdtget >/dev/null 2>&1 || return 1
+  # "Version: DTC v1.8.1" on Arch, "Version: DTC 1.6.1" on Debian.
+  version=$(dtc --version 2>/dev/null | sed -n 's/^Version: DTC v\{0,1\}\([0-9]*\)\.\([0-9]*\)\.\([0-9]*\).*/\1 \2 \3/p')
+  # shellcheck disable=SC2086 # split into major minor patch
+  set -- $version
+  [ $# = 3 ] || return 1
+  [ "$1" -gt 1 ] || { [ "$1" = 1 ] && { [ "$2" -gt 7 ] || { [ "$2" = 7 ] && [ "$3" -ge 1 ]; }; }; }
+}
+
+# True when UPDATE_M1N1 has the DTBS default dtb_overlays_update_m1n1
+# reproduces: asahi-scripts with the default Arch Linux ARM adds.
+dtb_overlays_supported() {
+  # shellcheck disable=SC2016 # the literal default line
+  grep -Fqx -- ': ${DTBS:=$(/bin/ls -d /lib/modules/*-ARCH | sort -rV | head -1)/dtbs/*.dtb}' "$1"
+}
+
+# True when DTB has an available node whose compatible list holds COMPATIBLE.
+# Available is Linux's of_device_is_available(): the node has no status, or
+# its status is "okay" or "ok". A disabled node does not count.
+dtb_overlays_has_compatible() {
+  dtc -q -I dtb -O dts -o - "$1" 2>/dev/null | awk -v compatible="\"$2\"" '
+    {
+      if ($0 ~ /^[[:space:]]*\}[;]?[[:space:]]*$/) {
+        if (matched[depth] && (status[depth] == "" || status[depth] == "okay" ||
+          status[depth] == "ok")) {
+          found = 1
+          exit
+        }
+        delete matched[depth]
+        delete status[depth]
+        depth--
+      } else if ($0 ~ /\{[[:space:]]*$/) {
+        depth++
+      } else if ($0 ~ /^[[:space:]]*compatible[[:space:]]*=/ && index($0, compatible)) {
+        matched[depth] = 1
+      } else if ($0 ~ /^[[:space:]]*status[[:space:]]*=/ && match($0, /"[^"]*"/)) {
+        status[depth] = substr($0, RSTART + 1, RLENGTH - 2)
+      }
+    }
+    END { exit found ? 0 : 1 }
+  '
+}
+
+# Writes OUT: DTB with every overlay in OVERLAYS (newline-separated) that
+# applies to it. Returns 1, and writes nothing, when none applies or one fails.
+dtb_overlays_build() {
+  local dtb="$1" out="$2" overlays="$3" name="${1##*/}" applied=0 overlay prefix skip compatible key
+  local opt_in="${OMARCHY_DTB_OVERLAYS_ROOT:-}/etc/omarchy-platform/dtb-overlays.opt-in"
+  cp -- "$dtb" "$out.base" || return 1
+  for overlay in $overlays; do
+    prefix=${overlay%/*}
+    prefix=${prefix##*/}
+    case "$name" in
+      "$prefix.dtb" | "$prefix"-*) ;;
+      *) continue ;;
+    esac
+    skip=0
+    for key in $(fdtget -t s "$overlay" / omarchy,opt-in 2>/dev/null); do
+      grep -Fqx -- "$key" "$opt_in" 2>/dev/null || skip=1
+    done
+    for compatible in $(fdtget -t s "$overlay" / omarchy,skip-if-compatible 2>/dev/null); do
+      if dtb_overlays_has_compatible "$out.base" "$compatible"; then
+        skip=1
+      fi
+    done
+    [ "$skip" = 0 ] || continue
+    if fdtoverlay -i "$out.base" -o "$out.next" "$overlay" 2>/dev/null &&
+      dtc -q -I dtb -O dtb -o /dev/null "$out.next" 2>/dev/null; then
+      mv -f -- "$out.next" "$out.base"
+      applied=1
+    else
+      echo "dtb-overlays: $overlay does not apply to $name; $name stays as the kernel shipped it" >&2
+      rm -f -- "$out.next" "$out.base"
+      return 1
+    fi
+  done
+  if [ "$applied" = 0 ]; then
+    rm -f -- "$out.base"
+    return 1
+  fi
+  mv -f -- "$out.base" "$out"
+}
+
+# Prints each DTB, or the copy of it in OUTDIR that carries its overlays, one
+# per line and in the same order.
+dtb_overlays_apply() {
+  local outdir="$1" overlays dtb
+  shift
+  overlays=$(dtb_overlays_list)
+  if [ -z "$overlays" ] || ! dtb_overlays_tools; then
+    overlays=""
+  fi
+  for dtb in "$@"; do
+    if [ -n "$overlays" ] && dtb_overlays_build "$dtb" "$outdir/${dtb##*/}" "$overlays"; then
+      printf '%s\n' "$outdir/${dtb##*/}"
+    else
+      printf '%s\n' "$dtb"
+    fi
+  done
+}
+
+# Sets DTBS for update-m1n1 when an overlay applies to one of the newest
+# kernel's device trees; otherwise leaves DTBS as it was.
+dtb_overlays_update_m1n1() {
+  local root="${OMARCHY_DTB_OVERLAYS_ROOT:-}" modules outdir list="" path
+  outdir=$root/run/omarchy-dtb-overlays
+  [ "${OMARCHY_DTB_OVERLAYS:-1}" != 0 ] || return 0
+  [ -z "${DTBS:-}" ] || return 0
+  [ -n "$(dtb_overlays_list)" ] || return 0
+  if ! dtb_overlays_supported "$root/usr/bin/update-m1n1"; then
+    echo "dtb-overlays: /usr/bin/update-m1n1 has a DTBS default this does not reproduce; device tree overlays are not applied" >&2
+    return 0
+  fi
+  if ! dtb_overlays_tools; then
+    echo "dtb-overlays: device tree overlays need dtc 1.7.1 or newer (dtc, fdtoverlay and fdtget); install or update dtc" >&2
+    return 0
+  fi
+  modules=$(/bin/ls -d "$root"/lib/modules/*-ARCH | sort -rV | head -1)
+  rm -rf -- "$outdir"
+  mkdir -p -- "$outdir" || return 0
+  for path in $(dtb_overlays_apply "$outdir" "$modules"/dtbs/*.dtb); do
+    list="$list $path"
+  done
+  case "$list" in
+    *" $outdir/"*) DTBS=${list# } ;;
+  esac
+}