fix(startup): warn when an installed systemd unit has drifted from the repo's - #470
fix(startup): warn when an installed systemd unit has drifted from the repo's#470ChuckBuilds wants to merge 1 commit into
Conversation
…e repo's
Nothing re-applies systemd units after the first install. `git pull` -- which
is what the web UI's update button runs -- brings a new template into the
checkout, but no code in web_interface/ or src/ copies it to
/etc/systemd/system, and nothing anywhere runs `systemctl daemon-reload`. The
unit that actually runs is whatever first_time_install.sh wrote on day one.
So every hardening added to a unit is inert on existing installs, silently.
Measured on a live rig:
installed /etc/systemd/system/ledmatrix.service 2026-08-06
template systemd/ledmatrix.service 2026-08-19
contents differ
with the practical result that the MemoryMax=85% the repo's template specifies
was not being enforced at all -- `systemctl show` reported
MemoryMax=infinity. Anyone reading the template would reasonably believe the
service was capped.
Startup now compares each installed unit against its substituted template and
warns when they differ, naming install_service.sh as the remedy.
A warning, not an error, and deliberately not a silent rewrite: editing files
under /etc and restarting services is the installer's job, not something a
display process should do to a machine while it is booting. Making it fatal
would also brick every development checkout whose unit is legitimately absent
or hand-edited.
Comparison ignores comments, blank lines and ordering. The template carries
explanatory comments the installed copy will not have, and systemd does not
care about order within a section, so a literal comparison would warn on every
boot and be ignored within a week.
Mutation-checked three ways: never reporting drift fails, making it fatal
fails, and -- after the first attempt missed it -- comparing raw text now fails
too. That last gap is worth noting: the comment-insensitivity tests originally
exercised the helper directly, so a comparison that stopped calling the helper
passed them all. The test that catches it goes through _validate_systemd_units.
29 startup-validator tests pass.
|
Warning Review limit reached
Next review available in: 29 minutes Limit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?Wait for the limit to reset, then comment An organization admin can change what happens after included review limits in Billing. How do review limits work?CodeRabbit enforces per-developer PR review limits within each organization. For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 11 |
| Duplication | 0 |
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
|
Superseded by #476, which carries this commit unchanged (cherry-picked with |
1 participant
Found while checking whether #469 would actually reach anyone. It wouldn't — and neither has anything else added to a unit file.
Nothing re-applies systemd units after the first install
git pull— what the web UI's update button runs — brings a new template into the checkout. But no code inweb_interface/orsrc/copies it to/etc/systemd/system, and nothing anywhere runssystemctl daemon-reload. The unit that actually runs is whateverfirst_time_install.shwrote on day one.Measured on a live rig:
With a concrete consequence: the **** the repo's template specifies was not being enforced at all. Version=257.13-1~deb13u1
Features=+PAM +AUDIT +SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK +BTF -XKBCOMMON -UTMP +SYSVINIT +LIBARCHIVE
Architecture=arm64
Tainted=unmerged-bin
FirmwareTimestampMonotonic=0
LoaderTimestampMonotonic=0
KernelTimestamp=Wed 1969-12-31 19:00:00 EST
KernelTimestampMonotonic=0
InitRDTimestampMonotonic=0
UserspaceTimestamp=Wed 1969-12-31 19:00:04 EST
UserspaceTimestampMonotonic=4495583
FinishTimestamp=Sun 2026-08-02 16:49:49 EDT
FinishTimestampMonotonic=24320169
ShutdownStartTimestampMonotonic=0
SecurityStartTimestamp=Wed 1969-12-31 19:00:04 EST
SecurityStartTimestampMonotonic=4524765
SecurityFinishTimestamp=Wed 1969-12-31 19:00:04 EST
SecurityFinishTimestampMonotonic=4528411
GeneratorsStartTimestamp=Sun 2026-08-02 16:49:30 EDT
GeneratorsStartTimestampMonotonic=5028048
GeneratorsFinishTimestamp=Sun 2026-08-02 16:49:31 EDT
GeneratorsFinishTimestampMonotonic=5854480
UnitsLoadStartTimestamp=Sun 2026-08-02 16:49:31 EDT
UnitsLoadStartTimestampMonotonic=5854512
UnitsLoadFinishTimestamp=Sun 2026-08-02 16:49:31 EDT
UnitsLoadFinishTimestampMonotonic=6182052
UnitsLoadTimestamp=Sun 2026-08-02 16:49:40 EDT
UnitsLoadTimestampMonotonic=14819125
InitRDSecurityStartTimestampMonotonic=0
InitRDSecurityFinishTimestampMonotonic=0
InitRDGeneratorsStartTimestampMonotonic=0
InitRDGeneratorsFinishTimestampMonotonic=0
InitRDUnitsLoadStartTimestampMonotonic=0
InitRDUnitsLoadFinishTimestampMonotonic=0
LogLevel=info
LogTarget=journal-or-kmsg
NNames=364
NFailedUnits=0
NJobs=0
NInstalledJobs=407
NFailedJobs=0
Progress=1
Environment=LANG=en_GB.UTF-8 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
ConfirmSpawn=no
ShowStatus=yes
UnitPath=/etc/systemd/system.control /run/systemd/system.control /run/systemd/transient /run/systemd/generator.early /etc/systemd/system /etc/systemd/system.attached /run/systemd/system /run/systemd/system.attached /run/systemd/generator /usr/local/lib/systemd/system /usr/lib/systemd/system /run/systemd/generator.late
DefaultStandardOutput=journal
DefaultStandardError=inherit
WatchdogDevice=/dev/watchdog0
WatchdogLastPingTimestamp=Thu 2026-08-20 00:48:06 EDT
WatchdogLastPingTimestampMonotonic=1497505168054
RuntimeWatchdogUSec=1min
RuntimeWatchdogPreUSec=0
RebootWatchdogUSec=2min
KExecWatchdogUSec=0
ServiceWatchdogs=yes
SystemState=running
DefaultTimerAccuracyUSec=1min
DefaultTimeoutStartUSec=1min 30s
DefaultTimeoutStopUSec=1min 30s
DefaultTimeoutAbortUSec=1min 30s
DefaultDeviceTimeoutUSec=1min 30s
DefaultRestartUSec=100ms
DefaultStartLimitIntervalUSec=10s
DefaultStartLimitBurst=5
DefaultCPUAccounting=yes
DefaultBlockIOAccounting=no
DefaultIOAccounting=no
DefaultIPAccounting=no
DefaultMemoryAccounting=yes
DefaultTasksAccounting=yes
DefaultLimitCPU=infinity
DefaultLimitCPUSoft=infinity
DefaultLimitFSIZE=infinity
DefaultLimitFSIZESoft=infinity
DefaultLimitDATA=infinity
DefaultLimitDATASoft=infinity
DefaultLimitSTACK=infinity
DefaultLimitSTACKSoft=8388608
DefaultLimitCORE=infinity
DefaultLimitCORESoft=0
DefaultLimitRSS=infinity
DefaultLimitRSSSoft=infinity
DefaultLimitNOFILE=524288
DefaultLimitNOFILESoft=1024
DefaultLimitAS=infinity
DefaultLimitASSoft=infinity
DefaultLimitNPROC=13226
DefaultLimitNPROCSoft=13226
DefaultLimitMEMLOCK=8388608
DefaultLimitMEMLOCKSoft=8388608
DefaultLimitLOCKS=infinity
DefaultLimitLOCKSSoft=infinity
DefaultLimitSIGPENDING=13226
DefaultLimitSIGPENDINGSoft=13226
DefaultLimitMSGQUEUE=819200
DefaultLimitMSGQUEUESoft=819200
DefaultLimitNICE=0
DefaultLimitNICESoft=0
DefaultLimitRTPRIO=0
DefaultLimitRTPRIOSoft=0
DefaultLimitRTTIME=infinity
DefaultLimitRTTIMESoft=infinity
DefaultTasksMax=3967
DefaultMemoryPressureThresholdUSec=200ms
DefaultMemoryPressureWatch=auto
TimerSlackNSec=50000
DefaultOOMPolicy=stop
DefaultOOMScoreAdjust=0
CtrlAltDelBurstAction=reboot-force
SoftRebootsCount=0 reported . Anyone reading the template would reasonably conclude the service was capped.
The change
Startup compares each installed unit against its substituted template and warns when they differ, naming as the remedy.
A warning, not an error, and deliberately not a silent rewrite. Editing files under and restarting services is the installer's job, not something a display process should do to a machine while it is booting. Making it fatal would also brick every development checkout whose unit is legitimately absent or hand-edited.
Comparison ignores comments, blank lines and ordering — the template carries explanatory comments the installed copy won't have, and systemd doesn't care about order within a section. A literal comparison would warn on every boot and be tuned out within a week.
Relationship to #469
#469 adds
MALLOC_ARENA_MAX=2to the same unit. On its own it only helps new installs. With this, an existing install at least gets told its unit is stale. Fully closing the loop would mean the updater re-runninginstall_service.sh— a bigger change, since it needs root and restarts services, so I'd rather propose it separately than fold it into a detection fix.Verification
7 new tests, 29 passing across the startup validator.
Mutation-checked three ways:
That third one is worth calling out: my first attempt missed it. The comment-insensitivity tests exercised the
_unit_bodyhelper directly, so a comparison that stopped calling the helper passed all of them. The test that catches it goes through_validate_systemd_unitsend-to-end. Same lesson as the threaded-race test in ledmatrix-plugins#303 — a property asserted on a helper is not a property asserted on the code path.🤖 Generated with Claude Code
https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW