kernel: three C fixes - #619
Open
cyclistmass wants to merge 3 commits into
Open
cyclistmass wants to merge 3 commits into
cyclistmass wants to merge 3 commits into
Conversation
walk_stack_frames (albt.c at 84021ff) advances through the control stack by interpreting a non-frame word. Two of its arms follow that word as a POINTER: } else if ((header & fixnummask) == 0) { next = (lisp_frame *)header; } else if (header == stack_alloc_marker) { next = (lisp_frame *)(current[1]); Zero satisfies (header & fixnummask) == 0, and a control stack is full of zero words. So next becomes NULL, the loop assigns start = NULL, the `while (start < end)' test passes, and lisp_frame_p(NULL) dereferences address 0. The kernel backtrace printer therefore faults while printing the backtrace for another fault, and the report that would have identified the original defect is lost. Any small fixnum reaches the same end by a different address, and a candidate below `start' either walks backwards or spins forever, so the test needed is not "is this fixnum-tagged" but "is this a forward address inside the region being walked". advance_or_stop() applies that test to both arms. A candidate is accepted only when start < candidate <= end; anything else ends the walk with a diagnostic naming the rejected value. A truncated backtrace is useful. A SIGSEGV in the debugger destroys the only evidence the crash produced. Also in the same function: the "Bad frame!" diagnostic printed a pointer with %x, which truncates it to 32 bits on every 64-bit target. A truncated address in a crash report is worse than no address, because it looks like data. It is now %p. This file is compiled for androidarm, darwinarm, linuxarm and linuxarm64 (the four Makefiles that list albt.o), and it is linked into the shipped kernel rather than a separate debug tool -- linuxarm64/Makefile:69 has ../../arm64cl depend on $(DEBUGOBJ). The defect and the fix are therefore live for ARM32 as well as arm64, and nothing here is arm64-specific. ;;; ARM64-DEVIATION: none. This is shared ARM-family kernel C. OBSERVED, not inferred. A crash report carries the fault itself, in a core dump, with the NULL visible in the frame arguments: Clozure#7 lisp_frame_p (spPtr=spPtr@entry=0x0) at ../arm64-exceptions.c Clozure#8 walk_stack_frames (start=0x0, start@entry=0xffff829fc1f0, end=0xffff82a00000) at ../albt.c `start' entered the function as 0xffff829fc1f0 and reached the lisp_frame_p call as 0x0, which is one iteration of the loop assigning next to start. albt.c is that call. So the zero word, the NULL and the dereference are all present in one captured instance. The report's original fault was elsewhere -- in the GC, marking a tstack area -- and it is still unexplained. This defect is why that fault has no usable backtrace: the printer died inside the crash it was reporting, and turned an unhandled SIGSEGV into a SIGABRT with nothing to read. Verified on linuxarm64 with the Makefile's own command, read out of `make -n albt.o' rather than guessed: pristine rc=0 0 warnings albt.o = 25896 bytes patched rc=0 0 warnings albt.o = 29176 bytes The object grows, which is how we know the new code is compiled in and not folded away. Under -Wall -Wextra -Wno-unused-parameter both files emit ONE warning and the set difference between them is empty, so the patch adds no diagnostic. Two positive controls were watched failing IN THE SUBJECT FILE: a broken paren in the new helper (errors) and a void return type on the helper (`void value not ignored as it ought to be' at both call sites, :120), the second of which also proves both arms really call it. NOT verified: no kernel has been linked and booted with this change, and the fault has not been reproduced on demand -- the observation above is a captured instance, not a repro. The forward-progress bound is also the minimum fix; it does not validate that a candidate points at a real frame, only that following it cannot leave the region.
…piler that implements it
SUPPORT_PRAGMA_UNUSED is #ifdef'd in ten places and #define'd in none, so the
`#pragma unused(...)' annotations it guards have never reached any compiler:
$ git grep -c SUPPORT_PRAGMA_UNUSED -- 'lisp-kernel/*.c'
lisp-kernel/arm-exceptions.c (936, 1002, 1305)
lisp-kernel/arm64-exceptions.c (1114, 1196, 1825)
lisp-kernel/ppc-exceptions.c (983, 1146, 1367, 1540)
$ git grep -n 'define[[:space:]]*SUPPORT_PRAGMA_UNUSED'
(nothing)
Each site says that a parameter forced on the function by a handler-dispatch
signature is deliberately unused -- do_hard_stack_overflow needs only xp,
do_spurious_wp_fault needs none of the three, allocate_no_stack needs none. The
intent is right and recorded; only the channel to the compiler is dead.
=== MEASURED, because #pragma unused is not a thing to reason about from memory
It is a Metrowerks/MPW-lineage extension and gcc and clang do not agree about
it. So it was measured rather than recalled, on the real do_hard_stack_overflow
shape, with a positive control (the same file WITHOUT the pragma, which must
emit `unused parameter' before any other result is interpretable). gcc 11.5.0
20240719 on x86_64 and aarch64, clang 15.0.7 on aarch64:
gcc 11.5.0 clang 15.0.7
recognised? NO YES
"ignoring '#pragma (no diagnostic
unused ' [-Wunknown- at all)
pragmas]"
suppresses `unused parameter'? NO (still 2) YES (2 -> 0)
diagnostics added 1 per site 0
-Wall -Wextra -Werror exit code 1 (2 errors -> 3) 0
Verbatim, gcc, with the pragma present:
B.c: In function 'do_hard_stack_overflow':
B.c: warning: ignoring '#pragma unused ' [-Wunknown-pragmas]
B.c:53: warning: unused parameter 'area' [-Wunused-parameter]
B.c:67: warning: unused parameter 'addr' [-Wunused-parameter]
exit code: 0
Verbatim, clang, same file, same flags: no output, exit 0.
So an UNCONDITIONAL #define would be measurably worse than the dead guard: on
gcc it suppresses nothing and adds one -Wunknown-pragmas at each of the ten
sites, and turns a -Wall -Wextra -Werror build from 2 errors into 3. Hence
clang only.
Two more measurements that bear on the risk, both on clang 15.0.7:
* A pragma naming an identifier that does not exist is DIAGNOSED, not ignored:
"undeclared variable 'x' used as an argument for '#pragma unused'
[-Wignored-pragmas]". clang parses the argument list, so these annotations
stay honest as the code changes -- something -Wno-unused-parameter cannot
do.
* A pragma naming a parameter that IS used is silently accepted: no
diagnostic, exit 0 even under -Werror. That matters here, because
ppc-exceptions.c has `#pragma unused(where)' in handle_uuo and `where'
IS used,. That annotation is stale. Under this patch it
becomes a no-op rather than a new warning, so I have left it alone rather
than widen this patch into a second fix -- but it is wrong and worth
deleting separately.
* At file scope the names are not in scope and clang warns twice and
suppresses nothing. All ten sites are inside a function body, which is
where it works.
=== Where the #define goes
lisp.h, because all three files include it first and this is a property of the
COMPILER, not of a platform -- putting it in the per-platform headers would mean
repeating a __clang__ test in each of them.
=== Relationship to -Wno-unused-parameter on the arm64 kernel
I added -Wno-unused-parameter to lisp-kernel/linuxarm64/Makefile in a separate
arm64 patch, and said there that what to do with the #pragma blocks was your
call. This is that follow-up, and it does NOT make the flag redundant:
linuxarm64/Makefile:11 is `CC = ${CROSS}gcc', so on our own kernel build this
patch changes nothing at all and the flag is still what suppresses the 78
-Wunused-parameter there. Measured, not assumed -- see below, where gcc's
warning census is bit-for-bit the same with and without this patch. The two are
complementary: the pragma covers the compiler that understands it, the flag
covers the one that does not.
=== The alternative, if you would rather not
Delete the ten #ifdef/#pragma/#endif blocks and keep warning suppression at the
build flags. That is a defensible call and a smaller thing to maintain; it just
throws away suppression that measurably works on clang, and clang is the
compiler the Darwin targets use. I have written the version I think is right;
either is easy from here and it is your tree.
=== NOT TESTED
⛔ Of the three files this affects, only arm64-exceptions.c has been compiled.
RED-then-GREEN on the real translation unit, aarch64, tree at the pin:
lisp-kernel/arm64-exceptions.c, -Wall -Wextra, otherwise the Makefile's own
flags (-include ../platform-linuxarm64.h -DLINUX -DARM64 ... -g -O2 -Wno-format).
Three configurations, because the third is the one that justifies "clang only":
unused-param -Wunknown-pragmas total warn .o md5
RED gcc, no macro 19 0 61 96930241
GREEN gcc, no macro 19 0 61 96930241
COUNTER gcc, macro ON 19 3 64 96930241
RED clang, no macro 19 0 23 7d5242da
GREEN clang, macro ON 13 0 17 7d5242da
* clang 19 -> 13. Grouped by parameter name, so that "six warnings went
away" cannot be six DIFFERENT warnings:
addr 4->2 area 2->0 size 1->0 xp 4->3
info 1->1 instruction 1->1 param 1->1 signum 1->1 tcr 4->4
Total -6, and the three #pragma lines in the file name exactly six
identifiers: (area,addr) + (xp,area,addr) + (size) = 2+3+1. Every
parameter NOT named by a pragma is untouched.
* ZERO -Wignored-pragmas on the green clang run -- the check that every
identifier the three pragmas name really is in scope.
* gcc RED vs GREEN is BYTE-IDENTICAL output, because __clang__ is not
defined and the guard stays false. 0159's flag therefore still does all
the work on our own kernel build.
* COUNTER is what an unconditional #define would do to gcc: three
-Wunknown-pragmas, one per site, 61 -> 64 warnings, and not one
unused-parameter removed. Worse than the dead guard, measured rather than
predicted.
* THE .o md5 IS IDENTICAL in all three gcc configurations and in both clang
configurations. So this changes diagnostics and provably nothing else --
no generated code moves, on either compiler, with or without the macro.
* Positive control: the RED clang run emitted 19 > 0, i.e. the instrument was
seen reporting the un-suppressed state before the suppressed one was
believed.
* Reproduced on a second, independent pin tree by a red/green control
script that carries these gates.
NOT COMPILED, at all: arm-exceptions.c (ARM32) and ppc-exceptions.c -- I have no
ARM32 or PPC machine. Their seven sites were READ, and every identifier named
there is a parameter of the enclosing function, so I expect the same result --
but that is inspection, not a build. The one thing inspection did turn up is
ppc-exceptions.c's stale `where', and that one is measured harmless rather
than assumed so.
Confidence: high that the #define is correct for clang and inert for gcc (red
and green both watched, on the real TU); high that it is a no-op for our own
kernel build; medium that the seven un-compiled sites are clean, on inspection
alone.
Signed-off-by: Mauro DiBenedetto <maurodibenedetto@gmail.com>
In both arms of recursive_lock_trylock (futex and non-futex), when the calling thread already owns the lock and was_free is NULL, the owner branch increments m->count and then falls through into the store_conditional on m->avail. That store_conditional fails, because the caller already holds the lock, so the function returns EBUSY with the count already raised. The caller believes it never got the lock and will not unlock it, so the extra count is never undone and the lock's count can never return to zero: the lock stays owned forever. The early return was conditional on was_free being non-NULL when only the *was_free store should have been. Make the owner branch return 0 unconditionally, storing through was_free only when it is provided. No caller exists in the tree today; the function is exported only through the kernel-import tables. Reported as Clozure/ccl issue Clozure#597.
Member
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
1. The kernel backtrace follows a zero stack word to NULL.
walk_stack_framesadvances through the control stack by interpreting anon-frame word, and two of its arms follow that word as a pointer:
Zero satisfies
(header & fixnummask) == 0, and a control stack holds manyzero words.
nextbecomes NULL, the loop tail doesstart = next, thewhile (start < end)test passes, andlisp_frame_p(NULL)dereferencesaddress 0. The kernel backtrace therefore faults while it prints the backtrace
for another fault, and the report that would identify the original defect is
lost. Any small fixnum reaches the same end at a different address, and a
candidate below
startwalks backwards or spins forever. The test needed isnot "does this word carry a fixnum tag" but "is this a forward address inside the
region under walk".
2.
recursive_lock_trylockleaks a recursive count on EBUSY.In both arms, futex and non-futex, when the calling thread already owns the
lock and
was_freeis NULL, the owner branch incrementsm->countand fallsinto the
store_conditionalonm->avail. That store fails, because thecaller already holds the lock. The function then returns EBUSY with the count
already raised, and the caller never releases a lock it believes it never got.
The early return was conditional on
was_freebeing non-NULL, when only the*was_freestore needed that condition. No caller exists in the tree today,and the kernel-import tables export the function. This is the fix for issue
#597.
3.
SUPPORT_PRAGMA_UNUSEDis tested in ten places and defined in none.The patch defines it for clang only.
Most sites mark a parameter that a handler-dispatch signature forces on the
function and the body does not use. The intent is written down, and only the
channel to the compiler is dead.
Measured on the
do_hard_stack_overflowshape, with a positive control thatcompiles the same source without the pragma and emits
unused parameterasrequired. clang 15.0.7 accepts the pragma and those warnings go to 0. gcc 11.5.0
does not recognize it, so the warnings survive and
-Wunknown-pragmasadds onediagnostic of its own. That is why the define is clang-only: unconditional, it
would suppress nothing on gcc and add a warning per site.
Two boundaries on that. I compiled the arm64 shape, not all ten sites. And one
site is not an unused parameter at all:
handle_uuoinppc-exceptions.cdeclares
#pragma unused(where), then passeswheretohandle_errortwice.That pragma looks simply wrong, and this patch does not change it -- it only
gives the guard a definition on the compiler that implements it.
Verified at this base. Built and run on linuxarm64 at
ec578745with allten patches applied: ANSI 21679 tests, 0 failures.
tests/ccl.lsp243 tests, 0failures. Image
281a49e5, kernel67bb66b4.