Skip to content

kernel: three C fixes - #619

Open
cyclistmass wants to merge 3 commits into
Clozure:masterfrom
cyclistmass:kernel-three
Open

cyclistmass wants to merge 3 commits into
Clozure:masterfrom
cyclistmass:kernel-three

Conversation

@cyclistmass

Copy link
Copy Markdown
Contributor

1. The kernel backtrace follows a zero stack word to NULL.
walk_stack_frames advances through the control stack by interpreting a
non-frame word, and two of its arms follow that word as a pointer:

} else if ((header & fixnummask) == 0) {
  next = (lisp_frame *)header;
} else if (header == stack_alloc_marker) {
  next = (lisp_frame *)(current[1]);

Zero satisfies (header & fixnummask) == 0, and a control stack holds many
zero words. next becomes NULL, the loop tail does start = next, the
while (start < end) test passes, and lisp_frame_p(NULL) dereferences
address 0. The kernel backtrace therefore faults while it prints the backtrace
for another fault, and the report that would identify the original defect is
lost. Any small fixnum reaches the same end at a different address, and a
candidate below start walks backwards or spins forever. The test needed is
not "does this word carry a fixnum tag" but "is this a forward address inside the
region under walk".

2. recursive_lock_trylock leaks a recursive count on EBUSY.
In both arms, futex and non-futex, when the calling thread already owns the
lock and was_free is NULL, the owner branch increments m->count and falls
into the store_conditional on m->avail. That store fails, because the
caller already holds the lock. The function then returns EBUSY with the count
already raised, and the caller never releases a lock it believes it never got.

The early return was conditional on was_free being non-NULL, when only the
*was_free store needed that condition. No caller exists in the tree today,
and the kernel-import tables export the function. This is the fix for issue
#597.

3. SUPPORT_PRAGMA_UNUSED is tested in ten places and defined in none.
The patch defines it for clang only.

$ git grep -c SUPPORT_PRAGMA_UNUSED -- 'lisp-kernel/*.c'
lisp-kernel/arm-exceptions.c:3
lisp-kernel/arm64-exceptions.c:3
lisp-kernel/ppc-exceptions.c:4
$ git grep -n 'define[[:space:]]*SUPPORT_PRAGMA_UNUSED'
(nothing)

Most sites mark a parameter that a handler-dispatch signature forces on the
function and the body does not use. The intent is written down, and only the
channel to the compiler is dead.

Measured on the do_hard_stack_overflow shape, with a positive control that
compiles the same source without the pragma and emits unused parameter as
required. clang 15.0.7 accepts the pragma and those warnings go to 0. gcc 11.5.0
does not recognize it, so the warnings survive and -Wunknown-pragmas adds one
diagnostic of its own. That is why the define is clang-only: unconditional, it
would suppress nothing on gcc and add a warning per site.

Two boundaries on that. I compiled the arm64 shape, not all ten sites. And one
site is not an unused parameter at all: handle_uuo in ppc-exceptions.c
declares #pragma unused(where), then passes where to handle_error twice.
That pragma looks simply wrong, and this patch does not change it -- it only
gives the guard a definition on the compiler that implements it.


Verified at this base. Built and run on linuxarm64 at ec578745 with all
ten patches applied: ANSI 21679 tests, 0 failures. tests/ccl.lsp 243 tests, 0
failures. Image 281a49e5, kernel 67bb66b4.

walk_stack_frames (albt.c at 84021ff) advances through the
control stack by interpreting a non-frame word.  Two of its arms follow
that word as a POINTER:

      } else if ((header & fixnummask) == 0) {
        next = (lisp_frame *)header;
      } else if (header == stack_alloc_marker) {
        next = (lisp_frame *)(current[1]);

Zero satisfies (header & fixnummask) == 0, and a control stack is full
of zero words.  So next becomes NULL, the loop assigns start = NULL, the
`while (start < end)' test passes, and lisp_frame_p(NULL) dereferences
address 0.  The kernel backtrace printer therefore faults while printing
the backtrace for another fault, and the report that would have
identified the original defect is lost.

Any small fixnum reaches the same end by a different address, and a
candidate below `start' either walks backwards or spins forever, so the
test needed is not "is this fixnum-tagged" but "is this a forward
address inside the region being walked".

advance_or_stop() applies that test to both arms.  A candidate is
accepted only when start < candidate <= end; anything else ends the walk
with a diagnostic naming the rejected value.  A truncated backtrace is
useful.  A SIGSEGV in the debugger destroys the only evidence the crash
produced.

Also in the same function: the "Bad frame!" diagnostic printed a pointer
with %x, which truncates it to 32 bits on every 64-bit target.  A
truncated address in a crash report is worse than no address, because it
looks like data.  It is now %p.

This file is compiled for androidarm, darwinarm, linuxarm and linuxarm64
(the four Makefiles that list albt.o), and it is linked into the shipped
kernel rather than a separate debug tool -- linuxarm64/Makefile:69 has
../../arm64cl depend on $(DEBUGOBJ).  The defect and the fix are
therefore live for ARM32 as well as arm64, and nothing here is
arm64-specific.

;;; ARM64-DEVIATION: none.  This is shared ARM-family kernel C.

OBSERVED, not inferred. A crash report carries the fault itself, in a
core dump, with the NULL visible in the frame arguments:

  Clozure#7  lisp_frame_p (spPtr=spPtr@entry=0x0) at ../arm64-exceptions.c
  Clozure#8  walk_stack_frames (start=0x0, start@entry=0xffff829fc1f0,
                         end=0xffff82a00000) at ../albt.c

`start' entered the function as 0xffff829fc1f0 and reached the
lisp_frame_p call as 0x0, which is one iteration of the loop assigning
next to start. albt.c is that call. So the zero word, the NULL and
the dereference are all present in one captured instance.

The report's original fault was elsewhere -- in the GC, marking a tstack
area -- and it is still unexplained. This defect is why that fault has no
usable backtrace: the printer died inside the crash it was reporting, and
turned an unhandled SIGSEGV into a SIGABRT with nothing to read.

Verified on linuxarm64 with the Makefile's own command, read out of
`make -n albt.o' rather than guessed:

  pristine  rc=0  0 warnings  albt.o = 25896 bytes
  patched   rc=0  0 warnings  albt.o = 29176 bytes

The object grows, which is how we know the new code is compiled in and
not folded away.  Under -Wall -Wextra -Wno-unused-parameter both files
emit ONE warning and the set difference between them is empty, so the
patch adds no diagnostic.  Two positive controls were watched failing IN
THE SUBJECT FILE: a broken paren in the new helper (errors) and a void return type on the helper (`void value not ignored as it
ought to be' at both call sites, :120), the second of which
also proves both arms really call it.

NOT verified: no kernel has been linked and booted with this change, and
the fault has not been reproduced on demand -- the observation above is a
captured instance, not a repro. The forward-progress bound is also the
minimum fix; it does not validate that a candidate points at a real
frame, only that following it cannot leave the region.
…piler that implements it

SUPPORT_PRAGMA_UNUSED is #ifdef'd in ten places and #define'd in none, so the
`#pragma unused(...)' annotations it guards have never reached any compiler:

    $ git grep -c SUPPORT_PRAGMA_UNUSED -- 'lisp-kernel/*.c'
    lisp-kernel/arm-exceptions.c          (936, 1002, 1305)
    lisp-kernel/arm64-exceptions.c        (1114, 1196, 1825)
    lisp-kernel/ppc-exceptions.c          (983, 1146, 1367, 1540)
    $ git grep -n 'define[[:space:]]*SUPPORT_PRAGMA_UNUSED'
    (nothing)

Each site says that a parameter forced on the function by a handler-dispatch
signature is deliberately unused -- do_hard_stack_overflow needs only xp,
do_spurious_wp_fault needs none of the three, allocate_no_stack needs none.  The
intent is right and recorded; only the channel to the compiler is dead.

=== MEASURED, because #pragma unused is not a thing to reason about from memory

It is a Metrowerks/MPW-lineage extension and gcc and clang do not agree about
it.  So it was measured rather than recalled, on the real do_hard_stack_overflow
shape, with a positive control (the same file WITHOUT the pragma, which must
emit `unused parameter' before any other result is interpretable).  gcc 11.5.0
20240719 on x86_64 and aarch64, clang 15.0.7 on aarch64:

                                    gcc 11.5.0            clang 15.0.7
  recognised?                       NO                    YES
                                    "ignoring '#pragma    (no diagnostic
                                     unused ' [-Wunknown-  at all)
                                     pragmas]"
  suppresses `unused parameter'?    NO   (still 2)        YES  (2 -> 0)
  diagnostics added                 1 per site            0
  -Wall -Wextra -Werror exit code   1 (2 errors -> 3)     0

Verbatim, gcc, with the pragma present:

    B.c: In function 'do_hard_stack_overflow':
    B.c: warning: ignoring '#pragma unused ' [-Wunknown-pragmas]
    B.c:53: warning: unused parameter 'area' [-Wunused-parameter]
    B.c:67: warning: unused parameter 'addr' [-Wunused-parameter]
    exit code: 0

Verbatim, clang, same file, same flags: no output, exit 0.

So an UNCONDITIONAL #define would be measurably worse than the dead guard: on
gcc it suppresses nothing and adds one -Wunknown-pragmas at each of the ten
sites, and turns a -Wall -Wextra -Werror build from 2 errors into 3.  Hence
clang only.

Two more measurements that bear on the risk, both on clang 15.0.7:

  * A pragma naming an identifier that does not exist is DIAGNOSED, not ignored:
    "undeclared variable 'x' used as an argument for '#pragma unused'
    [-Wignored-pragmas]".  clang parses the argument list, so these annotations
    stay honest as the code changes -- something -Wno-unused-parameter cannot
    do.
  * A pragma naming a parameter that IS used is silently accepted: no
    diagnostic, exit 0 even under -Werror.  That matters here, because
    ppc-exceptions.c has `#pragma unused(where)' in handle_uuo and `where'
    IS used,.  That annotation is stale.  Under this patch it
    becomes a no-op rather than a new warning, so I have left it alone rather
    than widen this patch into a second fix -- but it is wrong and worth
    deleting separately.
  * At file scope the names are not in scope and clang warns twice and
    suppresses nothing.  All ten sites are inside a function body, which is
    where it works.

=== Where the #define goes

lisp.h, because all three files include it first and this is a property of the
COMPILER, not of a platform -- putting it in the per-platform headers would mean
repeating a __clang__ test in each of them.

=== Relationship to -Wno-unused-parameter on the arm64 kernel

I added -Wno-unused-parameter to lisp-kernel/linuxarm64/Makefile in a separate
arm64 patch, and said there that what to do with the #pragma blocks was your
call.  This is that follow-up, and it does NOT make the flag redundant:
linuxarm64/Makefile:11 is `CC = ${CROSS}gcc', so on our own kernel build this
patch changes nothing at all and the flag is still what suppresses the 78
-Wunused-parameter there.  Measured, not assumed -- see below, where gcc's
warning census is bit-for-bit the same with and without this patch.  The two are
complementary: the pragma covers the compiler that understands it, the flag
covers the one that does not.

=== The alternative, if you would rather not

Delete the ten #ifdef/#pragma/#endif blocks and keep warning suppression at the
build flags.  That is a defensible call and a smaller thing to maintain; it just
throws away suppression that measurably works on clang, and clang is the
compiler the Darwin targets use.  I have written the version I think is right;
either is easy from here and it is your tree.

=== NOT TESTED

⛔ Of the three files this affects, only arm64-exceptions.c has been compiled.

RED-then-GREEN on the real translation unit, aarch64, tree at the pin:
lisp-kernel/arm64-exceptions.c, -Wall -Wextra, otherwise the Makefile's own
flags (-include ../platform-linuxarm64.h -DLINUX -DARM64 ... -g -O2 -Wno-format).
Three configurations, because the third is the one that justifies "clang only":

                          unused-param  -Wunknown-pragmas  total warn   .o md5
  RED     gcc,   no macro       19              0              61       96930241
  GREEN   gcc,   no macro       19              0              61       96930241
  COUNTER gcc,   macro ON       19              3              64       96930241
  RED     clang, no macro       19              0              23       7d5242da
  GREEN   clang, macro ON       13              0              17       7d5242da

  * clang 19 -> 13.  Grouped by parameter name, so that "six warnings went
    away" cannot be six DIFFERENT warnings:

        addr 4->2   area 2->0   size 1->0   xp 4->3
        info 1->1   instruction 1->1   param 1->1   signum 1->1   tcr 4->4

    Total -6, and the three #pragma lines in the file name exactly six
    identifiers: (area,addr) + (xp,area,addr) + (size) = 2+3+1.  Every
    parameter NOT named by a pragma is untouched.
  * ZERO -Wignored-pragmas on the green clang run -- the check that every
    identifier the three pragmas name really is in scope.
  * gcc RED vs GREEN is BYTE-IDENTICAL output, because __clang__ is not
    defined and the guard stays false.  0159's flag therefore still does all
    the work on our own kernel build.
  * COUNTER is what an unconditional #define would do to gcc: three
    -Wunknown-pragmas, one per site, 61 -> 64 warnings, and not one
    unused-parameter removed.  Worse than the dead guard, measured rather than
    predicted.
  * THE .o md5 IS IDENTICAL in all three gcc configurations and in both clang
    configurations.  So this changes diagnostics and provably nothing else --
    no generated code moves, on either compiler, with or without the macro.
  * Positive control: the RED clang run emitted 19 > 0, i.e. the instrument was
    seen reporting the un-suppressed state before the suppressed one was
    believed.
  * Reproduced on a second, independent pin tree by a red/green control
    script that carries these gates.

NOT COMPILED, at all: arm-exceptions.c (ARM32) and ppc-exceptions.c -- I have no
ARM32 or PPC machine.  Their seven sites were READ, and every identifier named
there is a parameter of the enclosing function, so I expect the same result --
but that is inspection, not a build.  The one thing inspection did turn up is
ppc-exceptions.c's stale `where', and that one is measured harmless rather
than assumed so.

Confidence: high that the #define is correct for clang and inert for gcc (red
and green both watched, on the real TU); high that it is a no-op for our own
kernel build; medium that the seven un-compiled sites are clean, on inspection
alone.

Signed-off-by: Mauro DiBenedetto <maurodibenedetto@gmail.com>
In both arms of recursive_lock_trylock (futex and non-futex), when the
calling thread already owns the lock and was_free is NULL, the owner
branch increments m->count and then falls through into the
store_conditional on m->avail.  That store_conditional fails, because
the caller already holds the lock, so the function returns EBUSY with
the count already raised.  The caller believes it never got the lock
and will not unlock it, so the extra count is never undone and the
lock's count can never return to zero: the lock stays owned forever.

The early return was conditional on was_free being non-NULL when only
the *was_free store should have been.  Make the owner branch return 0
unconditionally, storing through was_free only when it is provided.

No caller exists in the tree today; the function is exported only
through the kernel-import tables.  Reported as Clozure/ccl issue Clozure#597.
@cyclistmass
cyclistmass changed the base branch from arm64 to master September 9, 2026 03:59
@xrme

xrme commented Sep 25, 2026

Copy link
Copy Markdown
Member

The recursive_lock_trylock issue #597 was fixed in b5a00d1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants