arm64: the write-barrier memoize loops release with stlxr

[cyclistmass]

The 16 EGC memoize loops in arm64-spentry.s end with a store-exclusive and
then a separate dmb ish:

        ldxr    imm1, [temp0]
        orr     imm1, imm1, imm3
        stxr    w17, imm1, [temp0]
        cbnz    w17, 1b
        dmb ish                         /* ppc:508 isync */

This makes the store-exclusive itself a release and drops the barrier. A
release store-exclusive orders every prior access before the store becomes
observable, which is what the trailing dmb ish was providing, so it encodes
the same guarantee rather than a weaker one.

It is the change you already took for the LAP atomics in 76ed388c, applied to
the kernel side of the same family.

ARM32 is untouched and cannot do this. ARMv7 has no acquire/release load or
store forms, so a standalone dmb is the only tool there and 539ed520 stands
as written. The asymmetry is an ISA fact, not a claim about your change.

Measured

linuxarm64, Neoverse V3 (CPU part 0xd84), two cores. Clock measured at
3.29 GHz by the PMU cycle counter over a three-second spin, so one cycle is
0.304 ns and the nanoseconds below convert. Interleaved A/B, two cycles, medians
of five reps, one session, one box. The two kernels differ only by this patch
and the boot image is byte-identical across both.

	before	after	cycles
memoized store, 1 thread	14.5 ns	12.8 ns	47.7 -> 42.1
memoized store, 2 threads	13.9 ns	13.0 ns	45.7 -> 42.8

Each memoized store updates two bitmaps, refbits then ephemeral_refidx, so
it runs two of these loops -- two hunks per label in the diff. The 5.6 cycles
saved is therefore about 2.8 cycles per site.

Eight benches this patch cannot reach move by at most 0.2 ns: allocation,
semaphores, locks, uncontended atomic-incf, and the fixnum and old-object
store paths that exit before the memoize loop. 0.2 ns is the timer's resolution
at those rates.

Judged in absolute nanoseconds on purpose. A barrier costs a fixed number of
cycles, so one 0.2 ns quantum reads as 15 percent on a 1.2 ns bench and 1.4
percent on a 14.5 ns one. Percentages invert the comparison here.

Not in this patch

The conditional-store sites in egc_store_node_conditional and
egc_set_hash_key_conditional keep their barriers. Those order the CAS itself
rather than the memoize bit-set.

VERIFIED: built and run on linuxarm64. ANSI 21679 tests, 0 failures.
tests/ccl.lsp 243 tests, 0 failures. Kernel assembly only, so no other target
is affected. Not built on any other target.

One caveat stated plainly: both suites are single-threaded, so they show that
nothing gross broke. They cannot certify a barrier-ordering change in either
direction.

A separate question, not part of this patch

I also measured dropping these 16 barriers outright, with stxr left alone.
That reaches 11.2 ns, or 36.8 cycles, so a dmb ish at these sites costs
about 5.4 cycles and roughly half of that is the release ordering this patch
keeps.

I am not proposing that. It would need these sites to require no ordering at
all, which is a question about the EGC rather than about the encoding: the
refbits and ephemeral_refidx bitmaps are read by the collector only after
suspend_other_threads(), and pc_luser_xp re-memoizes in C for a thread
suspended inside the window. If that reasoning is wrong the barriers belong
where they are. If it is right, there is another 1.6 ns per memoized
store available, about 5 cycles. Your call, and I am happy to leave it alone.

[cyclistmass]

The argument in this pull request is a measurement, so here is the harness that
produced it. You should be able to get your own numbers rather than take mine.

This is not a patch and I am not asking you to take it into CCL. It sits on
a branch so you can fetch it and delete it. GitHub will offer you a "compare and
pull request" button on it. Please ignore that.

git fetch https://github.com/cyclistmass/ccl.git wb-bench-harness
git show FETCH_HEAD:contrib-wb-bench/wb-bench.lisp > wb-bench.lisp
./<your kernel> --image-name <your image> < wb-bench.lisp

contrib-wb-bench/README.md on that branch says what each bench measures. The
file needs nothing but a built CCL. No build system, no configuration, and it
uses only exported CCL entry points.

Three things about reading the output, because each one cost me time:

Compare absolute nanoseconds, not percentages. A barrier costs a fixed
number of cycles, so removing one is a constant saving on the paths that ran it.
WB-1T-FIXNUM runs at about 1.2 ns, which is three or four cycles and near the
timer's resolution, so one quantum there reads as 15 percent while the same
quantum on the 14.5 ns memoize bench reads as 1.4 percent. In percent the
controls appear to move more than the thing under test, and a real effect reads
as noise. In nanoseconds the controls sit inside 0.2 ns.

Interleave the arms and repeat. A single A-then-B pass cannot separate an
effect from a machine that sped up between runs. I ran A, B, A, B and required
both steps to agree.

Check the two kernels really differ. Verify the binary's md5 changes between
arms. A build system that decides the sources are unchanged will hand the same
binary to both halves, and then the patch looks inert and the benchmark looks
noisy. That happened to me twice while measuring this.

The rows that can move when the memoize loops change are WB-1T-YOUNG,
WB-2T-NEAR and WB-2T-FAR. Everything else is a control. The two sharpest
controls are WB-1T-OLD and WB-1T-FIXNUM, which take the same store path up
to the point where it decides no memoize is needed.

My cell, for comparison: Neoverse V3, CPU part 0xd84, two cores, clock measured
at 3.29 GHz with the PMU cycle counter over a three-second spin, so one cycle is
0.304 ns.

These are microbenchmarks. They answer "what does this instruction sequence
cost" and not "is the system faster", and they cannot certify a memory-ordering
change in either direction.

[xrme]

I believe that it's true that we probably don't need a barrier here: we're interested in consistencty/atomicity rather than ordering.

The only possible wrinkle I see is that we would have to be sure that thread suspend acts as a full barrier so that the stores are globally visible when we stop the world and read the bits.

However, I think I'd prefer to leave the release ordering. (Which your change preserves, and which I intend to accept.)