Repository navigation
scst_user: yield cleanup while a shared SGV remains checked out - #392
Conversation
c571a95 to
c573ef0
Compare
|
@lnocturno Can you help to have a check? |
|
Hi, The change looks correct to me. Returning to the outer cleanup loop allows other devices to release their checked-out SGV buffers, while the pending device remains alive until its cleanup completes. I did not find any regressions in the changed logic. Please add a commit message body explaining the shared-buffer scenario, why the additional flush from #378 is insufficient, and how yielding to the outer loop resolves the dependency. The checkpatch run reports “Missing commit description”. Thanks, Gleb |
With SGV sharing, device B can reuse a buffer whose allocation owner is a command from device A. If A's user handle closes while B still has the SGV checked out, A's command remains in its hash. The unjam pass sees that command, but there is no ready A command, so get_next_cmd() returns -EAGAIN. Once cleanup_done is set, the old inner loop keeps retrying A and prevents the single cleanup worker from reaching B. The post-unjam pool flush added by PR SCST-project#378 frees SGV objects that have already returned to the recycling lists. It cannot free the object still checked out by B, so it cannot resolve this dependency. Yield A to the outer cleanup loop when no ready command exists but the unjam pass still reports commands. The worker can process B and retry A after its normal sleep. Signal A's cleanup completion only when its release is done and no commands remain. Link: SCST-project#279
c573ef0 to
5bd0b24
Compare
Hi @lnocturno Commit message updated. |
2 participants
Addresses #279.
Reproducer and exact steps
The failing run used unmodified
SCST-project/scstcommit5908a8da46c3f98dc1524d31d761bd4dd642b151, which already contains#378. The test ran in a disposable Ubuntu 24.04 AArch64 VM on
6.8.0-142-generic. It usesscst_localand two file-backedscst_userdevices; no network target or kernel instrumentation is required.
The reproduction procedure
contains the exact clone, Lima copy, build, module load, run, and log
collection commands. The executable pieces are
run.sh,
the ioctl interposer,
and the SG READ helper.
The upstream evidence report
separates direct observations from the source-based explanation.
The runner registers A and B with a shared SGV pool, full memory reuse,
nonblocking commands, and
ON_FREE_CMD_IGNORE. It completes two 4 KiBREAD(10) commands on A, starts one on B, verifies that B receives A's
userspace buffer address, closes A's handle while B's READ is outstanding,
holds for 30 seconds, then closes B's handle. It captures the backend log,
SCST
commandssysfs files, release-thread states, and kernel log.Observed results on unmodified upstream
A_BUFFER,B_HOLDS_A_BUFFER, andA_HANDLE_CLOSEDall report0xea5674001000.ref=1,sent_to_user=0, andscst_cmd=NULL.sent_to_user=1and a non-NULLscst_cmd.scst_usr_cleanu, withsgv_pool_flush -> dev_user_cleanup_threadon the stack.SCST_REPRO_CLEANUP_TIMEOUT; the final state snapshot still shows release threads in D state.State 7 is
UCMD_STATE_ON_FREE_SKIPPED; state 3 isUCMD_STATE_EXECING. The upstream run did not produce a vmcore, and thekernel pointers printed through sysfs were obscured. The matching userspace
address establishes the test's reuse precondition; the kernel SGV ownership
path below is inferred from upstream source and the observed states.
Source-based cause and why #378 does not cover this case
dev_user_alloc_sg()getsbuf_ucmdfromsgv_get_priv(), which returns theSGV's allocation owner.
sgv_pool_flush()walks objects on the recyclinglists. #378 adds a flush after unjamming, which handles SGVs already returned
to those lists. B's READ remains outstanding during the hold, so its
checked-out SGV cannot be reclaimed by that flush.
dev_user_unjam_dev(A)counts a retained A command before skipping it whensent_to_user=0. With no ready A command,dev_user_get_next_cmd(A)returns-EAGAIN. The old inner loop retries A without returning to the singleouter cleanup worker, preventing it from advancing other devices. This
explains the observed runnable worker, watchdog trace, and blocked release
threads.
Change and before/after validation
When there is no ready command but a command remains pending, yield A to the
outer cleanup worker. That worker can process B and sleeps 100 ms before
retrying deferred devices. Once A's hash is empty and
cleanup_doneis set,the usual completion path runs.
The fixed run used the same workload, kernel, and matching module build.
Its backend log
again confirms the buffer reuse precondition. During the hold, the
state log
shows
scst_usr_cleanusleeping inmsleep. After B closes, therun log
reports
SCST_REPRO_CLEANUP_COMPLETE; both release threads and devicecommand files are gone. The captured fixed-run kernel log contains no soft
lockup.
This reproduces the failure pattern reported in #279 on a newer SCST and
kernel version; it does not claim an identical deployment to the reporter's.