Skip to content

libsgxstep: restore APIC before enclave destroy to fix D-state hang - #109

Open
Hayao0819 wants to merge 1 commit into
jovanbulck:masterfrom
Hayao0819:fix-sgx-destroy-enclave-hang
Open

Hayao0819 wants to merge 1 commit into
jovanbulck:masterfrom
Hayao0819:fix-sgx-destroy-enclave-hang

Conversation

@Hayao0819

Copy link
Copy Markdown
Contributor

Fixes #90.

Calling sgx_destroy_enclave while SGX-Step still holds /dev/sgx-step open wedges the process in uninterruptible D state, and a reboot is the only way out. As you already suspected in the thread, the culprit is the APIC mode. When the in-kernel driver's sgx_release unregisters its mmu-notifier it waits on synchronize_srcu, but SGX-Step has left the victim core's timer in one-shot mode with no pending tick, so the SRCU grace period never advances and the wait blocks forever.

Closing /dev/sgx-step fixes this, since step_release restores the timer. The catch is the timing. libsgxstep closes the fd from a destructor, which runs after main returns, i.e. long after sgx_destroy_enclave has already hung. So this splits the close out into a new step_close() and calls it just before sgx_destroy_enclave in app/bench, so the victim core is ticking normally again before the driver reaches synchronize_srcu. The destructor still calls step_close(), so the crash-cleanup path is unchanged and the second close is a guarded no-op.

Verification

On a Gemini Lake machine (Debian 12, kernel 6.1) the bench reliably wedged in D state at sgx_destroy_enclave on master. With this change it single-steps the NOP slide and sgx_destroy_enclave returns normally, and the process exits cleanly.

[ all done; counted ... IRQs (AEP/IDT) ]
APP_EXIT=0

@Hayao0819

Copy link
Copy Markdown
Contributor Author

The problem recurred even after applying this fix. The cause may lie elsewhere. We are continuing our investigation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sgx_destroy_enclave blocks app in kernel

1 participant