Skip to content

compat: honor the epoll interest mask in the macOS shim - #202

Merged
wiedehopf merged 1 commit into
wiedehopf:devfrom
enricolorenzoni59:fix/epoll-shim-interest
Nov 4, 2026
Merged

wiedehopf merged 1 commit into
wiedehopf:devfrom
enricolorenzoni59:fix/epoll-shim-interest

Conversation

@enricolorenzoni59

Copy link
Copy Markdown
Contributor

Summary

The macOS epoll shim (compat/apple/epoll_shim.c) does not translate the epoll interest mask correctly:

  • with EPOLLIN | EPOLLOUT in one call, the second EV_SET overwrites the first, so only the write filter is submitted;
  • EPOLL_CTL_MOD only ever calls EV_ADD, so a filter that is no longer requested is left registered.

Impact

macOS only, no Linux code path is touched.

flushClient() adds EPOLLOUT with EPOLL_CTL_MOD when a client's send queue cannot be flushed and removes it again once the queue drains. On kqueue the removal never happened, so the stale EVFILT_WRITE keeps reporting the now writable socket: epoll_wait() returns immediately on every loop with an empty send queue, and the network thread spins at 100% until that client disconnects.

Repro

Small program built against the shim:

#include "epoll_shim.h"
#include <stdio.h>
#include <sys/socket.h>
#include <unistd.h>

int main(void) {
    int pair[2], ep = epoll_create1(0);
    struct epoll_event ev = { 0 }, out[4];
    socketpair(AF_UNIX, SOCK_STREAM, 0, pair);
    ev.events = EPOLLIN | EPOLLOUT;
    ev.data.ptr = &pair;
    epoll_ctl(ep, EPOLL_CTL_ADD, pair[0], &ev);
    int initial = epoll_wait(ep, out, 4, 10);
    ev.events = EPOLLIN;
    epoll_ctl(ep, EPOLL_CTL_MOD, pair[0], &ev);
    int idle = epoll_wait(ep, out, 4, 10);
    write(pair[1], "x", 1);
    int n = epoll_wait(ep, out, 4, 10), readable = 0;
    for (int i = 0; i < n; i++)
        readable += !!(out[i].events & EPOLLIN);
    printf("initial=%d idle_after_MOD=%d readable=%d\n", initial, idle, readable);
    return idle == 0 && readable == 1 ? 0 : 1;
}

Baseline on macOS: initial=1 idle_after_MOD=1 readable=1, exit 1. Native Linux epoll with the same sequence gives idle_after_MOD=0; with this patch the shim matches Linux.

Fix

Build the full change set for both directions and submit it in one kevent() call with EV_RECEIPT, so every change reports its own result instead of the call stopping at the first error. MOD adds/removes each direction to match the requested mask; DEL deletes both and returns ENOENT only when neither was registered.

Verification

  • repro above, plus MOD from IN to IN|OUT and back, MOD to OUT, DEL of a read-only registration, double DEL;
  • local macOS A/B: 4.2 MB fed to a raw output port with one slow sink and one fast sink, then the slow sink drained. Old shim: 1.000 s of process CPU in the second after the drain. Patched shim: 0.000 s. The slow sink received the same 473,680 bytes in both runs, the fast sink stream stayed valid, no epoll_ctl errors;
  • build with -W -Wall -Werror;
  • make test (cprtests, crctests) passes.

EPOLL_CTL_ADD on an already registered fd and EPOLL_CTL_MOD on an unregistered one keep their previous best-effort behavior (no EEXIST/ENOENT tracking). readsb does not rely on either.

epoll_ctl MOD replaces the interest mask, but the shim only ever called
EV_ADD and never removed a filter, and with EPOLLIN|EPOLLOUT in the same call
the second EV_SET overwrote the first, so only the write filter was submitted.

After a client recovered from backpressure, flushClient() clears EPOLLOUT with
EPOLL_CTL_MOD. The stale EVFILT_WRITE kept reporting the now writable socket,
so epoll_wait() returned immediately on every loop with an empty send queue.

Submit both filters, remove the ones a MOD no longer requests, and delete both
on DEL, all in a single kevent() call with EV_RECEIPT so every change reports
its own result. A missing filter is only an error on DEL when neither was
registered, matching the ENOENT epoll semantics the callers already expect.
@wiedehopf
wiedehopf merged commit bf8f8fd into wiedehopf:dev Nov 4, 2026
6 checks passed
@enricolorenzoni59
enricolorenzoni59 deleted the fix/epoll-shim-interest branch November 4, 2026 09:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants