Random Ideas

[remexre]

This discussion can serve as a place to put half-baked ideas before discussing them with the larger group.

[remexre]

Kernel-to-user message passing

errno kinda sucks as the only way to report errors -- usually, the kernel has more information about what failed and why that it could return to the user.

An "errno-like" interface also encourages people to think synchronously, encouraging people to use OS-level threads to write concurrent programs, and bringing in all the pain there.

A possible better way:

• All possibly-blocking syscalls are asynchronous.
  • Syscalls that provide buffers that userspace knows the size of a priori should still work the same way, though the ownership model around these is worth considering. I don't think we're planning on porting Rust, but reading about the troubles with io_uring buffer ownership and "async Drop" might be useful.
• The kernel reserves some chunk of VA space at the top of the memory map (before the vDSO?). This is read-only to userspace.
• The kernel basically treats this like a slotted page in a database, except going from the top to the bottom (i.e., the header is at the highest address, the pointers grow down below that, and the cells start at the lowest address and grow up).
• When a kernel thread wants to return a variable-sized result to the userspace process, it checks if there's enough free space for a cell containing the data and a pointer.
  • If so, it writes the data and pointer, updates the header, and continues on.
  • If not, it sees if it could reclaim enough space;
  • If not, it records in the header that there's a pending result, and goes back to sleep.
• Userspace has a syscall for "wait for new events," which lets it mark some entries as reclaimable and promises that it's not using any of the data in the chunk.
  • If there's a pending result, the area gets compacted to make room for it.
  • A low-priority kernel thread (see Scheduler #7) compacts chunks if there's nothing else to do.
• Errors and other fixed strings can be reported to userspace as 32-bit offsets into the vDSO, so an error code and a chain of e.g. 3 causes might only take up 16 bytes in the chunk.
• We need to make sure that any result being delivered to userspace can fit in the page, but this is probably not onerous; a cause chain of over 1000 causes is probably fine to just truncate and warn about.
  • We don't need (or want) to resize the buffer. If we start getting so many replies back to userspace that userspace isn't ACKing, it's probably an ill-behaved process anyway, and some backpressure will improve overall system responsiveness.