Add a directory-backed database, behind experimental-multiprocess (1/3)

CHANGELOG.md

@@ -21,6 +21,11 @@
   be added behind the `experimental_cursor` feature flag, like the table cursors' were.
 
 ## 4.2.0 - 2026-XX-XX
+* Add `MultiProcessDatabase` behind the new `experimental-multiprocess` feature. It stores a
+  database in a directory, alongside the lock file that coordinates the processes using it, and
+  takes its exclusion from that lock file rather than from a lock on the database file. This is the
+  first step of an incomplete feature: only one process may have the database open, so it has no
+  advantage over `Database` yet.
 * `Durability::None` commits are about 2x faster.
 * Commits now flush table root updates in a deterministic order, removing a source of
   nondeterminism that could make identical operation sequences produce differing database files

Cargo.toml

@@ -54,6 +54,9 @@ experimental_cursor = ["experimental-api-5"]
 # Unstable additions and changes to the public API, planned for redb 5. May change incompatibly, or
 # be removed, in any release
 experimental-api-5 = []
+# Incomplete support for using a database from several processes. May change incompatibly, or be
+# removed, in any release
+experimental-multiprocess = []
 
 [profile.bench]
 debug = true

docs/design.md

@@ -496,3 +496,42 @@ Therefore, we make only a few assumptions about the guarantees provided by the u
    a range of bytes in a file, no bytes outside of that range will change,
    even if the write occurs just before a crash or power failure. sqlite makes this same
    assumption, by default, in all modern versions.
+
+# Multi-process access (in progress)
+
+`MultiProcessDatabase`, behind the `experimental-multiprocess` feature, stores a database in a
+directory rather than a single file:
+
+| file | contents |
+|------|----------|
+| `data.redb` | the database, in the ordinary redb file format |
+| `write.lock` | empty; held exclusively by the process that has the database open |
+
+The point of the directory is to move exclusion off the database file. An ordinary `Database` takes
+an exclusive advisory lock on the file itself, which is what stops a second process opening it --
+and which also stops any other process reading it. A multi-process database takes an exclusive lock
+on `write.lock`, before it touches anything else in the directory, so that the process which gets
+it has the directory to itself, including while it is being created. That is the lock later steps
+build on: it is what will exclude other *writers* once readers are allowed in.
+
+Until then the database file keeps its ordinary exclusive lock too. `write.lock` is only visible to
+a process that goes through `MultiProcessDatabase`, and one that reaches past the directory to open
+`data.redb` directly would not be looking at it, so the file needs a lock of its own. It has to be
+the exclusive one: a shared lock would let a `ReadOnlyDatabase` in, and nothing yet stops the
+process holding the directory from freeing pages that such a reader is still using. So today the
+restriction is the same as a `Database`'s -- one process, and `DatabaseError::DatabaseAlreadyOpen`
+for the rest -- and the directory is scaffolding rather than a relaxation of it.
+
+That constraint on the database file's lock is worth recording, because it shapes what comes next.
+Making room for readers means *removing* this lock rather than weakening it: on Windows a shared
+range denies writes to every process including the one holding the lock, so a writer cannot hold a
+shared lock on a file it writes to at all. All reader/writer coordination therefore has to live in
+the directory's own lock files.
+
+The lock is held by the storage backend rather than beside the `Database`, because a live write
+transaction keeps the database open past the point where the handle is dropped. A lock released
+when the handle went away would let another process start writing while that transaction was still
+running; tying it to the backend gives it exactly the lifetime of the open file.
+
+A platform without file locking cannot support any of this safely, so opening fails there rather
+than warning and continuing as `Database` does.

src/db.rs

@@ -1120,7 +1120,7 @@ impl Database {
         root.map(|header| BtreeHeader::new(header.root, header.checksum, length))
     }
 
-    fn new(
+    pub(crate) fn new(
         file: Box<dyn StorageBackend>,
         allow_initialize: bool,
         page_size: usize,

src/lib.rs

@@ -80,6 +80,8 @@ pub use error::{
 };
 #[cfg(feature = "experimental-api-5")]
 pub use key_range::KeyRange;
+#[cfg(feature = "experimental-multiprocess")]
+pub use multi_process::{MultiProcessBuilder, MultiProcessDatabase};
 #[cfg(feature = "experimental-api-5")]
 pub use multimap_table::MultimapCursor;
 pub use multimap_table::{
@@ -106,6 +108,8 @@ mod db;
 mod error;
 #[cfg(feature = "experimental-api-5")]
 mod key_range;
+#[cfg(feature = "experimental-multiprocess")]
+mod multi_process;
 mod multimap_table;
 mod sealed;
 mod table;

src/multi_process/locks.rs

@@ -0,0 +1,196 @@
+//! The files that make up a multi-process database directory, and the lock that excludes other
+//! processes from it.
+//!
+//! Everything in here uses only `std::fs` file operations and the advisory file locks exposed by
+//! `std::fs::File`. See `docs/design.md` for the protocol these files implement.
+
+use crate::tree_store::file_backend::FileBackend;
+use crate::{DatabaseError, StorageBackend, StorageError};
+use std::fs::{File, OpenOptions, TryLockError};
+use std::io;
+use std::io::ErrorKind;
+use std::path::{Path, PathBuf};
+
+const DATA_FILE_NAME: &str = "data.redb";
+const WRITE_LOCK_FILE_NAME: &str = "write.lock";
+
+/// Maps the "this platform has no file locks" case to an error. A multi-process database has no way
+/// to be safe without them, so unlike [`crate::Database`] it refuses to open rather than warning
+/// and continuing.
+fn lock_unsupported(err: io::Error) -> DatabaseError {
+    if err.kind() == ErrorKind::Unsupported {
+        return StorageError::Io(io::Error::new(
+            ErrorKind::Unsupported,
+            "file locking is not supported on this platform, so a multi-process database cannot \
+             be opened safely",
+        ))
+        .into();
+    }
+    StorageError::Io(err).into()
+}
+
+fn open_or_create(path: &Path) -> Result<File, io::Error> {
+    OpenOptions::new()
+        .read(true)
+        .write(true)
+        .create(true)
+        .truncate(false)
+        .open(path)
+}
+
+/// The paths that make up a multi-process database directory.
+pub(super) struct DatabaseDir {
+    root: PathBuf,
+}
+
+impl DatabaseDir {
+    pub(super) fn new(root: impl AsRef<Path>) -> Self {
+        Self {
+            root: root.as_ref().to_path_buf(),
+        }
+    }
+
+    fn data_file(&self) -> PathBuf {
+        self.root.join(DATA_FILE_NAME)
+    }
+
+    fn write_lock_file(&self) -> PathBuf {
+        self.root.join(WRITE_LOCK_FILE_NAME)
+    }
+
+    /// Takes the write lock, which excludes every other process from the database. Held for as
+    /// long as this process has it open.
+    ///
+    /// Taken before anything else in the directory is opened, so that a process which gets it has
+    /// the directory to itself -- including while it is being created. The lock file is only made
+    /// when the database is being created: its absence is what tells `open()` that this directory
+    /// is not a multi-process database.
+    fn acquire_write_lock(&self, create: bool) -> Result<File, DatabaseError> {
+        let path = self.write_lock_file();
+        let file = if create {
+            open_or_create(&path)
+        } else {
+            OpenOptions::new().read(true).write(true).open(&path)
+        }
+        .map_err(|err| {
+            if err.kind() == ErrorKind::NotFound {
+                StorageError::Io(io::Error::new(
+                    ErrorKind::NotFound,
+                    "not a multi-process database directory",
+                ))
+            } else {
+                StorageError::Io(err)
+            }
+        })?;
+
+        match file.try_lock() {
+            Ok(()) => Ok(file),
+            Err(TryLockError::WouldBlock) => Err(DatabaseError::DatabaseAlreadyOpen),
+            Err(TryLockError::Error(err)) => Err(lock_unsupported(err)),
+        }
+    }
+
+    /// Opens the directory, creating it if `create` is set, and returns a backend for the database
+    /// file that holds the write lock for as long as the database is open.
+    pub(super) fn open(&self, create: bool) -> Result<Box<dyn StorageBackend>, DatabaseError> {
+        if create {
+            std::fs::create_dir_all(&self.root).map_err(StorageError::Io)?;
+        } else if !self.root.is_dir() {
+            return Err(StorageError::Io(io::Error::new(
+                ErrorKind::NotFound,
+                "no such multi-process database directory",
+            ))
+            .into());
+        }
+
+        let write_lock = self.acquire_write_lock(create)?;
+        let data = OpenOptions::new()
+            .read(true)
+            .write(true)
+            .create(create)
+            .truncate(false)
+            .open(self.data_file())
+            .map_err(StorageError::Io)?;
+        // The ordinary exclusive lock, the same one a Database takes. The write lock above is what
+        // other multi-process handles look at, but a process that reaches past the directory and
+        // opens this file directly would not be looking at it, so the file needs a lock of its own.
+        // It has to be the exclusive one: a shared lock would let a ReadOnlyDatabase in, and
+        // nothing yet stops this process from freeing pages that such a reader is still using.
+        // Making room for readers is what the later releases in this series are for, and this is
+        // the lock they have to replace
+        let data = FileBackend::new(data)?;
+
+        Ok(Box::new(DirectoryBackend { data, write_lock }))
+    }
+}
+
+/// The database file, plus the write lock that has to outlive it.
+///
+/// The lock is held here rather than alongside the [`crate::Database`] because a live write
+/// transaction keeps the database open past the point where the handle is dropped. A lock released
+/// when the handle went away would let another process start writing while that transaction was
+/// still running. Tying it to the backend gives it exactly the lifetime of the open file: redb
+/// calls `close()` once, when it has really finished.
+#[derive(Debug)]
+struct DirectoryBackend {
+    data: FileBackend,
+    write_lock: File,
+}
+
+impl StorageBackend for DirectoryBackend {
+    fn len(&self) -> Result<u64, io::Error> {
+        self.data.len()
+    }
+
+    fn read(&self, offset: u64, out: &mut [u8]) -> Result<(), io::Error> {
+        self.data.read(offset, out)
+    }
+
+    fn set_len(&self, len: u64) -> Result<(), io::Error> {
+        self.data.set_len(len)
+    }
+
+    fn sync_data(&self) -> Result<(), io::Error> {
+        self.data.sync_data()
+    }
+
+    fn write(&self, offset: u64, data: &[u8]) -> Result<(), io::Error> {
+        self.data.write(offset, data)
+    }
+
+    fn close(&self) -> Result<(), io::Error> {
+        self.data.close()?;
+        self.write_lock.unlock()
+    }
+}
+
+#[cfg(test)]
+mod test {
+    use super::*;
+
+    #[test]
+    fn the_write_lock_excludes_other_handles() {
+        let tmpdir = tempfile::tempdir().unwrap();
+        let dir = DatabaseDir::new(tmpdir.path().join("db"));
+
+        let first = dir.open(true).unwrap();
+        assert!(matches!(
+            dir.open(false),
+            Err(DatabaseError::DatabaseAlreadyOpen)
+        ));
+
+        // Closing the backend is what releases the lock, since that is when redb has finished
+        // with the file
+        first.close().unwrap();
+        let _second = dir.open(false).unwrap();
+    }
+
+    #[test]
+    fn a_directory_without_a_lock_file_is_not_a_database() {
+        let tmpdir = tempfile::tempdir().unwrap();
+        let path = tmpdir.path().join("db");
+        std::fs::create_dir(&path).unwrap();
+
+        assert!(DatabaseDir::new(&path).open(false).is_err());
+    }
+}