Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
f0d0c74
Add minimal Java build for storage implementations
sfloess Oct 20, 2026
e9f3154
Define minimal storage contract
sfloess Oct 20, 2026
31fe413
Implement simple in-memory store
sfloess Oct 20, 2026
e01fa38
Implement durable filesystem store
sfloess Oct 20, 2026
5ef6288
Add composable store decorator base
sfloess Oct 20, 2026
91b62b6
Add RAM cache decorator
sfloess Oct 20, 2026
c4e1fbc
Test RAM and filesystem store contract
sfloess Oct 20, 2026
74513a6
Test decorator cache coherence
sfloess Oct 20, 2026
e14a129
Document storage backends and decorator composition
sfloess Oct 20, 2026
2523027
Add Python package metadata
sfloess Oct 20, 2026
a332936
Expose Python storage API
sfloess Oct 20, 2026
a5b2a44
Define Python storage contract
sfloess Oct 20, 2026
35e4144
Implement Python RAM backend
sfloess Oct 20, 2026
015bd17
Implement Python filesystem backend
sfloess Oct 20, 2026
bf74f68
Implement composable Python decorators
sfloess Oct 20, 2026
0672ae1
Test Python RAM and filesystem contracts
sfloess Oct 20, 2026
9115866
Test Python cache decorator
sfloess Oct 20, 2026
ee75184
Decode filesystem IDs when listing records
sfloess Oct 20, 2026
777aa7b
Document Java and Python storage backends
sfloess Oct 20, 2026
1138104
Record Java and Python backend parity in storage ADR
sfloess Oct 20, 2026
5d438e9
Run Java and Python storage contract tests in CI
sfloess Oct 20, 2026
207cc53
Target Java 25 for memory implementation
sfloess Oct 20, 2026
1866ddf
Run Java tests on Java 25
sfloess Oct 20, 2026
39be4f2
Document Java 25 requirement
sfloess Oct 20, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
name: Tests

on:
push:
branches: ["main", "feat/**"]
pull_request:

jobs:
java:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v4
with:
distribution: temurin
java-version: "25"
cache: maven
- run: mvn --batch-mode test

python:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- run: python -m pip install --editable .
- run: python -m unittest discover -s tests -v
70 changes: 56 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,25 +2,67 @@

**Reusable memory capabilities for FlossWare applications.**

This repository is intended to provide explicit contracts and implementations for recording, retrieving, and managing application or agent memory.
This repository provides small storage contracts and implementations for recording and retrieving opaque memory payloads. Storage is a backend concern, not the memory retrieval/ranking policy or an orchestration runtime.

## Scope
## Initial storage implementations

- Memory item and session/context contracts
- Read, write, update, and lifecycle operations
- Retrieval interfaces and metadata
- Pluggable persistence backends
- Provenance, access controls, and retention semantics
- Tests for memory behavior independent of a specific application
Both Java and Python implementations provide the same minimal operations:

## Architectural boundaries
- `put(id, value)`: create or replace a UTF-8 text payload
- `get(id)`: return the payload or an explicit absent result
- `delete(id)`: delete a payload and report whether it existed
- `listIds()` / `list_ids()`: list IDs in deterministic sorted order

| Capability | Java | Python |
| --- | --- | --- |
| RAM backend | `RamMemoryStore` | `RamMemoryStore` |
| Filesystem backend | `FileSystemMemoryStore` | `FileSystemMemoryStore` |
| Decorator base | `DecoratingMemoryStore` | `DecoratingMemoryStore` |
| RAM cache decorator | `RamCacheMemoryStore` | `RamCacheMemoryStore` |

The RAM backend is process-local and ephemeral. Filesystem backends store one payload per file, encode IDs rather than treating them as paths, and atomically replace files where the platform supports it. Filesystem implementations are intended for single-process use and do not coordinate independent writers.

### Java

Java 25+, Maven:

```java
MemoryStore store = new RamCacheMemoryStore(
new FileSystemMemoryStore(Path.of("./memory-data"))
);
try (store) {
store.put("session-1", "remember this");
String value = store.get("session-1").orElseThrow();
}
```

Run tests with `mvn test`.

### Python

- `storage` provides persistence mechanisms; memory defines memory-specific semantics.
- `knowledge` manages durable, provenance-aware knowledge; memory focuses on remembered state and context. The exact boundary must be documented before overlapping APIs are introduced.
- `loom` and `loom-ai` define execution contracts. Memory is a capability Loom-compatible applications may invoke, not an orchestration runtime.
Python 3.10+, standard-library runtime, installable with pip:

```python
from flossware_memory import FileSystemMemoryStore, RamCacheMemoryStore

with RamCacheMemoryStore(FileSystemMemoryStore("./memory-data")) as store:
store.put("session-1", "remember this")
value = store.get("session-1")
```

Run tests with `python -m unittest discover -s tests`.

### Decorator semantics

The filesystem store remains authoritative in a RAM-cache/filesystem composition. The cache is read-through and write-through; puts update it only after the delegate write succeeds, and deletes evict cached values. Cache coherence applies to operations through the decorator. Changes made directly to the delegate are not observed by an existing cache. The outermost decorator owns and closes its delegate.

## Architectural boundaries

Do not assume memory must be a standalone service. Define a stable contract first, then support in-process or remote implementations as justified.
- Storage provides persistence mechanisms; memory defines memory-specific semantics.
- Knowledge manages durable, provenance-aware knowledge; memory focuses on remembered state and context. Document the boundary before introducing overlapping APIs.
- Loom and loom-ai define execution contracts. Memory is a capability Loom-compatible applications may invoke, not an orchestration runtime.
- The Java interface and Python protocol are implementation-level contracts. A language-neutral REST contract remains necessary for remote consumers; MCP and future protocol adapters should reuse the same service behavior.

## Status

Repository scaffold. No public API should be considered stable until contracts, backend behavior, and tests are present.
The initial RAM/filesystem backends and decorator examples are being established as reference implementations. The wider memory API, retrieval behavior, service deployment, and durable database backend remain separate work.
39 changes: 39 additions & 0 deletions docs/adr/0002-storage-implementations-and-decorators.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# ADR-0002: Minimal storage backends and composable decorators

- **Status:** Proposed
- **Date:** 2026-10-20

## Decision

Define a minimal storage contract for opaque UTF-8 text payloads keyed by caller-supplied string IDs. Provide self-written RAM and filesystem implementations in both Java and Python. Add a delegating decorator base and a RAM read-through/write-through cache decorator that can wrap any store implementation.

Java example:

```java
MemoryStore store = new RamCacheMemoryStore(new FileSystemMemoryStore(path));
```

Python example:

```python
store = RamCacheMemoryStore(FileSystemMemoryStore("./memory-data"))
```

The language-specific interfaces provide parity for in-process consumers. They are not substitutes for the eventual language-neutral remote service contract.

The filesystem remains authoritative in the cache/filesystem composition. The cache only observes operations made through the decorator and does not detect out-of-band changes.

## Constraints

- No framework, ORM, database, or third-party cache library in these reference backends.
- RAM storage is process-local and ephemeral.
- Filesystem storage writes a temporary sibling and renames it into place where supported. It is intended for single-process use and does not coordinate multiple processes.
- IDs are encoded into filenames, rather than used as paths, so path traversal is prevented.
- Listing returns IDs in deterministic sorted order.
- Decorators delegate lifecycle closure to the wrapped store; the outermost decorator owns the delegate.
- The same behavioral contract is tested against RAM and filesystem implementations in each language.
- Cache writes update the cache only after the delegate write succeeds. Deletes evict cached values. Out-of-band delegate changes are not automatically visible through an existing cache.

## Rationale

These implementations are deliberately small reference backends for local development, tests, and embedded use. They do not define memory retrieval/ranking policy and are not substitutes for the eventual PostgreSQL-backed service. Keeping Java and Python implementations behaviorally aligned supports both JVM and Python consumers without making either language the service-wide contract.
14 changes: 14 additions & 0 deletions flossware_memory/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
"""FlossWare Memory storage primitives."""

from .decorators import DecoratingMemoryStore, RamCacheMemoryStore
from .filesystem import FileSystemMemoryStore
from .ram import RamMemoryStore
from .storage import MemoryStore

__all__ = [
"DecoratingMemoryStore",
"FileSystemMemoryStore",
"MemoryStore",
"RamCacheMemoryStore",
"RamMemoryStore",
]
77 changes: 77 additions & 0 deletions flossware_memory/decorators.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
"""Composable decorators for MemoryStore implementations."""

from __future__ import annotations

from threading import RLock

from .storage import MemoryStore, validate_id, validate_value


class DecoratingMemoryStore(MemoryStore):
"""Delegating base decorator. The outer decorator owns the delegate lifecycle."""

def __init__(self, delegate: MemoryStore) -> None:
if delegate is None:
raise ValueError("delegate must not be None")
self._delegate = delegate

def put(self, memory_id: str, value: str) -> None:
self._delegate.put(memory_id, value)

def get(self, memory_id: str) -> str | None:
return self._delegate.get(memory_id)

def delete(self, memory_id: str) -> bool:
return self._delegate.delete(memory_id)

def list_ids(self) -> list[str]:
return self._delegate.list_ids()

def close(self) -> None:
self._delegate.close()


class RamCacheMemoryStore(DecoratingMemoryStore):
"""Read-through/write-through RAM cache around any MemoryStore.

The delegate remains authoritative. Cache coherence covers operations made
through this decorator only; out-of-band delegate changes are not observed.
"""

def __init__(self, delegate: MemoryStore) -> None:
super().__init__(delegate)
self._cache: dict[str, str] = {}
self._lock = RLock()

def put(self, memory_id: str, value: str) -> None:
checked_id = validate_id(memory_id)
checked_value = validate_value(value)
with self._lock:
self._delegate.put(checked_id, checked_value)
self._cache[checked_id] = checked_value

def get(self, memory_id: str) -> str | None:
checked_id = validate_id(memory_id)
with self._lock:
if checked_id in self._cache:
return self._cache[checked_id]
value = self._delegate.get(checked_id)
if value is not None:
self._cache[checked_id] = value
return value

def delete(self, memory_id: str) -> bool:
checked_id = validate_id(memory_id)
with self._lock:
deleted = self._delegate.delete(checked_id)
self._cache.pop(checked_id, None)
return deleted

def list_ids(self) -> list[str]:
with self._lock:
return self._delegate.list_ids()

def close(self) -> None:
with self._lock:
self._cache.clear()
super().close()
77 changes: 77 additions & 0 deletions flossware_memory/filesystem.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
"""Filesystem backend with safe ID encoding and atomic replacement."""

from __future__ import annotations

import os
import tempfile
from pathlib import Path

from .storage import MemoryStore, validate_id, validate_value


class FileSystemMemoryStore(MemoryStore):
"""One UTF-8 file per value.

Intended for use by one process. Atomic replacement is used where the platform
provides it; separate-process coordination and out-of-band change detection
are deliberately out of scope.
"""

def __init__(self, directory: str | os.PathLike[str]) -> None:
self._directory = Path(directory).expanduser().resolve()
self._directory.mkdir(parents=True, exist_ok=True)
if not self._directory.is_dir():
raise NotADirectoryError(str(self._directory))

def _path_for(self, memory_id: str) -> Path:
checked = validate_id(memory_id)
# IDs are data, never path fragments. Hex encoding prevents traversal and
# avoids filename portability problems with separators and special chars.
encoded = checked.encode("utf-8").hex()
path = (self._directory / f"{encoded}.mem").resolve()
if path.parent != self._directory:
raise ValueError("memory ID resolves outside storage directory")
return path

def put(self, memory_id: str, value: str) -> None:
target = self._path_for(memory_id)
data = validate_value(value).encode("utf-8")
fd, temporary_name = tempfile.mkstemp(prefix=".memory-", suffix=".tmp", dir=self._directory)
try:
with os.fdopen(fd, "wb") as stream:
stream.write(data)
stream.flush()
os.fsync(stream.fileno())
os.replace(temporary_name, target)
finally:
try:
os.unlink(temporary_name)
except FileNotFoundError:
pass

def get(self, memory_id: str) -> str | None:
path = self._path_for(memory_id)
try:
return path.read_text(encoding="utf-8")
except FileNotFoundError:
return None

def delete(self, memory_id: str) -> bool:
try:
self._path_for(memory_id).unlink()
return True
except FileNotFoundError:
return False

def list_ids(self) -> list[str]:
ids: list[str] = []
for path in self._directory.glob("*.mem"):
try:
encoded = path.name[:-4]
memory_id = bytes.fromhex(encoded).decode("utf-8")
if self._path_for(memory_id) == path.resolve():
ids.append(memory_id)
except (ValueError, UnicodeDecodeError):
# Ignore unrelated or malformed files in the storage directory.
continue
return sorted(ids)
31 changes: 31 additions & 0 deletions flossware_memory/ram.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
"""Simple process-local RAM backend."""

from __future__ import annotations

from threading import RLock

from .storage import MemoryStore, validate_id, validate_value


class RamMemoryStore(MemoryStore):
"""Thread-safe in-process dictionary; contents are intentionally ephemeral."""

def __init__(self) -> None:
self._values: dict[str, str] = {}
self._lock = RLock()

def put(self, memory_id: str, value: str) -> None:
with self._lock:
self._values[validate_id(memory_id)] = validate_value(value)

def get(self, memory_id: str) -> str | None:
with self._lock:
return self._values.get(validate_id(memory_id))

def delete(self, memory_id: str) -> bool:
with self._lock:
return self._values.pop(validate_id(memory_id), None) is not None

def list_ids(self) -> list[str]:
with self._lock:
return sorted(self._values)
Loading
Loading