diff --git a/README.md b/README.md index b0c1be0..7bccc90 100644 --- a/README.md +++ b/README.md @@ -5,9 +5,9 @@ **Bidirectional, JSON-patch-compliant diffs between Python data structures.** -๐Ÿ“– [Documentation](https://fork-tongue.github.io/patchdiff/) โ€” [Quick Start](https://fork-tongue.github.io/patchdiff/getting-started/quick-start/) โ€” [API Reference](https://fork-tongue.github.io/patchdiff/reference/api/) +๐Ÿ“– [Documentation](https://fork-tongue.github.io/patchdiff/) | [Quick Start](https://fork-tongue.github.io/patchdiff/getting-started/quick-start/) | [API Reference](https://fork-tongue.github.io/patchdiff/reference/api/) -Patchdiff diffs composite structures of dicts, lists, sets and tuples, and gives you **both directions** in one call: the patches to get from `input` to `output`, and the patches to get back. Patches are [RFC 6902](https://datatracker.ietf.org/doc/html/rfc6902) JSON-patch style, serializable to JSON, and can be applied in place or to a copy โ€” which makes undo/redo, change synchronization and state auditing one-liners. +Patchdiff diffs composite structures of dicts, lists, sets and tuples, and gives you **both directions** in one call: the patches to get from `input` to `output`, and the patches to get back. Patches are [RFC 6902](https://datatracker.ietf.org/doc/html/rfc6902) JSON-patch style, serializable to JSON, and can be applied in place or to a copy, which makes undo/redo, change synchronization and state auditing one-liners. ```python from patchdiff import apply, diff, iapply, to_json @@ -26,7 +26,7 @@ assert input == output print(to_json(ops, indent=4)) ``` -## Don't diff โ€” record +## Proxy based patch generation When your own code makes the changes, `produce()` (inspired by [Immer](https://immerjs.github.io/immer/produce)) skips the comparison entirely: it hands your recipe a draft, records every mutation, and returns the result plus both patch directions. Cost scales with the number of mutations instead of the size of the state: @@ -45,7 +45,7 @@ assert base == {"count": 0, "items": [1, 2, 3]} # base is untouched assert result == {"count": 5, "items": [1, 2, 3, 4]} ``` -With `in_place=True`, mutations (and patches applied with `iapply`) write straight through proxy-backed state โ€” the natural companion to [observ](https://github.com/fork-tongue/observ) reactive objects, where mutating through the proxy is what triggers watchers. See [Observ Integration](https://fork-tongue.github.io/patchdiff/guide/observ/) for reactive state with undo/redo. +With `in_place=True`, mutations (and patches applied with `iapply`) write straight through proxy-backed state, the natural companion to [observ](https://github.com/fork-tongue/observ) reactive objects, where mutating through the proxy is what triggers watchers. See [Observ Integration](https://fork-tongue.github.io/patchdiff/guide/observ/) for reactive state with undo/redo. ## Install diff --git a/ROADMAP.md b/ROADMAP.md deleted file mode 100644 index 76c89c0..0000000 --- a/ROADMAP.md +++ /dev/null @@ -1,179 +0,0 @@ -# Roadmap to patchdiff 1.0.0 - -This roadmap takes patchdiff from 0.3.x to its first stable release, following the same -program that took [observ](https://github.com/fork-tongue/observ) to 1.0.0 (observ -PRs #175โ€“#197): a documentation site, full modern typing, hardened test and benchmark -suites, modern CI, and several rounds of foundational and micro optimization โ€” each item -delivered as its own PR. - -## Where patchdiff stands today (0.3.12) - -- **Code**: `pointer.py`, `diff.py` (O(mยทn) DP list diff with prefix/suffix trim), - `apply.py` (per-patch `hasattr` dispatch, unconditional `deepcopy` of values), - `produce.py` (Immer-style proxies with parent-link path tracking), `serialize.py`, - `types.py`. -- **Typing**: old-style (`List`, `Dict`, bare `List` for op lists), no TypedDicts for - operations, no `py.typed` marker, no type checker in CI. -- **Tests**: ~6,400 lines with 100% coverage enforced in CI โ€” already strong, but - missing property-based round-trip fuzzing and an RFC 6902 compliance corpus. -- **Benchmarks**: solid suite (list/dict/set diff, apply, pointer, produce-vs-diff), but - the CI guard is the design observ replaced in #181/#187: a single master run vs a - single PR run at `--benchmark-compare-fail=mean:5%`, which sits below the noise floor - of GitHub-hosted runners. -- **CI**: one generation behind observ #182 (checkout v6, setup-uv v8.1.0, no uv - caching, redundant steps). -- **Docs**: README only; no docs site. - -## The PR series - -Five tracks, roughly in order: infrastructure first (the rest relies on it), then -docs / typing / test hardening in any order, then the optimization iterations (measured -by the new benchmark guard and locked in by typing and property tests), capped by the -release PR. - -### Track A โ€” CI & benchmark infrastructure - -**PR 1 โ€” Modernize CI workflows** *(port of observ #182)* - -- Bump all actions in `ci.yml` and `benchmark.yml`: checkout v7, setup-uv v8.3.2 - (SHA-pinned), upload-artifact v7 / download-artifact v8, action-gh-release v3 - (SHA-pinned). -- Enable uv caching (`enable-cache: true`, keyed on `pyproject.toml`, with a - per-Python-version suffix in the test matrix). -- Trim steps: drop `uv sync` in the build job (`uv build` uses an isolated build env), - drop checkout in the publish job (it only needs the dist artifact), drop the - benchmark artifact upload. - -**PR 2 โ€” Noise-robust benchmark guard** *(port of observ #181 + #187)* - -- Replace the single-run `--benchmark-compare-fail=mean:5%` gate with: - - interleaved master/PR benchmark runs (A/B/A/B/A/B, 3 runs each), - - `--benchmark-disable-gc` and `PYTHONHASHSEED=0` to remove the two dominant noise - sources, - - a `benchmarks/compare_runs.py` that fails a benchmark only when the *fastest* PR - run is more than 25% slower than the *slowest* master run (per-benchmark medians). -- Document the threshold for what it is: a tripwire for gross accidental regressions, - not a precision instrument. Deltas below it are measured deliberately with repeated - local runs. - -### Track B โ€” Documentation - -**PR 3 โ€” MkDocs docs site + GitHub Pages deployment** *(port of observ #176)* - -- MkDocs + Material theme + mkdocstrings (`mkdocs.yml` in the same shape as observ's), - a `docs` dependency group, and a `docs.yml` workflow that builds with - `uv sync --only-group docs` and deploys via `upload-pages-artifact`/`deploy-pages`. -- Content: - - *Getting Started*: installation, quick start. - - *Guide*: diffing, applying patches (`apply` vs `iapply`), JSON pointers, - `produce()` and drafts (including `in_place`), serialization (`to_json`), - observ integration, gotchas & best practices (set/tuple extensions vs strict - RFC 6902, patch value snapshotting). - - *Reference*: API reference generated from docstrings; add docstrings to `diff`, - `apply`, `iapply`, `produce`, `to_json` and `Pointer` where missing (no behavior - changes). -- Every code example executed and verified; the site builds warning-free with the exact - CI recipe. Before the first deploy, GitHub Pages must be set to "GitHub Actions" as - the source in the repository settings. - -**PR 4 โ€” Internals docs section** *(analog of observ #179)* - -- Architecture documentation: the DP list diff with prefix/suffix trimming, traceback - and index padding; reverse-op construction; `Pointer.evaluate` semantics (strict - parent walk, tolerant leaf); and `produce`'s proxy design (parent links, on-demand - path computation, detach semantics, `_snapshot`/`_unwrap`, the `PatchRecorder`). - -**PR 5 โ€” README rewrite** *(analog of observ #180)* - -- Links up front (docs site, PyPI, CI), a sharper pitch, slimmed to pitch + quick - example pointing at the docs site, and a referral to sibling project observ. - -### Track C โ€” Typing - -**PR 6 โ€” Runtime cleanups split out from the typing work** *(analog of observ #184)* - -- Only the behavioral changes the typing PR needs, reviewable in isolation: introduce - `Operation` TypedDict shapes in `types.py` (add/remove/replace ops carrying - `path: Pointer`), use them in `serialize.py`, and normalize any signatures that - type-check poorly. No public API changes. - -**PR 7 โ€” Full modern type hinting, checked with `ty` in CI** *(analog of observ #185, stacked on PR 6)* - -- PEP 604 unions and builtin generics everywhere via `from __future__ import - annotations` (the runtime floor stays Python 3.9); precise signatures โ€” - `diff(...) -> tuple[list[Operation], list[Operation]]`, - `produce(...) -> tuple[Diffable, list[Operation], list[Operation]]`; a `py.typed` - marker in the package. -- A `ty` dependency group, `[tool.ty]` config in `pyproject.toml` scoped to - `patchdiff/` with `python-version = "3.9"`, and a dedicated Typecheck job in CI added - to `publish`'s `needs`. -- Fix whatever genuine annotation bugs `ty` surfaces (observ found several). - -### Track D โ€” Test hardening - -**PR 8 โ€” Property-based round-trip tests + RFC 6902 compliance corpus** - -- Hypothesis-based property tests (new dev dependency): for randomly generated nested - structures of dicts, lists, sets, tuples and scalars, assert - `apply(input, ops) == output`, `apply(output, rops) == input`, `iapply` equivalence, - `to_json` round-trips through `Pointer.from_str`, and pointer escape/unescape - round-trips. -- Vendor the applicable cases of the - [json-patch-tests](https://github.com/json-patch/json-patch-tests) corpus; patchdiff - deliberately extends RFC 6902 (sets, tuples), so divergent cases are skipped with - documented reasons. -- The analog, in spirit, of observ #189: test the library's core promise directly. - -### Track E โ€” Optimization iterations - -Each PR carries benchmark numbers in its body and is guarded by PR 2's tripwire. Two -foundational-design passes, then micro passes over each hot path. - -**PR 9 โ€” Foundational: replace the O(mยทn) DP list diff with Myers O(ND)** - -- `diff_lists` currently builds a full DP table over the changed region. With - prefix/suffix trimming already in place, switch the changed-region diff to Myers' - greedy algorithm (linear-space refinement if warranted). Patch semantics stay - identical โ€” the existing tests plus PR 8's properties pin behavior. Expected large - wins on the `list-diff-similar` benchmark groups. - -**PR 10 โ€” Foundational: `produce()` hot-path design pass** - -- Design-level wins in the proxy layer: avoid per-write `Pointer` allocation (record - token paths as tuples until finalize), skip `_snapshot` for scalar values at the - call site, cache `_location` walks where safe. Measured with the produce benchmark - groups. - -**PR 11 โ€” Micro: diff + pointer pass** *(analog of observ's #190/#191/#194โ€“#196 cluster)* - -- Inner-loop trims in `diff_dicts`/`diff_sets` and the `diff_lists` traceback: op dict - construction, bound methods, positional calls; `Pointer.append`/`evaluate`/`__str__` - micro-costs. - -**PR 12 โ€” Micro: apply/iapply pass** - -- Dispatch on the parent type once per patch instead of up to three `hasattr` checks, - skip `deepcopy` for immutable scalar values, hoist the int-key conversion. - -**PR 13 โ€” Micro: produce trap-level pass** *(analog of observ #192/#193)* - -- Cut per-call overhead in the `DictProxy`/`ListProxy`/`SetProxy` methods: no - `**kwargs` where the wrapped method doesn't accept keywords, bind hot attributes to - locals/closures, hoist repeated lookups out of loops; pin the call conventions with - signature tests as observ did. - -### Release - -**PR 14 โ€” Bump version to 1.0.0** *(analog of observ #197)* - -- Version bump with a release-notes body covering features, correctness, docs and - tooling, and headline before/after benchmark tables (median times, both versions run - interleaved locally). - -## Ground rules throughout - -- Every change lands as its own reviewable PR; stacking only where required (PR 7 on - PR 6). -- Optimization PRs land after PR 2 (trustworthy guard), PR 7 (typecheck holds the - annotations) and PR 8 (property tests as the safety net). -- 100% test coverage stays enforced (`--cov-fail-under=100`) for the entire series. diff --git a/docs/getting-started/installation.md b/docs/getting-started/installation.md index ec1b3ab..ff7921d 100644 --- a/docs/getting-started/installation.md +++ b/docs/getting-started/installation.md @@ -1,6 +1,6 @@ # Installation -Patchdiff is published on [PyPI](https://pypi.org/project/patchdiff/) and has **no dependencies**. It requires Python 3.9 or newer. +Patchdiff is published on [PyPI](https://pypi.org/project/patchdiff/): ```sh pip install patchdiff @@ -12,4 +12,6 @@ Or with [uv](https://docs.astral.sh/uv/): uv add patchdiff ``` -That's it โ€” head over to the [Quick Start](quick-start.md). +There are no dependencies. Python 3.9 or newer is required. + +Next up: the [quick start](quick-start.md). diff --git a/docs/getting-started/quick-start.md b/docs/getting-started/quick-start.md index 3ef153a..85bee83 100644 --- a/docs/getting-started/quick-start.md +++ b/docs/getting-started/quick-start.md @@ -1,8 +1,8 @@ # Quick Start -This page walks through the whole API in a few minutes: diff two structures, apply the patches, undo them, and serialize them to JSON. +This page walks through the whole API: diff two objects, apply the patches, undo them, and serialize them to JSON. -## Diff two structures +## Diffing [`diff`][patchdiff.diff.diff] compares two objects and returns two lists of operations: one that turns `input` into `output`, and one that turns `output` back into `input`. @@ -15,7 +15,7 @@ output = {"a": [5, 2, 9, {"b", "c"}], "b": 6, "c": 7} ops, reverse_ops = diff(input, output) ``` -Each operation is a plain dict in JSON patch style โ€” an `"op"` (`"add"`, `"remove"` or `"replace"`), a `"path"` (a [`Pointer`][patchdiff.pointer.Pointer]), and a `"value"` for adds and replaces: +Each operation is a plain dict in JSON patch style, with an `"op"` (`"add"`, `"remove"` or `"replace"`), a `"path"` (a [`Pointer`][patchdiff.pointer.Pointer]), and a `"value"` for adds and replaces: ```python from patchdiff import diff @@ -27,9 +27,9 @@ assert ops == [{"op": "replace", "path": Pointer(["count"]), "value": 1}] assert reverse_ops == [{"op": "replace", "path": Pointer(["count"]), "value": 0}] ``` -## Apply patches +## Applying patches -[`apply`][patchdiff.apply.apply] patches a **deep copy** and leaves the original untouched; [`iapply`][patchdiff.apply.iapply] patches the object **in place**: +[`apply`][patchdiff.apply.apply] patches a **deep copy** and leaves the original untouched, while [`iapply`][patchdiff.apply.iapply] patches the object **in place**: ```python from patchdiff import apply, diff, iapply @@ -46,11 +46,11 @@ iapply(input, ops) # in-place assert input == output ``` -Applying `reverse_ops` is your undo; re-applying `ops` is your redo. +Applying `reverse_ops` is your undo, re-applying `ops` is your redo. -## Serialize to JSON +## Serializing to JSON -Patches are JSON-patch compliant, so they can be serialized with [`to_json`][patchdiff.serialize.to_json] and shipped anywhere: +Patches are JSON-patch compliant, so they can be serialized with [`to_json`][patchdiff.serialize.to_json]: ```python from patchdiff import diff, to_json @@ -75,9 +75,9 @@ print(to_json(ops, indent=4)) ] ``` -## Record patches while mutating +## Recording patches while mutating -When you're the one making the changes, you don't need to diff at all: [`produce`][patchdiff.produce.produce] hands you a draft, records every mutation you make to it, and gives you the result plus both patch directions โ€” without a full comparison pass: +When your own code is making the changes, you don't need to diff at all. [`produce`][patchdiff.produce.produce] hands you a draft, records every mutation you make to it, and gives you the result plus both patch directions, without a full comparison pass: ```python from patchdiff import produce @@ -95,4 +95,4 @@ assert base == {"count": 0, "items": [1, 2, 3]} # untouched assert result == {"count": 5, "items": [1, 2, 3, 4], "new_field": "hello"} ``` -Read on in the [Guide](../guide/diffing.md) for the details of each of these, or jump to the [API Reference](../reference/api.md). +The [guide](../guide/diffing.md) goes into the details of each of these. The complete public API is in the [API reference](../reference/api.md). diff --git a/docs/guide/applying.md b/docs/guide/applying.md index c0dc18d..a47967b 100644 --- a/docs/guide/applying.md +++ b/docs/guide/applying.md @@ -2,7 +2,7 @@ Patchdiff can apply a list of operations to an object in two ways: -* [`apply`][patchdiff.apply.apply] first makes a **deep copy**, patches that, and returns it โ€” the input is left untouched. +* [`apply`][patchdiff.apply.apply] first makes a **deep copy**, patches that, and returns it. The input is left untouched. * [`iapply`][patchdiff.apply.iapply] patches the object **in place** and returns the same object. This is faster (no copy) and is what you want for objects that must keep their identity, such as observ reactive proxies. ```python @@ -24,11 +24,11 @@ assert input == output ## Order matters -Operations are applied sequentially, and paths refer to the state of the object *at that point in the sequence* โ€” exactly like RFC 6902. List indices in particular shift as adds and removes are applied, and both lists that [`diff`][patchdiff.diff.diff] returns are already ordered accordingly. Apply them as-is; don't reorder or cherry-pick individual operations. +Operations are applied sequentially, and paths refer to the state of the object at that point in the sequence, just like in RFC 6902. List indices in particular shift as adds and removes are applied. Both lists that [`diff`][patchdiff.diff.diff] returns are already ordered accordingly, so apply them as-is. Don't reorder or cherry-pick individual operations. ## Undo and redo -Because every diff comes with its reverse, undo/redo is just a pair of stacks of patch lists: +Since every diff comes with its reverse, undo/redo is just a pair of stacks of patch lists: ```python from patchdiff import diff, iapply @@ -63,7 +63,7 @@ assert state == {"count": 2} ## Patch values are copied on write -When a patch is applied, its `"value"` is **deep-copied** before being written into the target. The patched object and the patch list therefore never share mutable state โ€” you can keep patches around (say, on an undo stack) and freely mutate the object afterwards: +When a patch is applied, its `"value"` is **deep-copied** before being written into the target. The patched object and the patch list never share mutable state, so you can keep patches around (on an undo stack for example) and freely mutate the object afterwards: ```python from patchdiff import diff, iapply diff --git a/docs/guide/diffing.md b/docs/guide/diffing.md index d6ea73f..d9c3feb 100644 --- a/docs/guide/diffing.md +++ b/docs/guide/diffing.md @@ -1,6 +1,6 @@ # Diffing -[`diff`][patchdiff.diff.diff] recursively compares two objects and emits JSON-patch-style operations in **both directions**: +[`diff`][patchdiff.diff.diff] recursively compares two objects and emits JSON-patch style operations in **both directions**: ```python from patchdiff import apply, diff @@ -11,19 +11,19 @@ assert apply({"a": 1}, ops) == {"a": 2, "b": 3} assert apply({"a": 2, "b": 3}, reverse_ops) == {"a": 1} ``` -The comparison starts with a plain equality check โ€” if `input == output`, both lists are empty. Otherwise the strategy depends on the types involved. +The comparison starts with a plain equality check: if `input == output`, both lists are empty. Otherwise the strategy depends on the types involved. ## What gets compared structurally -Containers are compared recursively when **both sides** are container-like (duck-typed, so third-party proxies such as observ's reactive objects work too): +Containers are compared recursively when **both sides** are container-like. The check is duck-typed, so third-party proxies such as observ's reactive objects work too: -| both sides haveโ€ฆ | treated as | operations emitted | +| both sides have | treated as | operations emitted | |---|---|---| | `.append` | list | minimal edit script: adds, removes, replaces per index | | `.keys` | dict | add/remove per key, recursion into common keys | | `.add` | set | add/remove per element | -Anything else โ€” scalars, but also **tuples and frozensets**, or two containers of different kinds โ€” is treated as an atomic value and replaced wholesale: +Anything else (scalars, but also tuples and frozensets, or two containers of different kinds) is treated as an atomic value and replaced wholesale: ```python from patchdiff import diff @@ -49,7 +49,7 @@ ops, _ = diff( assert to_json(ops) == '[{"op": "replace", "path": "/1/name", "value": "b"}]' ``` -Insertions at the end use the JSON pointer `-` token (RFC 6901 for "append"): +Insertions at the end use the `-` token from RFC 6901, which means "append": ```python from patchdiff import diff, to_json @@ -61,7 +61,7 @@ assert to_json(ops) == '[{"op": "add", "path": "/-", "value": 3}]' ## Dicts -Keys only in the input become removes, keys only in the output become adds, and common keys are diffed recursively โ€” so nested changes produce deep paths rather than replacing whole subtrees: +Keys only in the input become removes, keys only in the output become adds, and common keys are diffed recursively, so nested changes produce deep paths rather than replacing whole subtrees: ```python from patchdiff import diff, to_json @@ -76,7 +76,7 @@ assert to_json(ops) == '[{"op": "replace", "path": "/user/age", "value": 41}]' ## Sets -Sets have no indices, so patchdiff extends JSON patch slightly: an element is **added** with the `-` token (like a list append) and **removed** by addressing the element itself as the final path token: +Sets have no indices or keys, so patchdiff extends JSON patch slightly: an element is **added** with the `-` token (like a list append) and **removed** by addressing the element itself as the final path token: ```python from patchdiff import diff @@ -90,11 +90,11 @@ assert ops == [ ] ``` -See [Gotchas](gotchas.md) for the places where this deliberately diverges from strict RFC 6902. +See [gotchas](gotchas.md) for the places where this deliberately diverges from strict RFC 6902. ## Reverse operations -The second list `diff` returns is not just the first with `add`/`remove` swapped โ€” the operations are also **ordered for reverse application**, so indices resolve correctly as each patch is applied. Always apply `reverse_ops` as-is, in order: +The second list that `diff` returns is not just the first with `add`/`remove` swapped. The operations are also **ordered for reverse application**, so that indices resolve correctly as each patch is applied. Always apply `reverse_ops` as-is, in order: ```python from patchdiff import apply, diff diff --git a/docs/guide/gotchas.md b/docs/guide/gotchas.md index 557e469..eecb84e 100644 --- a/docs/guide/gotchas.md +++ b/docs/guide/gotchas.md @@ -1,51 +1,51 @@ -# Gotchas and Best Practices +# Gotchas ## Where patchdiff diverges from strict RFC 6902 -Patchdiff's patches are JSON-patch *compliant* for JSON-shaped data (dicts with string keys, lists, scalars), but the library deliberately supports more of Python than JSON has: +Patchdiff's patches are JSON-patch compliant for JSON-shaped data (dicts with string keys, lists, scalars), but the library deliberately supports more of Python than JSON has: * **Sets** are diffed and patched natively: elements are added with the `-` token and removed by addressing the element value itself as the final path token. Strict RFC 6902 has no set concept at all. -* **Tuples and frozensets** are treated as atomic values โ€” they are never diffed into, only replaced wholesale. -* **Pointer tokens can be non-strings** (integer list indices, set members). They stringify losslessly for lists, but set-member tokens can't be parsed back from a string โ€” see [Serialization](serialization.md#non-json-values). +* **Tuples and frozensets** are treated as atomic values. They are never diffed into, only replaced wholesale. +* **Pointer tokens can be non-strings** (integer list indices, set members). They stringify losslessly for lists, but set-member tokens can't be parsed back from a string. See [serialization](serialization.md#non-json-values). * **Only `add`, `remove` and `replace` are emitted.** `move`, `copy` and `test` from RFC 6902 are neither generated nor understood by [`apply`][patchdiff.apply.apply]/[`iapply`][patchdiff.apply.iapply]. -* **Operations on the document root are not supported.** Patches address locations *inside* a container. Diffing two documents of different top-level kinds (say a list against a dict) yields a whole-document `replace` at the root, which `apply`/`iapply` cannot execute โ€” keep the top-level type of your state stable. +* **Operations on the document root are not supported.** Patches address locations *inside* a container. Diffing two documents of different top-level kinds (say a list against a dict) yields a whole-document `replace` at the root, which `apply`/`iapply` cannot execute, so keep the top-level type of your state stable. If you feed patches to a strict third-party JSON patch implementation, stick to JSON-shaped data and everything lines up. ## Apply patches in order, as a unit -Paths refer to the state of the object *at that point in the patch sequence* โ€” list indices shift as adds and removes apply. Both lists returned by [`diff`][patchdiff.diff.diff] and [`produce`][patchdiff.produce.produce] are ordered for exactly this; reordering or cherry-picking operations from the middle of a list will corrupt paths. +Paths refer to the state of the object at that point in the patch sequence; list indices shift as adds and removes apply. Both lists returned by [`diff`][patchdiff.diff.diff] and [`produce`][patchdiff.produce.produce] are ordered for exactly this, so reordering or cherry-picking operations from the middle of a list will corrupt paths. Also, reverse operations undo the *whole* forward list, not individual forward ops one-for-one. ## Diffing compares by equality -`diff` starts with `input == output`. Anything Python considers equal produces no patch โ€” including e.g. `1 == True` and `0.0 == 0`. If you need to normalize such values, do it before diffing. +`diff` starts with `input == output`. Anything Python considers equal produces no patch, including e.g. `1 == True` and `0.0 == 0`. If you need to normalize such values, do it before diffing. ## Patches never share state with your objects -Patch values are snapshotted when recorded (by `produce`) and deep-copied when applied (by `apply`/`iapply`). You can keep patch lists on an undo stack indefinitely and freely mutate your state โ€” they won't drift. The flip side: don't rely on object identity surviving a round-trip through patches; equality survives, identity doesn't. +Patch values are snapshotted when recorded (by `produce`) and deep-copied when applied (by `apply`/`iapply`). You can keep patch lists on an undo stack indefinitely and freely mutate your state, they won't drift. On the other hand, don't rely on object identity surviving a round-trip through patches; equality survives, identity doesn't. ## `produce` drafts don't outlive the recipe -The draft proxy (and everything you read from it) is only wired up while the recipe runs. When [`produce`][patchdiff.produce.produce] returns, proxies are released; use the returned `result` instead of stashing draft references. Values you *detach* from the draft during the recipe (e.g. `popped = draft["items"].pop()`) stop recording โ€” reinserting `popped` later records its plain data, which is usually what you want. +The draft proxy (and everything you read from it) is only wired up while the recipe runs. When [`produce`][patchdiff.produce.produce] returns, proxies are released, so use the returned `result` instead of holding on to draft references. Values you *detach* from the draft during the recipe (e.g. `popped = draft["items"].pop()`) stop recording; reinserting `popped` later records its plain data, which is usually what you want. ## Choose `diff` or `produce` deliberately -* Use **`diff`** when you receive two complete states (e.g. from a form, a file, an API response). Cost scales with the size of the structures. -* Use **`produce`** when your code performs the mutations. Cost scales with the number of mutations, which is usually far smaller than the state. +* Use `diff` when you receive two complete states (e.g. from a form, a file, an API response). Cost scales with the size of the structures. +* Use `produce` when your own code performs the mutations. Cost scales with the number of mutations, which is usually far smaller than the state. The `produce-vs-diff` benchmark groups in the repository quantify the difference for typical shapes. ## Use `in_place=True` for proxy-backed state -For observ reactive objects (or anything where identity and write-through behavior matter), pass `in_place=True` to `produce` and use `iapply` rather than `apply` โ€” both write through the original object instead of replacing it. See [Observ Integration](observ.md). +For observ reactive objects (or anything where identity and write-through behavior matter), pass `in_place=True` to `produce` and use `iapply` rather than `apply`. Both write through the original object instead of replacing it. See [observ integration](observ.md). ## Keep the draft a tree, not a graph `produce` assumes the structure it's wrapping is a tree: every dict, list, or set reachable from the draft has exactly one path down from the root. Shared references (the same object reachable from two locations) and cycles aren't detected or specially handled, and they cause two distinct problems depending on where the sharing comes from. -**Aliasing already in your base object is silently dropped by the default (copy) mode.** Immutable mode copies the structure key by key rather than as one graph-aware deep copy, so two keys that pointed at the same object going in point at two independent copies coming out โ€” even if the recipe makes no changes at all: +**Aliasing already present in your base object is silently dropped by the default (copy) mode.** Immutable mode copies the structure key by key rather than as one graph-aware deep copy, so two keys that pointed at the same object going in point at two independent copies coming out, even if the recipe makes no changes at all: ```python from patchdiff import produce @@ -60,7 +60,7 @@ assert result == {"a": {"x": 1}, "b": {"x": 1}} assert result["a"] is not result["b"] # identity silently lost ``` -If your data model relies on that shared reference (a config object reused across several slots, a node referenced from two places), the default mode quietly forks it. Use `in_place=True` if identity must survive โ€” it mutates `base` directly, so pre-existing aliasing is preserved. +If your data model relies on that shared reference (a config object reused across several slots, a node referenced from two places), the default mode quietly forks it. Use `in_place=True` if identity must survive, since that mutates `base` directly and pre-existing aliasing is preserved. **Aliasing does survive `in_place=True`, but the recorded patches still won't capture the full mutation.** Each proxy only records what it directly observes, so if two drafted locations share an object, a write made through one location is invisible to the other's patch trail: @@ -83,8 +83,8 @@ assert to_json(patches) == ( ) # the patch list never mentions /b/x ``` -`result` is correct in your process because it's the same object either way. But replay those `patches` against a fresh, non-aliased copy of `base` โ€” which is what any JSON-based consumer gets, since JSON has no notion of shared references โ€” and `/b/x` never gets set: the replayed state ends up as `{"a": {"x": 2}, "b": {"y": 3}}`, silently missing `/b/x`. +`result` is correct in your process because it's the same object either way. But replay those `patches` against a fresh, non-aliased copy of `base` (which is what any JSON-based consumer gets, since JSON has no notion of shared references) and `/b/x` never gets set: the replayed state ends up as `{"a": {"x": 2}, "b": {"y": 3}}`, silently missing `/b/x`. The same reasoning applies to reference cycles (an object nested inside itself, directly or via another container): the draft's proxies track a parent chain and don't detect or break cycles, so building one during a recipe leads to unbounded recursion rather than a clean error. -It's on you to keep the object graph you hand to `produce` acyclic and reference-free โ€” i.e. a proper tree, not a graph. If the same data needs to live in two places, assign independent copies (`copy.deepcopy`, or your own constructor call) rather than the same object twice. +In short, keep the object graph you hand to `produce` acyclic and free of shared references, i.e. a proper tree. If the same data needs to live in two places, assign independent copies (`copy.deepcopy`, or your own constructor call) rather than the same object twice. diff --git a/docs/guide/observ.md b/docs/guide/observ.md index e69a616..2c2fe0e 100644 --- a/docs/guide/observ.md +++ b/docs/guide/observ.md @@ -1,12 +1,12 @@ # Observ Integration -[observ](https://github.com/fork-tongue/observ) provides reactive state for Python: mutate a `reactive` proxy and watchers and computed values update automatically. Patchdiff is observ's natural companion โ€” it turns those same mutations into patches, which is how you add **undo/redo or change synchronization on top of reactive state**. +[observ](https://github.com/fork-tongue/observ) provides reactive state for Python: mutate a `reactive` proxy and watchers and computed values update automatically. Patchdiff turns those same mutations into patches, which is how you add **undo/redo or change synchronization on top of reactive state**. -Patchdiff has no hard dependency on observ; everything on this page also degrades gracefully to plain dicts and lists. +Patchdiff has no hard dependency on observ; everything on this page also works on plain dicts and lists. ## Recording patches on reactive state -Use [`produce`][patchdiff.produce.produce] with `in_place=True`. The mutations are written *through* observ's reactive proxy, so watchers fire exactly as if you had mutated the state directly โ€” and you get both patch directions for free: +Use [`produce`][patchdiff.produce.produce] with `in_place=True`. The mutations are written *through* observ's reactive proxy, so watchers fire exactly as if you had mutated the state directly, and you get both patch directions for free: ```python from observ import reactive, watch @@ -34,7 +34,7 @@ assert observed == [5] # ...and the watcher fired ``` !!! note "Why `in_place=True`?" - Without it, `produce` copies the state and mutates the copy โ€” the reactive object stays untouched and no watcher fires. In-place mode is also faster, since it skips the deep copy. + Without it, `produce` copies the state and mutates the copy, so the reactive object stays untouched and no watcher fires. In-place mode is also faster, since it skips the deep copy. ## Undo/redo for reactive state @@ -64,7 +64,7 @@ assert state["todos"] == ["write docs", "release 1.0"] ## Diffing reactive state -[`diff`][patchdiff.diff.diff] duck-types its inputs, so observ proxies can be diffed directly โ€” against plain data or other proxies: +[`diff`][patchdiff.diff.diff] duck-types its inputs, so observ proxies can be diffed directly, against plain data or other proxies: ```python from observ import reactive diff --git a/docs/guide/pointers.md b/docs/guide/pointers.md index 93acb98..6c6cb33 100644 --- a/docs/guide/pointers.md +++ b/docs/guide/pointers.md @@ -1,6 +1,6 @@ # JSON Pointers -Every operation's `"path"` is a [`Pointer`][patchdiff.pointer.Pointer] โ€” patchdiff's implementation of a JSON pointer ([RFC 6901](https://datatracker.ietf.org/doc/html/rfc6901)): a sequence of *reference tokens* that address a location inside a nested structure. +Every operation's `"path"` is a [`Pointer`][patchdiff.pointer.Pointer], patchdiff's implementation of a JSON pointer ([RFC 6901](https://datatracker.ietf.org/doc/html/rfc6901)): a sequence of *reference tokens* that address a location inside a nested structure. ```python from patchdiff import diff @@ -14,7 +14,7 @@ assert str(ptr) == "/a/b/0" ## Immutable, hashable, comparable -Pointers are immutable โ€” [`append`][patchdiff.pointer.Pointer.append] returns a *new* pointer โ€” and they support equality and hashing, so they can be dict keys or set members: +Pointers are immutable ([`append`][patchdiff.pointer.Pointer.append] returns a *new* pointer) and they support equality and hashing, so they can be used as dict keys or set members: ```python from patchdiff.pointer import Pointer @@ -29,7 +29,7 @@ assert str(child) == "/a/0" ## String form and escaping -`str(pointer)` renders the RFC 6901 string form, escaping `~` as `~0` and `/` as `~1` inside tokens; [`Pointer.from_str`][patchdiff.pointer.Pointer.from_str] parses one back: +`str(pointer)` renders the RFC 6901 string form, escaping `~` as `~0` and `/` as `~1` inside tokens. [`Pointer.from_str`][patchdiff.pointer.Pointer.from_str] parses one back: ```python from patchdiff.pointer import Pointer @@ -39,11 +39,11 @@ assert str(ptr) == "/a~1b/c~0d" assert Pointer.from_str("/a~1b/c~0d").tokens == ("a/b", "c~d") ``` -Note that parsing is string-typed: `from_str` cannot know whether `"0"` was a list index or a dict key, so all parsed tokens are strings. That's fine for applying patches โ€” [`iapply`][patchdiff.apply.iapply] converts numeric-looking keys back to list indices as needed. +Note that parsing is string-typed: `from_str` cannot know whether `"0"` was a list index or a dict key, so all parsed tokens are strings. That's fine for applying patches, since [`iapply`][patchdiff.apply.iapply] converts numeric-looking keys back to list indices as needed. ## Resolving a pointer -[`evaluate`][patchdiff.pointer.Pointer.evaluate] resolves a pointer against an object and returns `(parent, key, value)` โ€” the container holding the addressed leaf, the leaf's key in it, and its current value: +[`evaluate`][patchdiff.pointer.Pointer.evaluate] resolves a pointer against an object and returns `(parent, key, value)`: the container holding the addressed leaf, the leaf's key in it, and its current value: ```python from patchdiff.pointer import Pointer @@ -56,8 +56,8 @@ assert key == 1 assert value == 20 ``` -The walk to the parent is strict โ€” a missing intermediate raises. Only the *leaf* may be missing (with a container parent), because that's a legitimate target for an `"add"`; its value then resolves to `None`. +The walk to the parent is strict, so a missing intermediate raises. Only the *leaf* may be missing (with a container parent), because that's a legitimate target for an `"add"`. Its value then resolves to `None`. ## Divergence from RFC 6901 -Strict JSON pointers only contain string tokens. Patchdiff pointers can hold **arbitrary hashable values**: integer list indices stay integers, and set members are addressed by the member value itself (see [Diffing sets](diffing.md#sets)). Rendering to a string (or [`to_json`][patchdiff.serialize.to_json]) stringifies each token, which is lossy for non-string tokens โ€” see [Gotchas](gotchas.md). +Strict JSON pointers only contain string tokens. Patchdiff pointers can hold **arbitrary hashable values**: integer list indices stay integers, and set members are addressed by the member value itself (see [diffing sets](diffing.md#sets)). Rendering to a string (or [`to_json`][patchdiff.serialize.to_json]) stringifies each token, which is lossy for non-string tokens. See [gotchas](gotchas.md). diff --git a/docs/guide/produce.md b/docs/guide/produce.md index 292e7c3..a4e9ad4 100644 --- a/docs/guide/produce.md +++ b/docs/guide/produce.md @@ -1,6 +1,6 @@ # Proxy-Based Patch Generation -Diffing compares two complete states after the fact. When *your code* is the thing making the changes, [`produce`][patchdiff.produce.produce] skips the comparison entirely: it hands your recipe a proxy-wrapped **draft**, records every mutation as it happens, and emits the patches directly. The idea (and the name) come from [Immer](https://immerjs.github.io/immer/produce). +Diffing compares two complete states after the fact. When your own code is the thing making the changes, [`produce`][patchdiff.produce.produce] can skip the comparison entirely: it hands your recipe a proxy-wrapped **draft**, records every mutation as it happens, and emits the patches directly. The idea (and the name) come from [Immer](https://immerjs.github.io/immer/produce). ```python from patchdiff import produce @@ -17,7 +17,7 @@ assert base == {"count": 0, "items": [1, 2, 3]} # base is unchanged assert result == {"count": 5, "items": [1, 2, 3, 4]} ``` -The recorded patches are exactly what [`diff`][patchdiff.diff.diff] would have produced โ€” the same operation dicts, appliable with [`apply`][patchdiff.apply.apply]/[`iapply`][patchdiff.apply.iapply] and serializable with [`to_json`][patchdiff.serialize.to_json]: +The recorded patches are exactly what [`diff`][patchdiff.diff.diff] would have produced: the same operation dicts, which can be applied with [`apply`][patchdiff.apply.apply]/[`iapply`][patchdiff.apply.iapply] and serialized with [`to_json`][patchdiff.serialize.to_json]: ```python from patchdiff import apply, produce @@ -29,7 +29,7 @@ assert apply(base, patches) == result assert apply(result, reverse_patches) == base ``` -For small mutations to large states this is much faster than diffing, because the cost scales with the number of *mutations* instead of the *size* of the state โ€” see the `produce-vs-diff` groups in the benchmark suite. +For small mutations to large states this is much faster than diffing, because the cost scales with the number of mutations instead of the size of the state. The `produce-vs-diff` groups in the benchmark suite quantify this. ## Immutable by default @@ -37,7 +37,7 @@ By default the recipe operates on a **copy**: `base` stays untouched and `result ## In-place mutation -With `in_place=True` the draft *is* the base object โ€” no copy is made, and mutations go straight through the proxy into it. That's the mode to use when the object's identity matters, most notably for [observ](observ.md) reactive state, where the writes must land on the reactive proxy so watchers fire: +With `in_place=True` the draft *is* the base object. No copy is made, and mutations go straight through the proxy into it. Use this when the object's identity matters, most notably for [observ](observ.md) reactive state, where the writes must land on the reactive proxy so watchers fire: ```python from patchdiff import produce @@ -54,17 +54,17 @@ assert result is state # same object assert state == {"count": 5} ``` -You still get both patch directions, so in-place mutation with undo/redo costs no deep copy at all. +You still get both patch directions, so in-place mutation with undo/redo doesn't cost a deep copy at all. ## What the draft supports -The draft mirrors the wrapped container type โ€” dicts, lists and sets each get a dedicated proxy with the full mutating and reading API, including operators: +The draft mirrors the wrapped container type. Dicts, lists and sets each get a dedicated proxy with the full mutating and reading API, including operators: -* **dicts**: item access/assignment/deletion, `get`, `pop`, `setdefault`, `update`, `clear`, `popitem`, `keys`/`values`/`items`, `|=`, iteration, โ€ฆ -* **lists**: indexing and slicing (read and write), `append`, `insert`, `extend`, `pop`, `remove`, `clear`, `reverse`, `sort`, `+=`, `*=`, iteration, โ€ฆ -* **sets**: `add`, `remove`, `discard`, `pop`, `clear`, `update`, the in-place operators (`|=`, `&=`, `-=`, `^=`) and their method forms, โ€ฆ +* dicts: item access/assignment/deletion, `get`, `pop`, `setdefault`, `update`, `clear`, `popitem`, `keys`/`values`/`items`, `|=`, iteration, ... +* lists: indexing and slicing (read and write), `append`, `insert`, `extend`, `pop`, `remove`, `clear`, `reverse`, `sort`, `+=`, `*=`, iteration, ... +* sets: `add`, `remove`, `discard`, `pop`, `clear`, `update`, the in-place operators (`|=`, `&=`, `-=`, `^=`) and their method forms, ... -Values you read from the draft are themselves wrapped, so nested mutations are tracked too, with correct deep paths โ€” even when list indices shift under them: +Values you read from the draft are themselves wrapped, so nested mutations are tracked too, with correct deep paths, even when list indices shift under them: ```python from patchdiff import produce, to_json @@ -87,4 +87,4 @@ assert '"path": "/todos/1/done"' in to_json(patches) Values recorded into patches are **snapshotted** (deep-copied, with proxies unwrapped) at the moment the mutation happens. Mutating an object after assigning it into the draft won't retroactively change earlier patches, and patches never share mutable state with the draft or the result. !!! note "The draft is only valid inside the recipe" - When `produce` returns, all proxies are released. Don't stash the draft (or values read from it) for use outside the recipe โ€” take what you need from `result` instead. + When `produce` returns, all proxies are released. Don't hold on to the draft (or values read from it) for use outside the recipe; take what you need from `result` instead. diff --git a/docs/guide/serialization.md b/docs/guide/serialization.md index b716904..4313c19 100644 --- a/docs/guide/serialization.md +++ b/docs/guide/serialization.md @@ -1,6 +1,6 @@ # Serialization -Operation lists are plain data โ€” dicts with `"op"`, `"path"` and `"value"` keys โ€” except for the paths, which are [`Pointer`][patchdiff.pointer.Pointer] objects. [`to_json`][patchdiff.serialize.to_json] renders the paths to their JSON pointer string form and serializes the whole list to an RFC 6902 JSON patch document: +Operation lists are plain data (dicts with `"op"`, `"path"` and `"value"` keys) except for the paths, which are [`Pointer`][patchdiff.pointer.Pointer] objects. [`to_json`][patchdiff.serialize.to_json] renders the paths to their JSON pointer string form and serializes the whole list to an RFC 6902 JSON patch document: ```python from patchdiff import diff, to_json @@ -33,7 +33,7 @@ print(to_json(ops, indent=4)) ] ``` -If you only want the paths stringified but not the JSON encoding โ€” for example to hand patches to another JSON patch library, or a different serializer โ€” use [`to_str_paths`][patchdiff.serialize.to_str_paths]: +If you only want the paths stringified but not the JSON encoding, for example to hand patches to another JSON patch library or a different serializer, use [`to_str_paths`][patchdiff.serialize.to_str_paths]: ```python from patchdiff import diff @@ -47,7 +47,7 @@ assert ops[0]["path"] != "/a" # the original ops are not mutated ## Non-JSON values -Serialization is only lossless for JSON-representable structures. Two things to watch for: +Serialization is only lossless for JSON-representable structures. Two things to watch out for: * **Values**: patches on structures containing sets, frozensets or tuples carry those objects in their `"value"` fields, and `json.dumps` can't encode them. Pass a `default=` to convert them (accepting that the type is lost), or keep such patches in memory instead. * **Paths**: non-string pointer tokens (integer list indices, set members) are stringified. For lists that's exactly RFC 6901; for sets it means the *member itself* becomes a string token, which cannot be parsed back into the original value. @@ -60,4 +60,4 @@ ops, _ = diff({"tags": {"a"}}, {"tags": {"a", "b"}}) assert to_json(ops) == '[{"op": "add", "path": "/tags/-", "value": "b"}]' ``` -See [Gotchas](gotchas.md) for the full list of RFC 6902 divergences. +See [gotchas](gotchas.md) for the full list of RFC 6902 divergences. diff --git a/docs/index.md b/docs/index.md index ec4238a..468b431 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,8 +1,8 @@ # Patchdiff ๐Ÿ” -Patchdiff computes **bidirectional, JSON-patch-compliant diffs** between composite Python data structures built out of dicts, lists, sets and tuples. Inspired by [rfc6902](https://github.com/chbrown/rfc6902), it has no dependencies and works on any Python >= 3.9. +Based on [rfc6902](https://github.com/chbrown/rfc6902) this library provides a simple API to generate **bi-directional** diffs between composite Python data structures composed out of lists, sets, tuples and dicts. The diffs are JSON-patch compliant, and can optionally be serialized to JSON format. Patchdiff has no dependencies and works on Python 3.9 and up. -One call gives you both directions: +A single call to [`diff`][patchdiff.diff.diff] gives you the patches in **both directions**: ```python from patchdiff import apply, diff @@ -16,13 +16,9 @@ assert apply(input, ops) == output assert apply(output, reverse_ops) == input ``` -That makes patchdiff a natural fit for: +Having both directions makes it easy to build undo/redo, to synchronize state between processes (send patches over the wire instead of whole documents), or to keep a log of exactly what changed. -* **Undo/redo** โ€” apply `reverse_ops` to go back, `ops` to go forward again. -* **Synchronization** โ€” serialize patches with [`to_json`][patchdiff.serialize.to_json] and ship them over the wire instead of whole documents. -* **Change tracking** โ€” record exactly what a piece of code did to your state. - -Besides after-the-fact diffing, patchdiff can also record patches **while mutations happen**, through [`produce`][patchdiff.produce.produce] โ€” a proxy-based recorder inspired by [Immer](https://immerjs.github.io/immer/produce): +As an alternative to diffing, patchdiff can also record patches while mutations are being made, using a proxy mechanism like [Immer](https://immerjs.github.io/immer/produce). See [`produce`][patchdiff.produce.produce]: ```python from patchdiff import produce @@ -39,14 +35,15 @@ assert base == {"count": 0, "items": [1, 2, 3]} # base is untouched assert result == {"count": 5, "items": [1, 2, 3, 4]} ``` -## Where to go next +## Where to start -* Follow the [Quick Start](getting-started/quick-start.md) to learn the core API in a few minutes. -* Read the [Guide](guide/diffing.md) for an in-depth look at [diffing](guide/diffing.md), [applying patches](guide/applying.md), [pointers](guide/pointers.md), [proxy-based patch generation](guide/produce.md) and [serialization](guide/serialization.md). -* Using [observ](https://github.com/fork-tongue/observ)? See the [Observ Integration](guide/observ.md) page for reactive state with undo/redo. -* Browse the [API Reference](reference/api.md) for the complete public API. +* The [quick start](getting-started/quick-start.md) walks through the core API. +* The guide covers [diffing](guide/diffing.md), [applying patches](guide/applying.md), [pointers](guide/pointers.md), [produce](guide/produce.md), [serialization](guide/serialization.md) and [gotchas](guide/gotchas.md) in more detail. +* If you use [observ](https://github.com/fork-tongue/observ), have a look at [observ integration](guide/observ.md). +* The complete public API is documented in the [API reference](reference/api.md). +* The [internals](internals/architecture.md) page describes how everything works under the hood. ## Related projects -* [observ](https://github.com/fork-tongue/observ): reactive state management for Python; patchdiff's `produce(..., in_place=True)` is built to work with its reactive proxies. -* [rfc6902](https://github.com/chbrown/rfc6902): the TypeScript library that inspired patchdiff's diffing approach. +* [observ](https://github.com/fork-tongue/observ): reactive state management for Python. Patchdiff's `produce(..., in_place=True)` is designed to work with its reactive proxies. +* [rfc6902](https://github.com/chbrown/rfc6902): the TypeScript library that patchdiff's diffing approach is based on. diff --git a/docs/internals/architecture.md b/docs/internals/architecture.md index c0aaac3..bfbf814 100644 --- a/docs/internals/architecture.md +++ b/docs/internals/architecture.md @@ -1,60 +1,60 @@ # Architecture -This page documents how patchdiff works under the hood. Nothing here is part of the public API โ€” it exists so that contributors (and the curious) can find their way around the four core modules: `pointer.py`, `diff.py`, `apply.py` and `produce.py`. +This page documents how patchdiff works under the hood. Nothing here is part of the public API; it exists to help contributors find their way around the four core modules: `pointer.py`, `diff.py`, `apply.py` and `produce.py`. ## Pointers -A [`Pointer`][patchdiff.pointer.Pointer] is a tuple of reference tokens behind `__slots__`. Immutability is what makes sharing safe: `append` builds a *new* pointer from `(*tokens, token)`, so the diff recursion can hand the same prefix pointer to many child operations without copies or aliasing bugs. Tokens are kept in their native Python types โ€” integers for list indices, arbitrary hashable values for set members โ€” and only stringified (with RFC 6901 `~0`/`~1` escaping) when a pointer is rendered. +A [`Pointer`][patchdiff.pointer.Pointer] is a tuple of reference tokens behind `__slots__`. Immutability is what makes sharing safe: `append` builds a *new* pointer from `(*tokens, token)`, so the diff recursion can hand the same prefix pointer to many child operations without copies or aliasing bugs. Tokens are kept in their native Python types (integers for list indices, arbitrary hashable values for set members) and only stringified, with RFC 6901 `~0`/`~1` escaping, when a pointer is rendered. `Pointer.evaluate` resolves a path in two phases with deliberately different strictness: -* the walk **to the parent** is strict โ€” a missing intermediate container raises, because silently landing on a partial parent would let `iapply` write to the wrong place; -* the **leaf** lookup tolerates `KeyError`/`IndexError`/`TypeError` and resolves to `None`, but only when the parent is a container that can be written into โ€” a missing leaf is a legitimate target for an `"add"` (dict inserts, the list `-` append token). +* the walk to the parent is strict: a missing intermediate container raises, because silently landing on a partial parent would let `iapply` write to the wrong place; +* the leaf lookup tolerates `KeyError`/`IndexError`/`TypeError` and resolves to `None`, but only when the parent is a container that can be written into. A missing leaf is a legitimate target for an `"add"` (dict inserts, the list `-` append token). ## Diffing -`diff()` dispatches on duck type: both sides having `.append` means list, `.keys` means dict, `.add` means set โ€” which is what lets observ proxies and other container look-alikes flow through unchanged. Everything else (scalars, tuples, frozensets, mismatched container kinds) is one `replace` op. The first check is always `input == output`; equal inputs short-circuit to empty patch lists. +`diff()` dispatches on duck type: both sides having `.append` means list, `.keys` means dict, `.add` means set. This is what lets observ proxies and other container look-alikes flow through unchanged. Everything else (scalars, tuples, frozensets, mismatched container kinds) becomes one `replace` op. The first check is always `input == output`; equal inputs short-circuit to empty patch lists. ### Lists: Myers edit script with prefix/suffix trimming `diff_lists` computes a minimal edit script in four steps: -1. **Trim.** The common prefix and suffix are stripped first โ€” for the common case of a localized edit in a large list, this collapses the problem to a few elements before any real work happens. -2. **Myers' greedy search.** Over the trimmed region, `_myers_script` runs Myers' O((m+n)ยทD) algorithm (*An O(ND) Difference Algorithm and Its Variations*, 1986): it explores diagonals of the edit graph, following "snakes" of equal elements for free, until it finds a shortest path of D insertions/deletions. Cost scales with the number of actual differences, not the product of the list sizes โ€” nearly-equal lists are cheap regardless of length. Memory is O(Dยฒ) for the backtrack trace. The search also carries a git-style "too expensive" cutoff: once D exceeds half the combined length (the lists share less than a quarter of their elements), it gives up on minimality and emits the whole region as one hunk of element-wise replaces โ€” exactly what the old O(mยทn) DP produced for such inputs, at O(m+n) cost. Small regions are always solved exactly. -3. **Hunks and replace pairing.** Myers scripts contain only insertions and deletions. Consecutive edits with no kept element in between are grouped into hunks, and within each hunk the k-th deletion is paired with the k-th insertion as a `replace` โ€” restoring the replace semantics the DP produced. When a replace pairs two containers, `diff` recurses into them with the element's pointer as the new prefix, so nested changes become deep paths instead of wholesale element replacement. Unpaired remainders stay plain removes/adds. +1. **Trim.** The common prefix and suffix are stripped first. For the common case of a localized edit in a large list, this collapses the problem to a few elements before any real work happens. +2. **Myers' greedy search.** Over the trimmed region, `_myers_script` runs Myers' O((m+n)ยทD) algorithm (*An O(ND) Difference Algorithm and Its Variations*, 1986). It explores diagonals of the edit graph, following "snakes" of equal elements for free, until it finds a shortest path of D insertions/deletions. Cost scales with the number of actual differences, not the product of the list sizes, so nearly-equal lists are cheap regardless of length. Memory is O(Dยฒ) for the backtrack trace. The search also carries a git-style "too expensive" cutoff: once D exceeds half the combined length (meaning the lists share less than a quarter of their elements), it gives up on minimality and emits the whole region as one hunk of element-wise replaces. That is exactly what the old O(mยทn) DP produced for such inputs, but at O(m+n) cost. Small regions are always solved exactly. +3. **Hunks and replace pairing.** Myers scripts contain only insertions and deletions. Consecutive edits with no kept element in between are grouped into hunks, and within each hunk the k-th deletion is paired with the k-th insertion as a `replace`, restoring the replace semantics the DP produced. When a replace pairs two containers, `diff` recurses into them with the element's pointer as the new prefix, so nested changes become deep paths instead of wholesale element replacement. Unpaired remainders stay plain removes/adds. 4. **Padding.** `_pad_ops` re-emits the operations in application order while tracking a running `padding` offset: every applied `add` shifts subsequent indices up by one, every `remove` shifts them down. Adds that land past the end of the list become the `-` (append) token. The reverse operations are built from the same hunks with input/output roles swapped, then padded against the output list. ### Dicts and sets -`diff_dicts` splits keys into three groups โ€” input-only (`remove`), output-only (`add`), and common (recurse). `diff_sets` is the same with elements instead of keys: removals address the element value itself as the final token; additions use the `-` token. In both cases the reverse lists are assembled so that applying them in order undoes the forward list applied in order. +`diff_dicts` splits keys into three groups: input-only (`remove`), output-only (`add`), and common (recurse). `diff_sets` is the same with elements instead of keys: removals address the element value itself as the final token, additions use the `-` token. In both cases the reverse lists are assembled so that applying them in order undoes the forward list applied in order. ## Applying -`iapply` is a straight interpreter: for each patch it resolves `path.evaluate(obj)` to `(parent, key, _)`, then dispatches on the parent's duck type. Dict writes are direct; list writes convert numeric string keys (from parsed pointers) back to `int` and translate `add` at `-` into `append`; set `add` inserts the value, set `remove` discards the *key* (the addressed element). Patch values are `deepcopy`'d before writing so the patch list and the patched object never share mutable state. `apply` is literally `iapply(deepcopy(obj), patches)`. +`iapply` is a straight interpreter: for each patch it resolves `path.evaluate(obj)` to `(parent, key, _)`, then dispatches on the parent's duck type. Dict writes are direct. List writes convert numeric string keys (from parsed pointers) back to `int` and translate `add` at `-` into `append`. Set `add` inserts the value, set `remove` discards the *key* (the addressed element). Patch values are deep-copied before writing so the patch list and the patched object never share mutable state. `apply` is literally `iapply(deepcopy(obj), patches)`. ## produce(): proxy-based recording -`produce()` wraps the draft in a proxy tree (`DictProxy` / `ListProxy` / `SetProxy`, dispatched by duck type) and lets the recipe mutate it. Three design decisions shape the implementation: +`produce()` wraps the draft in a proxy tree (`DictProxy` / `ListProxy` / `SetProxy`, dispatched by duck type) and lets the recipe mutate it. Three design decisions shape the implementation. ### Paths come from parent links, not stored strings -A proxy stores only its parent proxy and its key within that parent โ€” never an absolute path. Its location is computed on demand by `_location()`, walking up to the root and collecting keys. This is what keeps recorded paths correct when the tree changes under a handed-out reference: when `insert(0, โ€ฆ)` shifts list elements, the list proxy renumbers the keys of its children (`_shift_cache`), and a child that was at index 0 now reports index 1 โ€” no stored path to invalidate. +A proxy stores only its parent proxy and its key within that parent, never an absolute path. Its location is computed on demand by `_location()`, walking up to the root and collecting keys. This is what keeps recorded paths correct when the tree changes under a handed-out reference: when `insert(0, โ€ฆ)` shifts list elements, the list proxy renumbers the keys of its children (`_shift_cache`), and a child that was at index 0 now reports index 1. There is no stored path to invalidate. -The walk is **memoized**: each proxy caches its location (reusing the parent's cached location recursively), and every structural change โ€” a detach, a re-attach, a list index shift, `sort`/`reverse` re-keying, finalize โ€” bumps an epoch counter on the recorder that invalidates all cached locations at once. Repeated writes into a stable tree therefore pay O(1) per write instead of an O(depth) walk, while any change that could move a proxy falls back to recomputation on the next write. +The walk is **memoized**: each proxy caches its location (reusing the parent's cached location recursively), and every structural change (a detach, a re-attach, a list index shift, `sort`/`reverse` re-keying, finalize) bumps an epoch counter on the recorder that invalidates all cached locations at once. Repeated writes into a stable tree therefore pay O(1) per write instead of an O(depth) walk, while any change that could move a proxy falls back to recomputation on the next write. ### Detachment stops recording -Each proxy keeps a registry (`_proxies`) of the child proxies it has handed out. Removing or replacing an element marks the corresponding child proxy *detached*: it still mutates its underlying data (so user code holding it keeps working), but `_location()` returns `None` and nothing gets recorded โ€” its data will be captured by the snapshot of whatever write re-inserts it. Re-inserting a *detached* proxy directly is a move: `_adopt` re-attaches it at the new location, and later mutations through the held reference record at the new path. Adopting a still-attached proxy (or one wrapping duck-typed data) snapshots instead, since a value can only live in one place. +Each proxy keeps a registry (`_proxies`) of the child proxies it has handed out. Removing or replacing an element marks the corresponding child proxy *detached*: it still mutates its underlying data (so user code holding it keeps working), but `_location()` returns `None` and nothing gets recorded. Its data will be captured by the snapshot of whatever write re-inserts it. Re-inserting a *detached* proxy directly is a move: `_adopt` re-attaches it at the new location, and later mutations through the held reference record at the new path. Adopting a still-attached proxy (or one wrapping duck-typed data) snapshots instead, since a value can only live in one place. -Parent links are **weak references**, so the proxy tree contains no cycles and is reclaimed by plain reference counting the moment `produce` returns. `PatchRecorder.finalize` additionally detaches the root, which severs every remaining proxy's path to the root โ€” a proxy leaked out of the recipe can never append to the already-returned patch lists. +Parent links are **weak references**, so the proxy tree contains no cycles and is reclaimed by plain reference counting the moment `produce` returns. `PatchRecorder.finalize` additionally detaches the root, which severs every remaining proxy's path to the root: a proxy leaked out of the recipe can never append to the already-returned patch lists. ### Values are snapshotted at record time -Every non-scalar value that goes into a patch passes through `_snapshot`: a hand-rolled deep copy for the JSON-like types (dict, list, set, frozenset, tuple), which also replaces nested proxies with copies of their data; unknown types fall back to `copy.deepcopy`, which unwraps third-party proxies (like observ's) through their `__deepcopy__` hooks. Snapshotting at record time โ€” not at finalize โ€” is what makes patches immune to later mutations of the same object. +Every non-scalar value that goes into a patch passes through `_snapshot`: a hand-rolled deep copy for the JSON-like types (dict, list, set, frozenset, tuple), which also replaces nested proxies with copies of their data. Unknown types fall back to `copy.deepcopy`, which unwraps third-party proxies (like observ's) through their `__deepcopy__` hooks. Snapshotting at record time, not at finalize, is what makes patches immune to later mutations of the same object. -Forward patches are appended in order; reverse patches are also appended (O(1)) and reversed once in `finalize`, since reverse application order is the mirror of forward order. `record_replace` compares old and new first and skips no-op writes entirely. +Forward patches are appended in order. Reverse patches are also appended (O(1)) and reversed once in `finalize`, since reverse application order is the mirror of forward order. `record_replace` compares old and new first and skips no-op writes entirely. ### The proxy classes diff --git a/docs/reference/api.md b/docs/reference/api.md index 5ce937e..20f87be 100644 --- a/docs/reference/api.md +++ b/docs/reference/api.md @@ -1,6 +1,6 @@ # API Reference -The public API is small: two ways to obtain patches ([`diff`][patchdiff.diff.diff] and [`produce`][patchdiff.produce.produce]), two ways to apply them ([`apply`][patchdiff.apply.apply] and [`iapply`][patchdiff.apply.iapply]), and serialization helpers. All of these are importable directly from `patchdiff`; the [`Pointer`][patchdiff.pointer.Pointer] class lives in `patchdiff.pointer`. +The public API is small: two ways to obtain patches ([`diff`][patchdiff.diff.diff] and [`produce`][patchdiff.produce.produce]), two ways to apply them ([`apply`][patchdiff.apply.apply] and [`iapply`][patchdiff.apply.iapply]), and serialization helpers. All of these can be imported directly from `patchdiff`. The [`Pointer`][patchdiff.pointer.Pointer] class lives in `patchdiff.pointer`. ## Diffing diff --git a/mkdocs.yml b/mkdocs.yml index 938ee56..1682125 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -1,5 +1,5 @@ site_name: Patchdiff -site_description: Bidirectional, JSON-patch-compliant diffs between Python data structures +site_description: Bi-directional, JSON-patch compliant diffs between composite Python data structures site_url: https://fork-tongue.github.io/patchdiff repo_url: https://github.com/fork-tongue/patchdiff repo_name: fork-tongue/patchdiff @@ -59,7 +59,7 @@ nav: - Proxy-Based Patch Generation: guide/produce.md - Serialization: guide/serialization.md - Observ Integration: guide/observ.md - - Gotchas and Best Practices: guide/gotchas.md + - Gotchas: guide/gotchas.md - Reference: - API Reference: reference/api.md - Internals: