A running log of decisions made and limitations discovered while building mutare_ecto —
the plugin-side counterpart of core's ../mutare/NOTES.md. CLAUDE.md is the map and each
module's doc says what is; this file holds what's intentionally left for later and the
design history — what a thing used to be and why it changed — so moduledocs don't have to
read like logbooks. Referenced from code and docs as NOTES "title".
Query.drops/3 attributes a removed from clause to its value with
Mutation.at_drop(entry.value). The delivered mutant removes the complete clause, but
applying the recorded deletion range directly to source leaves its keyword key and separator
behind (where: condition becomes where: ). This predates origin-preserving collection;
hosted delivery now preserves the same deletion attribution as in-place delivery.
The origin regression suite proves source-patch parity for replacements. Clause drops need an attribution that covers the complete removable syntax, including the appropriate separator, while retaining the clause's selection line. Until that is implemented, their report ranges must not be treated as standalone executable source patches.
Inner from inside a pin interior: whole-call rewrites only, no condition swaps — RESOLVED (in core) [done]
The full-set island sub-contract closed the inner-dynamic gap, but the analogous inner
from was only half covered: a standalone query built inside a pin interior
(^repo.all(from p in P, where: p.x > 1, …)) got Query's whole-from rewrites through the
whole-call offer, while its where:/having: conditions' in-fragment swaps were produced
by nobody — at top level those are delivered by the selector host, and hosting was masked
inside the sub-contract (host/2 disabled in collect), so the :hosted-routed condition
stayed raw.
The planned closing mechanism here was plugin-side (the Mutare.Ecto.Subquery pattern —
recurse the catalogs into the inner from and rebuild). What actually closed it is more
general and lives in core: collect no longer masks host/2 — it runs a nested host's targets
through the same attachment the transform uses and lowers each target mutant to a
whole-call rebuild (splice(wrap(mutant)), exactly HostedEmit's composition with the
selector degenerated to its selected branch — by construction the value the weave takes when
that mutant's branch is active). Hosting is a delivery optimization, not a semantic category:
where the weave is unavailable, the same mutant ships as the rebuilt call. Hosted delivery
still never nests (no selector is built inside an island); hosted semantics are never lost.
Consequences: the inner from's condition swaps and literal arms now surface as
where: ^dynamic(...)-pinned rebuilds relayed through the outer weave, with zero plugin code
change — every host plugin gets this for free, per macro, with the producer's finalize/2
(the families: filter, the equivalence note) still running at generation. Proven by
subcontract_test.exs ("an inner from inside a pin"), the semantic fixture ("an inner from's
lowered condition mutant is live through the woven pin"), and core's
subcontract_full_set_test.exs (three delivery layers live at runtime).
Should the plugin mutate the second argument of apply_action/apply_action!? No — and it
should also stop core from doing so when core's families run alongside. The atom is
semantically inert where it matters: on a valid changeset it is never consulted
({:ok, apply_changes(changeset)}), on an invalid one it only stamps changeset.action —
metadata whose one real consumer, Phoenix's form-error display, gates on action != nil, not on
which action. A swapped atom is therefore killable only by asserting the label itself — a test
about a name rather than about behaviour. The discriminator against the superficially similar
:on_conflict swaps: those fork real persistence behaviour (raise/skip/overwrite) with precise
kill conditions; this forks nothing observable.
Note the direction this cuts, which is not "the atom does not matter": tests do occasionally
read changeset.action (and Ecto.InvalidChangesetError prints it), which is exactly why
RepoWrite's own rewrite has to reproduce whichever atom the real write would have stamped —
see the design-history entry "Persistence: the rewrite restates the write's Repo and action".
Mutating the atom is a cheap mutant we decline to offer; getting it wrong inside another
family's mutant is a spurious kill. Both say: reproduce it, never fork it.
So the surface is suppression shipped by the plugin, not mutation: argument_marks/1 on
Mutare.Ecto declares {Ecto.Changeset, :apply_action(!), 2, [1]} under core's shared
:structural label — promoted in core for this case (Mutare.Mutator.structural_label/0 +
pinned?/1; every :skip_arguments-honouring value family declines at the marked position; see
core NOTES "The :structural shared mark"). Rejected shapes: reusing :timeout (its readers are
value-aware — AtomLiteral holds back only :infinity, so the mislabel wouldn't even work);
forging the reserved __mutare_self__.* label (breaks under :as, abuses the namespace core
carved out); per-family user :skip_arguments config (works today but inverts ownership — every
app would re-encode plugin knowledge). Known limit: a mark pins the named node only, so
identifier-list positions (validate_required(cs, [:name]), cast/3's permitted list) are not
coverable this way — the interior atoms are offered unmarked. If those ever need pinning, core
would need descent-propagating marks. Proven in structural_marks_test.exs (held back with the
plugin enabled, mutating without it — plus the piped/imported spellings and a positional
sibling-atom control).
Mutare.Ecto.StaticCondition declines to weave a having whose condition carries a subquery,
because Ecto rejects one inside a dynamic having when the query is built. It decided by
reading source (Mutare.Ecto.Subquery.present?/1), so a subquery that appears only once an
author macro expands was invisible to it:
defmacro over_threshold(p), do: quote(do: count(unquote(p).id) > subquery(…))
having: over_threshold(p) and count(p.id) > 1
That condition was woven, and the function raised "subqueries are not allowed in having
expressions" on every call, baseline included; condition_delivery: :static was the documented
remedy. Round thirty-five found the same blind spot hiding a splice (below), and the rule now
reads the macro call itself: a condition holding a call Ecto can only be expanding is rebuilt,
whatever the expansion. :static remains, as a policy that rebuilds every condition.
(p in Post) |> from(where: p.x > 1) now exposes its source through core's Call.pipe_left.
FromCall reads that source, and Host.Bindings re-declares p in the hosted dynamic.
The source itself stays raw, so no core family treats the binding declaration as an expression.
Core's pipe-left proposal Part B now preserves the declaration in each whole-call mutant branch.
Dispatcher no longer withholds Query's rewrites or StaticCondition's fallbacks for a piped
in source. Order flips and clause drops build alongside hosted conditions and bounds, and the
runtime tests compare direct and piped query structures under every mutant, ignoring only their
diagnostic file/line locations. This requires the core fix planned for 0.3.1; test the unreleased
pair with a local path dependency until that release is available.
Still deferred: source binding reorders. A piped source is read-only:
FromCall.replace_source/2 accepts only a directly written source. Changing delivery does not
let the plugin edit the left operand; the Query.binding_reorders/1 guard remains.
These were initially omitted, sometimes described as equivalent under EXISTS. The audit's zero-limit and offset examples showed why that blanket claim fails. The catalog now composes bound tags into static inner-query rebuilds and enables ordering for windowed value queries. The remaining gaps are unwindowed scalar ordering and general clause drops; their observation rules need finer wrapper information than the current existence/value distinction.
The source audit exposed three decisions that had been coupled: filter classification with
dynamic argument layout, join-condition discovery with hostability, and window grammar with
the delivery path. Window grammar is now shared; dynamics accept general value bodies; every
explicit join predicate reaches the delivery decision. Repeated explicit on: predicates
rebuild statically, while a sole association on: can weave because association predicates
attach during planning, after dynamic expansion. EXISTS projection pruning now considers set
comparisons. Complete composed query stages use core's existing island lowering, closing the
terminal-stage gap without implementing another query-stage catalog.
The follow-up audit found one shared mistake in four places. A property of an expression by itself had been used where the expression's context decides the question:
- The coalesce drop keeps the SQL type, but not the syntactic kind that the parent requires.
Both walks now thread the call-argument slot (
Walk.child_slot/3), andScalarwithholds the drop in the two slots Ecto restricts. - A window's options inherited
is_nil's NULL-ness-only observation. Their values select the rows the function reads, so they are now always observed by value. - EXISTS projection pruning recognized only the set comparisons. It now prunes only when no projected value can change whether a row survives (a dedup followed by an offset, an alias read-back, or an opaque source all keep the projection).
- An EXISTS limit drop was judged by the current limit alone. It is now judged by the current limit and the limit the drop uncovers.
While checking these cases, we found that SQLite ignores DISTINCT inside EXISTS even when an
OFFSET follows it, so the DISTINCT case is live only on Postgres. The rule stays, because
standard SQL counts those rows.
The next audit found two more places where a shortcut dropped information:
- A keyword-filter pair value walked as a separate root lost the
==Ecto builds around it, so the coalesce guard never saw a comparison.Fragment.mutants/3now takes the root's slot from its caller. The same gap existed at clause roots: a coalesce drop to a bare pin at aselectororder_byroot, or as a window option entry, reads as fields. Clause values now start at the slot of their macro's expression argument, and window option entries atover/2's option argument. - "Projection mutation" does not imply "row-count preserving". A drop that removes the query's only aggregate turns one row into a row per input row. Under EXISTS, a projection mutant is now also kept when it may change the query's aggregation.
The follow-up review of that check found it compared aggregate counts, where the question
is whether the query aggregates. coalesce(min(a), max(b)) → min(a) holds fewer aggregates
and still aggregates. The check also stopped at over/1,2, which hid the ordinary sum in
over(sum(sum(v))). Only the windowed call aggregates the window; its operands and options
belong to the query. The review also proposed that Ecto discards a having: true. That holds
only for a runtime value: a literal is applied to the quoted expression at compile time and
stays in havings. It turned out not to matter. SQLite decides aggregation from the select
list alone and rejects HAVING on a query that does not aggregate, while Postgres counts
HAVING. So a group_by and projection aggregates are the only facts both engines agree on.
A further review found three gaps in the same check. It summed aggregates over every written
projection clause, but a later select_merge key replaces an earlier field. The check now
folds the clauses the way Ecto merges them, and judges the original and the mutant (the rebuilt
from, parsed again) on what survives. Its window traversal skipped every windowed call,
including a fragment or author macro that may hide an ordinary aggregate. And under EXISTS
no projection pin reached core at all, so a pinned dynamic that holds the only aggregate went
unmutated. Those pins now reach core, and all their mutants are kept. A per-mutant filter was
considered and rejected. Core attributes a mutant at the Elixir node it changed, and a control
flow change (negating the if that picks between two dynamics) is attributed at a node with no
aggregate, while it still switches which projection runs.
The next review found both of those fixes too narrow. The fold read a map update,
%{map(r, [:value]) | value: coalesce(0, sum(r.value))}, as an empty literal map: its
comprehension skipped the | member instead of rejecting it. So both sides counted no
aggregate, and the drop was pruned. The fold now expands a clause into the merges Ecto's
subquery planner applies (a map update's base, then its pairs; merge/2's left operand, then
its right), and a map with any member that is not a literal-keyed field contributes whole. The
pin gate asked whether the interior builds a dynamic, but a pin can pick between dynamics
built elsewhere, and a merge key or a map/2 field list decides which fields survive with no
dynamic at all. The gate now asks what the pin is to the projection, by a descent over Ecto's
select grammar (Mutare.Ecto.Subquery.projection_roots/1): only a query parameter is withheld
under EXISTS. That descent also changed two roles in value mode. A map key pin and a take's
field-list pin were read as :value by the condition walk, which has no map grammar. They are
now :structural, so Mutare.Ecto.Island holds the names they can evaluate to.
A broader audit then looked for the same kinds of error elsewhere in the subquery pruning, and
found three more. The alias check searched each clause for a written selected_as/1, so a
pinned group_by: ^[dynamic(selected_as(:bucket))] counted as reading no alias. An unknown is
now read as a possible read wherever a clause admits a dynamic, or holds a fragment, but
only if the projection may define an alias at all, so an ordinary pinned filter keeps its
pruning. Aggregation alone also turned out not to settle an aggregate swap under EXISTS: SQLite
hands a bare having column the value from the row min/max picked, so min → max can
flip having: r.y == 1. And the value-mode ordering gate read only the inner from's own
bounds, although a query source may bring the limit. The audit's fourth finding, that a float
overflow breaks the is_nil rules for +/-/*, was kept as an accepted limitation and
documented in Mutare.Ecto.Fragment.
A second round of that audit found three more. Two continue the same pattern. A grouping or
DISTINCT ON term can read the projection by position with no alias at all (group_by: 1 is
GROUP BY 1), and grouping/1 read group_by: nil as grouping, though Ecto List.wraps it into
no grouping. The third was outside the subquery pruning, and broke baselines. Ecto's runtime
filter path discards a condition that expands to true whatever its operator, so a woven
where: false, or_where: true returned no rows. A weave of ^true would keep the rows but not
the statement (WHERE ? for WHERE TRUE), so the condition is rebuilt instead, whenever it or a
catalog mutant of it is the literal true.
A third round found four more, three of them gaps in the round before's fixes: a keyword
having: [y: 1] names a bare column by key, a module attribute or macro in group_by may
expand to a position, and max(parent_as(:outer).x) inside a subquery aggregates the outer
query, not the subquery. The fourth is another difference between Ecto's two build paths: the
static builder types a condition :boolean and a dynamic types it :any, and coalesce
and arithmetic pass that type to their operands, so where: coalesce(^"true", false) binds
true statically and the string woven. Such a condition is now rebuilt as well.
A fourth round found filter(true) passing the literal-true check (Ecto's escape erases
filter/1) and an alias read hidden in an unregistered macro. The macro case exposed a wider
assumption: each check that concludes from absence (no alias read, no hidden aggregate, no bare
column, not true, not clause-typed) recognised only macros whose routes were registered, while
the walk reads an unregistered call as standard syntax. Ecto expands any call outside its own
query vocabulary, so those checks now ask Mutare.Ecto.Walk.opaque_call?/1, which reads that
vocabulary from the compiled Ecto.Query.API and Ecto.Query.WindowAPI. The walk's descent is
unchanged. A fifth round keyed that vocabulary by arity as well: Ecto dispatches on name and
arity before it expands, so an author's coalesce/1 or sum/2 is a macro while a same-arity
sum/1 never is. The same round found the name-keyed swaps (like → ilike) renaming an
author's like/1 into a function that does not exist, and a macro in the projection defining
the alias a fragment reads.
A sixth round keyed it by grammar too: map/2 is Ecto's only in a select and constant/1
only as a fragment argument, so elsewhere a same-named macro is expanded. It also found a
remote Mod.over/2 read as window grammar (Ecto expands every remote call), and a min →
max swap pruned beneath is_nil although, on SQLite, the aggregate picks the row a bare
column is read from; min/max lost their NULL rule.
A seventh round showed that the grammar must follow the descent: inside coalesce in a
projection, map/2 is an ordinary expression again. The aggregate count now threads the
grammar through its walk. It also found a binary literal's integer segment mutated to a
negative number, which Ecto's binary escape rejects; a binary literal is now a leaf.
Then a named window (windows: [w: [order_by: sum(r.value)]], used as over(row_number(), :w))
turned out to hide its aggregate from the count, which read :w as an atom; it now reads each
use from the definition, and a second use of the window keeps the drop of the first pruned. The
reproduction first showed the mutant already kept, for the wrong reason: the grammar split had
left over/2, which Ecto.Query.WindowAPI rather than Ecto.Query.API defines, out of the
expression vocabulary, so every window read as a possible macro. Also, over/1 took the generic
descent beneath is_nil, resetting its function's observation to by-value, where over/2's
window rule keeps it; both now share the rule.
Round nine found two more gaps of the same family. as/1 and parent_as/1 are Ecto's only
as a field's receiver, so standing alone they are author macros; the prewalk-based checks now
skip a receiver (Walk.contains_opaque_call?/2) instead of asking about it. And the aggregate
count marked a macro as possibly hiding an aggregate while still descending into its arguments
and counting a sum there as certain; it now stops at an opaque call, whose arguments the
expansion may discard.
Round ten found the window's function read in the wrong grammar. Ecto validates over's
function against Ecto.Query.WindowAPI before escaping it and expands any other call, so
there even an expression name (coalesce/2) is an author macro; the vocabulary had added
WindowAPI to the expression names instead of replacing them. The catalogs now neither mutate
nor enter such a function (Mutare.Ecto.Window), and the aggregate count reads a function it
keeps whole in the window's grammar. The same round found fragment/n accepted at any arity,
though every fragment head takes at least the query.
Round eleven found two more slots of the same kind: type/2's operand, which Ecto also
validates before escaping (so type(is_nil(x), :integer) expands an author's is_nil/1), and
a select take's field list, which Ecto Macro.expands to atoms and never escapes. Both rules
now live in Mutare.Ecto.Walk (expanded_argument?/2, and the field-list skip), which every
catalog's descent and contains_opaque_call?/2 read, rather than in one catalog each.
Round twelve added a fragment's template, which Ecto also expands at compile time, and found a
reader with its own traversal that had not been told (the having bare-column check, now
asking Walk.expands_argument?/1), and a positional grouping hidden by filter/1, which
Ecto's escape erases.
Round thirteen found a string sigil missing from the clause-typing literals (Ecto escapes
~s/~S/~w/~W as literals), and a shortcut in the EXISTS aggregation judgement that
assumed the syntax around a mutated node keeps its meaning: equal aggregate counts before and
after pruned the mutant even where an enclosing author macro could pattern-match on the
aggregate's name. The shortcut now stands only outside such a macro.
Round fourteen found over recognised by name and arity alone, where Ecto's head also needs a
call-shaped function: over(nil, x) and over/3 are macros (Walk.window?/1, which every
reader matching over now asks). And round thirteen's opaque-ancestor guard had used projection
grammar at every depth, so merge/2 inside a coalesce passed as the select builder's.
Round fifteen, asked to favour ordinary code, found the island seam's root role fixed at
:condition for every caller. A dynamic's whole-body pin is a parameter (Ecto's dynamic
builder escapes the body as an expression; only a nested DynamicExpr is expanded), so holding
its keyword keys dropped live mutants. Island.subcontracted/4 now takes the root role from its
caller.
Round sixteen found the EXISTS readers trusting a projection fragment in two ways. The
aggregate count marked it as possibly hiding an aggregate but still counted the aggregates in its
arguments as certain, though raw SQL can move one into a window (? OVER ()); it now stops at a
fragment as at an opaque call. And the alias check did not count a fragment's raw SQL as possibly
defining an alias (? AS n), which SQLite lets a where read.
Round seventeen found the ordering pruned wholesale where no window makes the order visible,
though an aggregate in ORDER BY decides aggregation (Postgres aggregates by it; SQLite rejects
it on an otherwise non-aggregating query). An ordering value mutant is now kept there when it may
change whether the query holds an aggregate (aggregating_ordering/2), in both modes.
Round eighteen found the in-list element drop reading identical syntax as one value, a proxy
for "evaluates to the same member": two ^next_id() calls need not. Grouping now requires an
element with no call Ecto does not own and no fragment.
Round nineteen found that fix incomplete: contains_opaque_call?/2 skipped every zero-argument
dot call as a field access, Counter.next() included. And round seventeen's ordering rule had
judged ordering mutants by aggregation alone, missing SQLite's bare-column row choice: a lone
min/max anywhere, ORDER BY included, picks the row an ungrouped, unaggregated column
reads. Where such a column is observed (the projection of a value query, a having), every
ordering value mutant is now kept (picks_bare_row?/2).
Round twenty found the in-list fix still a proxy: it read a pin's interior in Ecto's query
grammar, where a module held in a variable (provider.next()) looks like a field access. A
pin's interior is Elixir, and repeatability is now an allow-list of plain values there
(a variable, a literal, a module attribute, a map field or key, a collection of these).
Round twenty-one then struck the field and key reads from it (a module in the variable, a
struct's fetch/2), and found the having bare-column check reading an unread non-empty list
([{:name, "Carol"}], keyword pairs to Ecto) as naming no column.
Round twenty-two was cut short by codex's usage limit; its one demonstrated probe showed the same
spelling gap in named_windows/1, which read only keyword-syntax pairs, so an explicit-tuple
window definition's aggregate went uncounted.
Round twenty-three found three more readers with that gap (a DISTINCT ON pair, a window
option or sort pair, an order_by direction flip). Every pair reader now reads through one
helper, Mutare.Ecto.AST.unwrap_pair/1, and a flip keeps the spelling it found. One reader
still takes keyword syntax only: Mutare.Ecto.AST.KeywordList, so a from whose clause list is
written in explicit tuples (from(p in P, [{:where, …}])) gets only whole-call mutants. The gap
loses mutants without pruning any, and closing it means an Entry that remembers its spelling,
so that a re-key (put_key/3) writes an atom literal and not a keyword key into a tuple.
Round twenty-four found the in-list repeat check trusting Ecto's own vocabulary: ago/2 and
from_now/2 each build on a fresh ^DateTime.utc_now(), so two written occurrences are two
values, and each now drops alone, as a fragment does.
Round twenty-five found a dynamic's select map walked as a condition: its keys, which name
the result's fields or, in a map update, the struct's, were data to the literal arms, and a
pinned key was a :value island. A map key is now a structural position. Round twenty-seven
found that fix reading a map pair by its slot, which the pair's value inherited, so a 2-tuple
in a value (%{pair: {11, 22}}) lost its first element as a key; a value now has a slot of
its own.
Round twenty-eight found the repeat check's allow-list still admitting a bare pinned variable,
which may hold a DynamicExpr: Ecto expands such a parameter at each occurrence, rerunning its
pins and SQL. A variable inside a pinned collection stays plain, since such a parameter is data.
The cost is an equivalent drop where the variable holds plain data ([^x, ^x] → [^x]).
Round twenty-nine found a compound cast spec ({:array, ^inner}) skipped whole, so a pin
inside it never reached core; its pins are now islands, :structural, while its literals stay
unentered.
Round thirty found a weave that broke the baseline: Ecto's dynamic path appends a spliced
list's parameter at each of its placeholders, so a condition with splice(^values) is now
rebuilt. It also found two more unreached pins, in the receiver of as(^name).id and in a
window's {^direction, term} sort pair.
Round thirty-one found the bare-column reader behind the ordering retention skipping a window
whole, as an aggregate: a window runs over the aggregated rows, so over(sum(r.y)) reads r.y
bare. A window's function arguments are now read as bare, and its options conservatively.
Round thirty-two found filter/2 around the window's aggregate still hiding them.
Round thirty-three found a fragment as the window function hiding them too (it is now read
whole), and an EXISTS projection pin pruned as data where it is a spliced list, whose length
is an arity (such a projection is now observed).
Round thirty-four found the literal-true rule applied condition by condition: weaving an
earlier condition made the rest of its from runtime, where a later or_where: true was
discarded, and the baseline changed. See "A piped literal-true filter after an instrumented
static chain" for the fix and the piped spelling it leaves.
Round thirty-five found round thirty's splice rule reading only a written splice/1, so an
author macro expanding to one (member_sql(p.id, ^values)) was woven and broke the baseline
as before. A condition holding any call Ecto can only be expanding is now rebuilt, outside a
pin's interior and a subquery wrapper's argument, which are Elixir built apart from the
condition's parameters; a written splice there no longer forces the rebuild either. The same
rule closed "A macro that expands to a subquery in a having".
Round twenty-six found the binary-literal leaf rule held only by the condition catalog: the
value walk read a bitstring's specifiers (unsigned-integer-size(128)) as subtraction, and the
mutant failed the whole build. The rule now lives in Mutare.Ecto.Walk.structural/3, under
every catalog.
The fourth round also found that a keyword filter's scalar value, routed :interpolated, reaches Ecto
as a parameter, which the planner cast/1s before dump/1ing, where a written literal is only
dump/1ed. For a custom type whose cast/1 is not the identity on the literal (one that
downcases, say), the baseline binds a different value. Three options were weighed: leave such
a pair value unmutated, rebuild the whole call per mutant (a new core delivery), or document the
limitation. The first was chosen. It needs the column's type, so the source's schema module,
and a source is almost always written through an alias (from p in Post), while a classifier
sees its arguments before core resolves them. Core now hands the classifier the call site's
aliases (Mutare.CallRouting.Call.resolved_module/2), and a pair on a column of any type but
the primitives whose cast/1 keeps a dumpable literal stays raw. A source the plugin cannot
read (a variable, a composed query, an assoc/2 join) is still interpolated: leaving every such
pair unmutated would lose active: true → false in nearly every piped query.
The coalesce family's equivalence note ("differ only where x is NULL") still describes scalar evaluation only; an aggregate-removing drop is killed by an empty input instead.
The README used to say "both query syntaxes are covered", which was true of the families and
false of the capabilities: Mutare.Ecto.Surface declares what a name gets as a from key
(:from/:from_drop) and as a composable stage (:mutations/:stage_drop) independently, and
the two sides had drifted apart without any doc saying so. The README's "Coverage by spelling"
table now states each gap and Mutare.Ecto.SpellingCases pins it — comparing spellings by the
statements their mutants reach, since rendered diffs differ between spellings even where the
mutants agree. A gap is asserted as "this family reaches nothing here", so closing one fails
its test until the table is rewritten. None of the gaps below rests on an equivalence argument;
each is unimplemented.
clause_dropon thefromkeys still held back —join:,select:,windows:,update:(design history "Clause drop: thefromkeys nothing else needs"). Unlike a pipeline, afromshows every clause at once, so three of the four could be constrained rather than refused. Ajoin:is the hard one: the keyword list expands at compile time, so dropping a join whose variable another clause reads fails the single build rather than one mutant. It has to prove the variable unreferenced (every other clause value, every other join's source andon:), take the join's ownon:keys with it, and decide what a droppedas:name means for stages composed later. Aselect:could drop when the source is a schema rather than a table name, awindows:when noover/2in the samefromnames it. Anupdate:cannot be decided here: whetherupdate_allstill has something to set depends on its call site.- A computed
fromsource (from p in recent(2)). The source argument is aninpattern, and core's treatments are per argument: nothing routes "the right side of thisin":expressionwhile the left stays raw. A nested treatment is a core seam.
Ecto builds a query at compile time while every stage does (Ecto.Query.Builder.apply_query/4
expands the query argument to an escaped struct), and at runtime from the first stage that
does not. Ecto.Query.Builder.Filter.apply/3 discards a filter whose expression is the literal
true, but only at runtime: at compile time it is handed the filter as escaped AST, which its
%{expr: true} head does not match. So from(p in Post, where: p.views > 10) |> or_where(true)
returns every row natively, and any instrumentation of the upstream from makes the or_where
runtime: a weave, a whole-call selector case (even :filter_drop alone), so
condition_delivery: :static does not help. The baseline then returns only the rows with
more than 10 views.
Only a chain that is static natively is affected: one rooted at a written from(...), every
stage built at compile time. A pipe rooted at a schema (Post |> where(…) |> or_where(true)) or
a variable, or passing through a root-pin condition (where(^dynamic)), is runtime natively and
discards the filter already. The discard matters where the true is an or_ filter, or the
first filter with an or_ filter after it (where(true) |> or_where(x)); an and-ed true
is the identity either way.
The keyword spelling is handled: a condition followed by such a filter in its from is rebuilt
(Mutare.Ecto.StaticCondition.from_receiver/3), and every selector branch is then a static
from. The piped spelling is not, because each stage is offered to mutate/2 alone, and the
instrumented stage cannot see the filter downstream of it. The options weighed:
- deliver every mutant of such a chain as a rewrite of the whole chain, which per-call delivery has no seam for;
- skip instrumenting a static chain whose downstream stages hold such a filter, which needs the same downstream view;
- rewrite at the
truestage instead, needing no view of the chain: every branch of a literal-truefilter stage ([]and a call that may expand totrueincluded), the baseline too, writes a form the runtime path keeps (not false, ortype(^true, :boolean); not^true, a root pin, which Ecto builds and discards at runtime even in a static chain), but only where the stage's own query argument is static natively, since where it is not, Ecto discards thetrueand so must the baseline. That decision can be Ecto's own rather than a copy of its builders' rules: the rewrite emits a plugin macro over the original query argument (kept_true(upstream)), which Ecto's escape expands in the caller's environment, and which asksMacro.expand(upstream, __CALLER__)whether it is an escaped%Ecto.Query{}. The blocker is delivery: only a host target's:wraprewrites the original branch, and a target needs a mutant, but the stage's one mutant (true→false) is:boolean_literal's, which is opt-in. A default run would need a core seam for a rewrite-only target. Its costs besides: the baseline's SQL text changes (NOT (FALSE)forTRUE, loud in a test asserting SQL), and the metamutant calls a plugin module at compile time.
None is taken. A literal true beside an or_ filter makes that filter trivially true, so the
pattern is close to absent from written code. Where a test observes the query's rows, the
changed baseline fails core's baseline run, and the run aborts rather than scoring mutants.
Unobserved, it misscores only the upstream mutants of that one query (a mutant equivalent
natively, under the OR TRUE, may be killed). The third option is the one to take if the
pattern turns up.
Mutare.Ecto.ClauseDrop emits both kinds under one family (its moduledoc has the taxonomy, and
Mutare.Ecto.SpellingCases pins where each break surfaces — build, plan, or run). A break is
killed by any test that executes the query, so it is scored as a kill that says nothing about
the dropped stage's effect; it also costs a mutant run.
It cannot be constrained from where the mutant is made: mutate/2 is offered one stage, with
pipe_mode and nothing of the pipeline around it, and the stages that would reveal the
dependency may sit in another function altogether. Nor is a local proxy sound — a join naming
its binding or carrying as: is as likely to be a pure row filter as a provider. The options,
none taken:
- a sibling-aware seam in core — the pipeline's later stages offered alongside the stage, which would also let a repeated pipe bound be recognised (design history "Bound: only the effective occurrence of a repeated bound is mutated"). Decidable only within one written pipeline; a query finished elsewhere stays opaque. Consult before adding.
- classifying the kill rather than the mutant — core records
:killedwithout the reason, so "killed by a raisedEcto.QueryError" is not reportable today. That is the general form of the problem, not an Ecto one, and would be core's to add. - a separate family for the provider stages (
join,windows,with_cte), so a project could exclude them. Rejected for now: the filtering join is among the family's most useful mutants, and excluding by stage name discards it along with the breaks.
Mutare.Ecto.Island sub-contracts a pin's interior to core from the two places that own a
condition. Every other clause value is routed :raw, so limit: ^(page_size + 1),
order_by: ^[asc: dynamic([p], p.a + p.b)] and select: %{v: ^(min * 2)} have interiors nobody
mutates, while the same code bound to a variable first is mutated where it is built. The
delivery is not the obstacle: the interior is already behind a pin, so a selector case there
is ordinary Elixir and poisons nothing. What is missing is an owner for the position —
either the host taking these values as :hosted targets, one per pin (the root-pin and
pin-only bound weaves in Mutare.Ecto.Host.Target are the precedent), or a core treatment
meaning "raw, except beneath a ^". The second is a core seam; consult before adding.
Core's Call.pipe_left lets the classifier read a piped source. Host.Routing now applies one
shape table to the source of from and every composable query stage, in both spellings:
structural queryables and in declarations stay raw, computed values remain expressions.
This removes alias/string mutants from Post |> where(…) and restores upstream mutants in
q |> where(…) |> from(limit: 5) as well as from(q |> where(…), limit: 5).
Previously a piped from hid every upstream mutation by routing its source raw, whereas a
composable stage always treated its pipe-left as an expression and mutated schema/table names.
The direct from source was also always raw despite being visible. FromCall now carries the
actual piped source rather than a nil sentinel, so the host reads its bindings and whether it
is composed just as it does for a direct source. Core's read-only boundary still applies to edits.
What a thing used to be, what it is now, and why. The moduledocs state only the current shape; each entry here is the before they no longer carry.
:persistence swaps a Repo write for Ecto.Changeset.apply_action/2, on the promise that the
mutant differs from the real write only when the write would have succeeded — that is what
makes its kill condition ("a test drives a successful write and asserts a persistence
consequence") precise rather than incidental. The first rewrite emitted
apply_action(change(arg), <action>) and nothing else, which broke the promise twice on the
failure path, where the two are supposed to be indistinguishable:
Ecto.Repo.Schema.put_repo_and_action/4stamps the Repo on the error changeset;apply_action/2does not, so every persistence mutant returnedrepo: nil.- the
@writestable fixed:insertasinsert_or_update's action, reasoning that "the atom only colours an error changeset's:action". But Ecto chooses that action per call, from the changeset data's__meta__state — a changeset over a row loaded from the DB routes toupdate. So an invalid loaded changeset came backaction: :insertwhere the baseline said:update, andinsert_or_update!raised "could not perform insert" against the baseline's "update". Any test asserting the rejection killed the mutant without ever exercising persistence: a spurious kill, and the expensive kind — it reports the write as tested when nothing tested it.
Both are now restated by the rewrite: a Map.replace!(…, :repo, …) stage carrying the configured
repo:, and — for insert_or_update only — a Kernel.then/2 binding whose fn reads
Ecto.get_meta(changeset.data, :state) and picks the same action Ecto would (binding rather than
re-reading the argument, which must not be evaluated twice). The two written forms come from one
stages/3 table folded either into nested calls or into a pipe, so the piped and unpiped mutants
cannot drift apart the way a hand-written pair can. Deliberately still unreproduced:
changeset.repo_opts (the real value can carry a live process stacktrace) and Ecto's argument
guards on insert_or_update — see Mutare.Ecto.RepoWrite's moduledoc. Proven in
repo_write_test.exs and, against both engines, by the semantic suite's "the mutant's failed
write is byte-for-byte the baseline's".
repo: accepted exactly one module, and the documented way to cover a second repo was to list the
plugin twice with repo:/as: — which also split the report into two families and doubled every
# mutare:ignore[…] label, though a survivor's file:line already says which repo it hit. The
sibling mutare_swoosh plugin took a list for its mailer: from the start, so the two gave a
third plugin author two rules to copy. Now both take one module or a list, normalised at init/1
into Mutare.Calls.module_key/1s, and :as is reserved for what it does everywhere in Mutare:
naming the report family. The one place the single-repo assumption had leaked into a mutant — the
:persistence rewrite's Map.replace!(…, :repo, …) stamp, which read the configured repo — now
takes the repo the call resolved to, which Mutare.Ecto.RepoCall.resolve/2 returns alongside the
call for exactly that purpose (repo_write_test.exs, "with several repos configured").
Producers used to return a mutation as one of three tuples — {family, node},
{family, node, label}, {family, node, label, attribution} — and every consumer that bridged
two producers (a walker rebuilding a clause, a host composing catalogs) had to pattern-match all
three. %Mutare.Ecto.Tag{family, node, label: nil, attribution: nil} replaced them: one shape
with nil defaults, Tag.map_node/2 as the shared "rebuild the surrounding form around each
mutant" step, and Tag.to_mutation/1 as the one normalizer both delivery paths return through.
Adding a finer label to a producer no longer touches delivery code.
Fragment's mutation walk and its island walk were two hand-rolled copies of one traversal, and
the select/order_by expression walker a third — three descents that could disagree about
which nodes a condition exposes (an island the catalog would never have walked past, or the
reverse). Mutare.Ecto.Walk.positions/3 is now the single traversal; every catalog is a per-node
reader over its positions and never descends on its own (a catalog's children/2 rule can only
narrow what the walk admits). So Fragment.mutants/2 and Fragment.islands/2 agree by
construction, not by two walks kept in step; fragment_descent_test.exs pins the policy.
The original dispatch was flat: every AST node was offered to all eight sub-mutators, each of
which re-ran macro and call resolution — even for ordinary literals and operators that no
producer could ever claim. Mutare.Ecto.Dispatcher classifies a node once (a registered query
macro by its Surface kind, otherwise the resolved call's module — Ecto.Query,
Ecto.Changeset, or the configured repo) and invokes only the producers that can apply.
The ±1 bump of a literal limit/offset first lived beside the other clause mutations —
Mutare.Ecto.Clause rebuilt the standalone/pipe call and Mutare.Ecto.Query the whole from —
so each bump duplicated the entire call/query per mutant, exactly the cost the selector host
exists to avoid. It is now hosted, pin-only: limit: ^(case …) with no dynamic/2 wrap and
no bindings, because a bound is an integer parameter and the pinned selector is plain Ecto
interpolation with a behaviourally identical baseline (Mutare.Ecto.Bound, Host.Target).
Bound.literal?/1 is defined as bumps/1 being non-empty, so the routing classifier and the
host agree on "literal bound" by definition. Only the bound drop remains a whole-from/stage
rewrite.
When a top-level-pin condition (where: ^cond) was first hosted in every form, it rode the
ordinary delivery: every branch dynamic(bindings, ^interior). The interiors that change had
in view were a runtime boolean and logic choosing which dynamic to splice, and its check was
that every woven metamutant compiles. A DynamicExpr does survive the wrap (dynamic/2
splices it), and that was the one arm ever observed at runtime (the semantic suite's
inner-dynamic fixture); the later keyword-filter fixtures again asserted compilation only. But
Ecto dispatches a root interpolation on its value, and inside dynamic/2 the same pin is a
parameter: ^[score: 5] stopped being p.score == ^5 and became the parameter [score: 5]
(an Ecto.QueryError when run), and ^true became a bound true instead of no condition. Since
the wrap also encloses the original branch, the instrumented baseline broke, not just the
mutants; the trigger was any core-mutable Elixir in a root pin whose value is not a
DynamicExpr. A root pin is now woven pin-only over its bare interior, by the branch wrap
alone — which also fixed the lowered rebuild of an inner query's root pin, since core lowers
through the same wrap (Mutare.Ecto.Host.Target; root_pin_delivery_test.exs holds the
baseline-parity check across every arm of Ecto's dispatch and every hosted position).
Host.Target first told a root pin by matching the written ^ itself — a second reading of a value
Mutare.Ecto.Host.Condition.shape/1 had already classified, as a predicate. The kind is now part of
that one classification ({:predicate, :root_pin} beside {:predicate, :expression}), carried by
locate/1's struct and the host's keyword-value paths into every condition target, so the same
decision says whether the host owns a value and how it weaves it.
order_by used to be droppable like any other clause. But an ORDER BY-less query's row order
is SQL-unspecified, so whether the drop was "killed" depended on the engine happening to return
rows in a different order — the mutant's survival tracked engine nondeterminism, not the test
suite. The drop was removed (order_by/prepend_order_by carry no stage_drop/from_drop in
Surface) and replaced by re-tagging a bare, implicitly-ascending ordering term (:name,
u.name) to an explicit desc — a :ordering mutant with a deterministic, behaviourally
distinct baseline, sound because bare-means-ascending is Ecto's own guarantee, engine-independent
(Mutare.Ecto.Ordering). It asks the question the drop was pretending to ask.
A nulls-qualified sort key used to flip both axes at once (:asc_nulls_first →
:desc_nulls_last). That was a weaker mutant: any order-pinning test killed it, so a missing
NULL-placement assertion never surfaced. Each axis now flips on its own — direction under
:ordering (keeping the placement), placement under :ordering_nulls (keeping the direction,
only for an explicitly qualified key, and equivalence-sensitive because it needs NULL rows to
kill). A bare :asc yields one mutant; an :asc_nulls_first yields two.
The join family once offered widening swaps — inner_join/join → left_join, * →
full_join — that introduced a join kind the author never wrote. They were mostly equivalent
(a join written to exclude unmatched rows rarely has an orphan for the widened query to expose)
and each needed a dialect gate for the introduced kind. The family now only narrows
(left_join → inner_join, full_join → left_join/right_join) plus the sideways
left_join ↔ right_join; every flip permutes a form already reachable from the source, so the
remaining gates (RIGHT JOIN under :postgres/:mysql) are purely about the target kind's
portability (Mutare.Ecto.Query).
The join-kind flips used to be two maps inside Mutare.Ecto.Query, keyed by from clause key
(left_join: [:inner_join]), and the only thing that read them was the whole-from key swap —
so join(q, :left, …) kept its qualifier, a gap nobody had decided on (it surfaced when the
spellings were first compared by the statements they reach). The flips are keyed by
qualifier now, the one name both spellings write, in a shared Mutare.Ecto.JoinType
catalog beside Mutare.Ecto.Combination: Query converts a join key to its qualifier and the
targets back, Mutare.Ecto.Clause swaps the qualifier argument in a whole-call rewrite, and
the policy, the dialects: gate and the # mutare:ignore labels are the same by construction
rather than by parallel maintenance. Only a literal qualifier is swapped: Ecto accepts a
computed one (validated at runtime, Ecto.Query.Builder.Join.qual!/1), which is a value
mutated where it is bound.
Routing kind, stage-drop family, whole-from drop family, hosted/binding/join capabilities, and
standalone mutation capabilities were once kept in parallel lists across the modules that
consumed them; adding a query builder meant finding and updating each. Mutare.Ecto.Surface is
the one descriptor table (with compile-time invariant checks per descriptor), every consumer
derives its view from it, and macro_kind_parity_test.exs pins that each dispatch on
macro_kinds/0 takes a real branch for every kind.
The clause drop — "is this filter/window/grouping tested at all?", the primary motivating
mutation for a query builder — was originally only reachable in the from-keyword syntax, as
Mutare.Ecto.Query's whole-from clause drop. The composable pipe/standalone forms
(q |> where(…)) are far more common in practice, so Mutare.Ecto.ClauseDrop added the stage
drop over the shared Mutare.Ecto.StageDrop delivery, recording the same family as Query
for the same semantic mutation regardless of which syntax wrote it.
After Mutare.Ecto.ClauseDrop gave every composable stage a drop, the from form was left
with the two it started with — where/having (:filter_drop) and limit/offset
(:bound) — so q |> group_by(…) could lose its grouping and from(…, group_by: …) could
not. No decision lay behind that; it surfaced when the spellings were compared by the
statements their mutants reach. Mutare.Ecto.Surface now declares from_drop: :clause_drop on
group_by/distinct/preload/lock/select_merge/with_ties and the six set operations,
through Mutare.Ecto.Query's existing drops/3 (so a repeated lock: drops only its effective
occurrence, like a repeated bound).
The line is drawn by what a from is: one keyword list, expanded as a whole when the
metamutant compiles, where a pipeline is assembled at runtime. A stage drop that breaks a
dependency raises under its own mutant; the same drop in a from can fail the build for
every mutant in the file. So a key drops only if the rest of the list cannot need it — it
binds no variable, nothing names it, no plan requires it — which holds back join: (binds a
variable), select: (a schemaless source requires one), update: (update_all does) and
windows: (named by over/2). Surface enforces that a from_drop is the same name's
stage_drop family, and surface_test.exs pins the held-back set, so a clause macro added
later has to be placed on one side of the line. What is left can still fail at the engine
(a projection that needs its group_by on Postgres), under that one mutant.
is_nil's argument used to be a hard boundary for every family: the walk never entered it, on
the claim that the value families preserve NULL-ness, so that any mutant there is equivalent. A
first correction read the coalesce drop — the one mutation named as NULL-ness-changing —
beneath it through a narrowed walk of the unit's own (is_nil(coalesce(u.name, u.role)) →
is_nil(u.name) is live for a nullable default), and kept the boundary for everything else,
islands included.
The claim itself was too strong, for the catalog's own families and for everything around them:
*→/ turns a product NULL on a zero divisor (SQLite, MySQL; Postgres raises); and↔or
is three-valued; a literal is data wherever a form's NULL-ness depends on a value — the
comparand of fragment("NULLIF(?, ?)", p.score, 0), a JSON path key, a divisor; and a pin's
interior is Elixir that can compute nil by any route (^(opts[:min] || default)), which the
boundary kept from core altogether. So the boundary is gone. The argument is ordinary descent
under a narrower observation (the walk's context carries :nullness beneath is_nil), and a
mutant is pruned only when one table of per-form NULL rules (Mutare.Ecto.Fragment's
nullness/1: literals, +/-/*, coalesce, the four aggregates) shows it NULL on exactly
the original's rows; the same table decides how far down the narrow observation reaches.
Everything outside the table is unknown and emitted — the error now costs an equivalent mutant
in code nobody writes (is_nil(p.a > 1)), where it used to cost live ones. Pinned in
fragment_test.exs/fragment_descent_test.exs/subcontract_test.exs, and live on both
engines in the semantic suite (the NULLIF literal, the connective, the pin).
Mutare.Ecto.Fragment's walk used to refuse every 2-tuple as a leaf, on the claim that the only
tuple a condition contains is a compound cast spec (type(x, {:array, :string})) — whose
literals are structural at every depth, beyond the reach of the structural-position registry
(which names only a form's direct argument). The claim missed Ecto's tuple comparison,
{p.views, p.id} > {1, 2} (SQL's row-value comparison, supported since Ecto 3.0): the swap
fired on the >, but the literals inside the tuple were never mutated and a pinned element was
never an island — while the {:{}, …} form of a wider tuple, which the refusal did not match,
was entered, with its elements at a {:{}, n, i} position no registry entry names. The walk
now tells the two roles apart by position: at a structural position either tuple form is a cast
spec and stays a leaf; anywhere else it is a value tuple, transparent like a written list (its
elements inherit the comparison's position, so they are data). Pinned in fragment_test.exs,
fragment_descent_test.exs, and live against both engines in the semantic suite.
The in-list element drop used to be one mutant per index of a written list, deleting that
position: p.id in [1, 1, 2] offered [1, 2] twice and [1, 1] once. SQL IN is set
membership, so a duplicated member made the per-index drop dead by construction — deleting one
1 leaves the other, the set {1, 2} is unchanged, and no test can tell the mutant from the
original (twice over). The drop is now one mutant per distinct written element, removing every
occurrence (Mutare.Ecto.Fragment's element_drops/1): [1, 1, 2] shrinks to [2] and
[1, 1], each a different set. Elements are compared as written expressions with their source
metadata stripped, so ^a/^a and u.x/u.x are one member exactly as 1/1 are; a
duplicate-free list is unchanged. The literal bumps inside a duplicated list are still per
node (bumping one 1 of [1, 1, 2] to 2 gives [2, 1, 2], the original set): rewriting every
occurrence would be a multi-point rebuild the walk does not offer.
Mutare.Ecto.Host.Routing used to decide whether a composable macro's first argument was the
threaded query by shape — a bare variable, a nested pipe, or a call whose name was a
registered query builder (Surface.query_builder?/1) routed :expression; anything else
:skip. The shape test stood in for the one fact that actually places the query — whether the
call is piped — and it was wrong for every legal computed queryable: where(base_query(2), …),
where(if(…), …), where(Ecto.Query.exclude(q, :order_by), …) all routed :skip, so core never
descended them and base_query(2)'s 2 → 3/1/0 silently vanished (while the same call piped,
base_query(2) |> where(…), kept them through from_visible's default). The classifier now
routes by core's pipe_mode: piped, no visible argument is the query; direct, the first one is,
and it routes :expression whatever its shape. The only shape still read there is the structural
queryable (a schema alias, a table-name string, a {"table", Schema} pair), which stays raw
because a table/schema swap is a broken query, not a mutant. Surface.query_builder?/1 went with
the heuristic.
Piped from: the hidden source
Superseded by "Source routing agrees across direct and piped queries" above; the account below
records the original implementation before core exposed Call.pipe_left.
Post |> from(as: :post, where: as(:post).views > 5, limit: 5) used to receive no mutations
at all — no comparison, no literal, no filter drop, no bound — while the identical
from(Post, as: :post, …) received eight. Mutare.Ecto.AST.FromCall took the source from the
argument list, so a piped from's one visible argument (the clause list) landed in the source
slot; Mutare.Ecto.Host.Routing then stamped it :skip, and the host and Mutare.Ecto.Query
each parsed a clause-less from and produced nothing. FromCall now places the clauses by the
call's pipe_mode (Mutare.Ecto.AST.QueryCall carries core's stamp): piped, the source is
nil — hidden, on the pipe's left — and every edit rebuilds the from(…) half at its written
arity, so the routing classifier, the host splice, and the whole-from rewrites all read the same
shape and the two spellings yield the same mutants. The hidden source itself routes :raw
(route_arguments/2's piped: override — the plugin never sees its shape, and from's source
is never routed in the direct form either), which also stops core's :alias family from swapping
a structural Post |> for a nonexistent module, as from_visible's :expression default had let
it. Inside a subquery, the inline-from finders (Mutare.Ecto.Subquery) still recognise only
the direct spelling — a subquery(Post |> from(…)) interior stays out of reach.
Mutare.Ecto.Aggregate's per-node catalog used to match {f, meta, args} when f in @agg_funcs and is_list(args) — any call wearing the name, at any arity. The is_list half was only ever a
crash guard (a bare variable's third slot is its hygiene context, not arguments); nothing checked
that the call was Ecto's. Every rung of the ladder is /1, so the pattern claimed calls Ecto does
not define, and a rename across the ladder emitted a call nobody defines: an author's
sum(a, b) — a query DSL is free to define one, and Kernel.min/2/max/2 already are two —
became avg(a, b), which Ecto's builder rejects while expanding the query
(** (Ecto.Query.CompileError) avg(p.views, p.likes) is not a valid query expression). That is a
failed build, not a wasted mutant: the metamutant embeds every mutant in one compilation unit,
so a single mis-swap takes the whole run down.
Mutare.Ecto.Walk's author-macro rule did not cover this. It governs descent into a registered
macro's arguments — select: clamp(sum(p.x), 10) never has its sum swapped — but the macro call
node is itself a walk position, offered to the catalog by its parent, so a macro named sum was
mutated however it was routed. The catalog now guards both halves of "is this Ecto's aggregate":
the arity ([_arg], the same binary//2 guard Mutare.Ecto.Scalar already carried for
arithmetic and coalesce), and ownership — Ecto's aggregates are plain Ecto.Query.API
functions, never routed macros, so a resolve-pass macro stamp (Mutare.Calls.macro_treatment/1)
proves the call belongs to somebody else's grammar. The name-collision fixture is the plugin's own
Mutare.Ecto.AuthorMacros; core's shipped RoutingExtension cannot stand in for it, because its
macros are named nothing the plugin mutates.
Both arms of the :bound family used to fire on every limit:/offset: clause of a from.
But Ecto applies a from's pairs in written order, and limit/offset (and lock) replace
their predecessor — "if limit is given twice, it overrides the previous value" — where every
other key accumulates. So for limit: 5, limit: 10 the built query only ever held the 10, and
the three mutants on the 5 (both bumps and its drop) were equivalent by construction: no test
could tell them from the original. Mutare.Ecto.Surface now declares the last-wins keys
(last_wins?/1), Mutare.Ecto.AST.FromCall.effective_clause?/2 turns that into "the clause at
this index reaches the query", and both producers consult it — Mutare.Ecto.Query before a drop,
Mutare.Ecto.Host before weaving a bump. The final occurrence keeps all three mutants; its drop is
live because it uncovers the previous one. The rule follows the same reasoning as the IN-list
element dedupe above: a mutation that provably leaves the built query unchanged is not offered.
Only the from keyword form is in view — a bound repeated across a pipe
(q |> limit(5) |> limit(10)) or across functions composes at runtime, where no single node sees
both occurrences, so those stay mutated (suppressing the pipe form would need a sibling-aware
seam in core).
Mutare.Ecto.Host.Bindings.from/2 used to build a from's binding list out of the joins that
declare a variable — it matched join: c in Source and passed over everything else. But Ecto
binds a join written without x in (cross_join: "audit", join: subquery(q),
left_join: assoc(p, :x), a fragment, a ^source) anonymously, and still advances the binding
count. So for from p in "posts", cross_join: "audit", join: c in "comments", … — p &0, the
audit join &1, c &2 — the host wove dynamic([p, c], …) and named &1 c.
That was worse than a bad mutant. Every branch of a hosted selector resolves through the
re-declared list, the original condition's included, so the miscount rewrote the baseline;
and a misplaced binding raises only if the table it lands on lacks the column the condition reads.
Where the two tables share that column name the SQL was valid over the wrong table — the
instrumented query ran and returned the wrong rows, invisible to assert_compiles.
Each join now yields exactly one slot (join_slots/1): its variable, or the _ placeholder
(Mutare.Ecto.Binding.placeholder/0). A ... could not have repaired it, because the miscount
runs in both directions: under a literal source slots count from the front, so a dropped slot
displaces the joins after it; under a composed source the ... anchor counts from the tail, so
it displaces the joins before it ([p, ..., c] named a trailing audit join c; the list is
[p, ..., c, _]). For the same reason a trailing _ is kept rather than trimmed. host_test.exs
checks positions against Ecto itself — the metamutant's baseline must build the query the untouched
source builds, over every named/unnamed pattern of one to three joins and both source kinds — and
the semantic suite runs the placements against comments/audit, two tables seeded with the same
columns so that a wrong slot returns different rows rather than an error.
The standalone join/4,5 never miscounted — it re-declares the author's own written list and
tail-anchors the one join it adds — but only because it declined an unnamed join altogether:
Bindings.join/1 matched x in Source, wove nothing for a bare Source, and the on: of
join(q, :inner, [p], "audit", on: …) kept only its stage drop. It now takes the join's slot from
the same join_slot/1, so that on: is hosted behind [p, ..., _]. The placeholder is
load-bearing there too: the on: is resolved with the new join in place, so after an
author-written [..., x] the list must be [..., x, _] — [..., x] alone would read the new
join as x.
Hosting unnamed joins exposed the same blind spot in Mutare.Ecto.Host.JoinOn: its assoc
exclusion matched x in assoc(p, :rel) only, so a bare assoc(p, :rel) join slipped past it (in
the from form its on: was already being hosted). An assoc join is now told by its source,
named or not.
Whether a condition-position value is a predicate or a keyword filter used to be decided in
several places that did not agree — or not decided at all. The from-clause classifier asked
for a non-empty keyword list (KeywordList.nonempty/1); the binding-less argument form refused
any list; the binding form (where(q, [p], …)) asked nothing, treating whatever followed a
binding list as a predicate; and the host asked nothing either — from_targets/2, the join
target and Mutare.Ecto.Subquery applied the predicate catalog to every value whose key
admits a condition. Mutare.Ecto.Fragment's doc promised [] for a keyword shorthand, but its
walk enters a written list and reads a score: 5 pair as a value tuple, data on both sides.
Two shapes fell through. where(q, [p], score: 5) routed :hosted. And in
from(p in "posts", where: [score: 5], limit: 10) the classifier routed the pairs to core
correctly, but the hosted limit: made core offer the whole call to the host (core does not
confine returned targets to :hosted positions), which then claimed the sibling filter as well.
Either way the list was wrapped as dynamic([p], score: 5) — Ecto's general expression builder,
which refuses a filter's pairs — so the metamutant failed to compile; the host's splice also
overwrote the pins core had placed inside the list, leaving core's recorded mutants with no
selector branch; and with the opt-in atom arm on, the catalog renamed the column
([mutare: 5]). The same held for a join's on: shorthand, and for a pinned pair value
(where: [score: ^(min + 1)]), whose interior — nobody's, by pair_treatment/1 — was
sub-contracted to core as soon as a sibling was hosted. The existing mixed-shorthand test used
[active: true] under the default families, where the boolean arm is off and the catalog finds
nothing, so it never exercised any of this.
Mutare.Ecto.Host.Condition.shape/1 is now the one classification — Ecto's own rule: a written list
literal is the filter builder's, anything else the host's (an expression, or a root pin Ecto
dispatches on its runtime value — "Root pin: from dynamic-wrapped to pin-only") — read by the
classifier, by both argument forms of locate/1, by the host's keyword-value paths, by the subquery
recursion, and by the host's whole-call fallback (Mutare.Ecto.StaticCondition), which would
otherwise take a subquery-bearing filter (having: [score: subquery(…)]) for a declined predicate
and catalog its keys. The subquery case differs in one respect, kept deliberately: an interior
mutant is delivered as the inner from rebuilt, so a filter there stays in a filter position and
its pair values are still the plugin's to mutate (the whole outer condition is hosted; core never
reaches them) — only the keys stopped being catalog roots. Pinned in shorthand_test.exs —
including the compositional regression (a hosted sibling changes neither the diffs recorded in a
shorthand nor its rendering in the metamutant) and the reachability invariant (every recorded mutant
id has a selector branch) — and live against the DB in the semantic suite.
The island sub-contract was built on one sentence — "a ^ pin's interior is ordinary Elixir,
analyzed exactly like top-level Elixir" — which is right about how an interior is analyzed and
silent about what its value is for. Fragment.islands/1 was driven by meeting a ^ and threw
the walk's position away, so the structural-position registry, which keeps the literal arms off
the written field(p, :score), said nothing once the same atom sat behind a pin: ^:score went
to core as unconstrained data and came back ^:mutare. So did a pinned interval unit
(ago(^n, ^"day") → ^"mutare", which Ecto rejects when the query is built), a pinned cast
type, a binding or select-alias name, and a fragment's identifier(^name). A test comment
recorded the belief behind it: "a pin can only sit at a data position". Ecto accepts a pin at
nearly every position the registry lists.
One guard did carry positional knowledge across the boundary — the pin-side keyword-key rule — but as a blanket: every island's keyword keys were held, wherever the pin sat, which is why its doc had to list a known cost (an option list inside a parameter pin was "protected" too).
islands/2 now reports each pin's role, read off the position the walk already threads
(:structural where the registry says so, :value elsewhere; a pin that is the root takes the
role its caller states — :condition for a predicate, and Mutare.Ecto.Subquery's for a filter
pair's value and a pinned projection), and Mutare.Ecto.Island maps role to what is held. The
keyword-key rule became the :condition row, so its known cost no longer reaches a :value
pin. Two policies were weighed for the :structural row:
- hold every atom/string literal anywhere in the interior — the keyword-key rule's own
over-approximation. Rejected: the idiomatic ways to compute a column name put data
literals beside it (
Keyword.get(opts, :sort, :inserted_at),params["sort"]), and:sort→:mutare— "is the sort option honoured?" — is a live, valid-query mutant this would silently lose; - hold a literal only where it is known to reach the slot: one the interior can evaluate
to, read through the forms whose value is one of their own sub-expressions. Chosen — it is
Fragment'sis_nilstance (prune what is known, emit the rest), and its error is the cheap one: a name that reaches the slot through a call keeps its sentinel swap, a broken-query mutant any test on that path kills, where the first policy's error is a live mutant never offered.
Three roles, not one per kind of name: every structural position takes the same policy, so "field name" versus "interval unit" would be a distinction no code reads. The registry's comments still say which name each position holds.
The host used to find a condition's binding declaration by searching the arguments for the
first list it could parse (BindingList.find/1), and BindingList.parse/1 doubled as the
reorder's eligibility test: a non-empty list of var / name: var / .... Ecto's
escape_bind/1 reads two more entry forms — an interpolated name ([{^name, p}]) and an
explicit index ([{p, 0}, {c, 2}]) — and a search can only find what it already understands.
So a declaration in either form was invisible: Host.Condition fell through to its binding-less
shape and the weave was dynamic([], p.score > 10), an unbound p that failed the single build.
"A declaration I cannot read" had silently become "no declaration". The from twin did not
degrade at all — Bindings.declarations/1 handed the unread list to clean_var/1 and host/2
crashed with a ContractError.
Three separations fixed it. Grammar vs. reorderability: Mutare.Ecto.Binding.parse/1 is the
entry grammar, mirroring escape_bind/1 clause for clause and in its order ({p, c} is an
indexed positional to Ecto before it is ever a named one); BindingList.parse/1 admits any list
of such entries, [] included; only transpositions/1 asks what is reorderable (plain
positionals — an indexed entry carries its own position, so transposing it is a no-op).
Position vs. shape: both condition macros end (…, binding \\ [], expr), so the call's
effective arity says whether a declaration was written and where; Host.Condition.locate/3
reads that slot and reports it written, omitted, or uninterpretable, and
Host.Bindings answers {:ok, declarations} | :error so the third can never be rendered as
the second. (find/1 survives for BindingReorder alone, where skipping an unread list costs
that reorder its mutants and nothing else.) Entries vs. nodes: placement used to tell a
named declaration from a positional one by re-reading the rendered node's shape
(named?({_, _})), which an indexed {p, 0} would have satisfied; it now works on parsed
entries and renders once.
Two forms are read narrower than Ecto reads them, because the woven dynamic/2 re-declares
the list beside the original, evaluating whatever an entry computes a second time: an index must
be a literal, and an interpolated name a variable or module attribute. {^next_name(), p} would
run next_name/0 twice in the unmutated branch. Such a condition is never woven behind a
dynamic/2. It was first left unmutated in-fragment, a whole-call delivery judged too rare a
need to carry a second delivery path for; but Mutare.Ecto.StaticCondition had meanwhile become
that path (for a subquery in a having), so the condition is now rebuilt whole-call there under
the written list, as Mutare.Ecto.Dynamic rebuilds a free-standing dynamic. To keep each
condition delivered exactly once, the host and the fallback now enumerate the same conditions
(Host.Condition.from_indices/1, locate_on/1) and split them by one decision,
StaticCondition.delivery/4 — which is also what brought a standalone join's on: into the
fallback. A condition that is itself a ^ pin stays woven under any declaration: its weave is
pin-only and re-declares nothing (Host.Target's root-pin rule), and it keeps reporting its
pin-interior mutants at the condition rather than at the whole call. The later grammar/delivery audit extended this fallback to repeated explicit join predicates.
The same rewrite surfaced a placement bug that corrupted the baseline, not just the mutants:
the contiguity rule counted a literal source's joins from the number of declared entries, where
a literal source is always exactly one binding, so [p, q] in Post (or [{p, 0}, {q, 0}]) put
the first join at position 2 instead of 1. The anchor rule is now exact — joins stay contiguous
only behind exactly one positional entry. (The rewrite ran into the unnamed-join miscount too,
fixed on its own in "Bindings: an unnamed join still holds its slot" above; entry-based
placement keeps that section's _ slot as a parsed positional entry, and a join's named left
side outside the grammar reads as an uninterpretable declaration rather than failing the run.)