feat: MQTT 5.0 outbound (broker-assigned) Topic Alias (#840) - #1117
BenjaminDobler wants to merge 12 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## mqttv5 #1117 +/- ##
========================================
Coverage 99.93% 99.93%
========================================
Files 16 16
Lines 3094 3275 +181
========================================
+ Hits 3092 3273 +181
Misses 2 2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
robertsLando
left a comment
There was a problem hiding this comment.
deep-review — feat: MQTT 5.0 outbound (broker-assigned) Topic Alias
Verdict: Ship with minor changes. The approach is sound and spec-correct on the wire (verified full-topic+alias and empty-topic+alias round-trip through mqtt-packet 9.0.2; the aliased frame parses back with topic === ""). Aliases are assigned 1..max, contiguous and never evicted, so size + 1 is always the correct next value; the per-connection Map is allocated only when the client opts in (spec-compliant Server MAY, §3.3.2.3.4).
Strength worth calling out: aedes-packet's Packet copies properties by reference into every fanout subscriber packet (this.properties = original.properties), so mutating it in place would corrupt other subscribers. withOutboundTopicAlias correctly returns a shallow clone with a fresh properties object instead — the subtle correctness lynchpin, handled right, and the inline comment explains why.
Major (out-of-diff, so noted here) — reconcile the Paho EXPECTED_GAPS entry
tools/mqtt-compat/run_compat.py:52 still annotates test_server_topic_alias as "broker-assigned (outbound) topic aliases are not implemented… (#840)". This PR implements #840, so that note is now false. Per the compat-harness convention (tools/mqtt-compat/README.md §Expected gaps), when a gap starts passing the harness emits an UNEXPECTED_PASS/🎉 warning and the entry must be removed. The diff doesn't touch run_compat.py, and the PR description itself says it "flips test_server_topic_alias from an EXPECTED_GAP toward a pass" — so this is a known, in-scope loose end.
Fix: run the MQTT-compat job — if the test now passes, delete the entry; if it still fails for a different reason, rewrite the note (don't leave "not implemented"). It only warns rather than fails CI, so it rots silently otherwise.
Line-anchored findings
1 Minor (connect.js), 1 Minor (test/mqtt5.js), 1 FYI (write.js) — see inline threads.
Coverage: correctness, MQTT-5 spec conformance (§3.1.2.11.5, §3.3.2.3.4, §3.3.2-8/11), security (memory amplification), performance (hot-path guard, clone cost), tests, docs. Clean on DRY, naming, doc accuracy.
…est + cap follow-up - Add a reconnect test pinning the §3.3.2-11 invariant: outbound Topic Alias mappings must not survive a Network Connection. A fresh Client gets a fresh alias map, so the first PUBLISH after reconnect on a previously-aliased topic carries the full topic + a newly-assigned alias, never a bare alias. - Reference the tracked follow-up (moscajs#1124) for a broker-side outbound alias cap in the connect.js comment, per the review thread (deferred to keep this PR focused). Refs moscajs#840 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
a306795 to
7f43786
Compare
|
Addressed the review, and rebased onto the current
Full suite green, lint + tsd clean. |
📊 MQTT Compatibility ReportAedes tested against the Eclipse Paho interoperability suite (the vendor-neutral broker conformance suite used by Mosquitto, EMQX, mochi-mqtt, …).
MQTT 5.0 — 17/26 passed (3 expected gaps)
MQTT 3.1.1 — 10/10 passed
|
robertsLando
left a comment
There was a problem hiding this comment.
deep-review: MQTT 5.0 outbound Topic Alias
Scope: 4 files, +146 / −2. Base mqttv5, head 7f43786. Sensitive: lib/write.js is the universal serialize path for every outbound packet.
Verdict: Needs work — one crash path introduced on client-controlled input, plus the compat gap list that this PR's own premise invalidates. Both fixes are small.
Top 3 risks
- A duplicated
Topic Alias Maximumproperty in CONNECT now kills the connection on first delivery (connect.js:311). EXPECTED_GAPSstill claims this feature is unimplemented — and the compat run proves otherwise.- The alias table is unbounded and sized by the client; measured 19.1 MB per connection for 10k topics.
Strengths
- The clone-not-mutate design is right, and I verified it empirically: persisted QoS packets keep their real topic across reconnect.
- Non-v5 clients are guarded twice (
version !== 5inwrite.js, and_outboundTopicAliasMaximum = 0inconnect.js). - The vendor-neutral Paho suite validates the feature in both directions, and the v5 score moves 16/26 → 17/26.
- All three prior review threads are genuinely addressed (#1124 filed and open; reset-on-reconnect test added).
Major — tools/mqtt-compat/run_compat.py:52-54 (outside the diff)
test_server_topic_alias is still listed as an expected gap — "broker-assigned (outbound) topic aliases are not implemented" — and this PR is what implements it.
Not theoretical. The compat artifact for this exact head (run 29258181729) records "unexpected_pass": true for that test, so the ::warning …update EXPECTED_GAPS:: annotation fired. The job still concludes success because the step only prints.
Cross-run comparison, so the delta is attributable:
| run | branch | v5 score | test_server_topic_alias |
|---|---|---|---|
| 29818164110 | mqttv5 (base) |
16/26 | fail [gap] |
| 28871723205 | unrelated branch | 16/26 | fail [gap] |
| 29258181729 | this PR | 17/26 | pass — UNEXPECTED PASS |
The other six failures are byte-identical across all three, so nothing regressed.
Fix: delete lines 52-54 in this PR. The v5 dict then holds exactly test_shared_subscriptions, test_flow_control1, test_subscribe_identifiers.
Workflow findings (outside the diff)
- [Minor]
.github/workflows/mqtt-compat.yml:103-119— the unexpected-pass guard is advisory.::warningcannot fail a job, which is precisely how a stale gap list reached a green PR.raise SystemExit(1)insideif xpass:makes the marker do its job. - [Minor]
.github/workflows/mqtt-compat-comment.yml:55-61—download-artifacthard-fails when no artifact exists, which is reachable since the producer uploads withif-no-files-found: ignore. The laterfs.existsSync('compat-result/report.md')guard is unreachable in that case. Addcontinue-on-error: trueto the download step. - [Minor]
.github/workflows/mqtt-compat-comment.yml:53—checkout@v7with defaultpersist-credentials: truein the job holdingpull-requests: write;ci.yml:24,41already sets it false. Hardening only — nothing untrusted executes after that checkout. - [Minor]
.github/workflows/ci.yml:23,28,39,45,66—checkout@v4,setup-node@v4,codecov-action@v5,dependency-review-action@v4are 1-3 majors behind and run on the deprecated Node 20 action runtime, whilemqtt-compat.ymlis already oncheckout@v7.ci.ymlalso has noconcurrency:block. Pre-existing, not this PR — noted for the branch. - [Nit]
tools/mqtt-compat/README.md:86— says the suite exercises "inbound topic aliases"; with this PR it exercises broker-assigned outbound aliases too.
The workflow_run pattern in mqtt-compat-comment.yml was audited specifically for the classic escalation bugs and is clean: the PR number comes from trusted workflow_run metadata cross-checked against head_sha, the artifact body reaches github-script via env: rather than [object Object] interpolation, and the $GITHUB_OUTPUT heredoc uses a random delimiter. The only residual is content spoofing — a fork controls report.md verbatim, so it can make the bot post arbitrary markdown under the repo's identity.
FYI
- Duplicated-property handling is a general gap, not specific to this PR:
receiveMaximumandmaximumPacketSizetake the same array shape today (connect.js:304-305). Rejecting duplicates with 0x82 at parse time would close the whole class, and would also fix the Blocker. - v5 compat failures unchanged by this PR:
test_maximum_packet_size,test_offline_message_queueing,test_publication_expiry,test_server_keep_alive,test_will_delay(timeout),test_will_message. None are inEXPECTED_GAPS— the list tracks a curated subset, and the workflow warns on unexpected passes but never on unlisted failures. lib/client.js:39allocates the inbound_topicAliasesMap unconditionally even whenbroker.topicAliasMaximumis0(the default), so every client pays for a Map it can never use.connect.js:312shows the better lazy pattern. Out of scope here.
Coverage
Ran: correctness/concurrency, compat-workflow/CI, tests + DRY + design, prior-thread reconciliation.
Clean on: DRY (inbound alias→topic bounded by the broker option, outbound topic→alias bounded by the client's advertised max — genuinely different structures, nothing to extract; a shared helper would be worse), types (_-prefixed internals are deliberately undeclared, consistent with _wireVersion/_requestProblemInformation; no broker option ⇒ types/instance.d.ts correctly unchanged), packet-mutation safety, QoS retransmission, init-runs-once lifecycle (a second CONNECT is rejected at handlers/index.js:26-34; takeover installs a new Client), t.plan counts, test isolation.
Not run: the clause-by-clause MQTT 5.0 conformance matrix — that agent died on a session limit. Substituted direct verification against 7f43786: aliases stay in 1..max and never reach 0; absent or 0 topicAliasMaximum produces no alias; v3/v4 subscribers never receive one; mappings reset per Network Connection; retained, will and QoS 1 deliveries all alias correctly; topic: '' + topicAlias serializes correctly in mqtt-packet 9.0.2 — each probed live, plus test_server_topic_alias passing in the Paho suite. No per-statement matrix was produced, and no [MQTT-x.x.x-x] numbers are cited that couldn't be confirmed verbatim.
Local verification: test/mqtt5.js 73/73 pass at 7f43786; all 17 PR checks green.
| version = (protocolVersion === 3 || protocolVersion === 5) ? protocolVersion : 4 | ||
| } | ||
| const result = mqtt.writeToStream(packet, client.conn, WRITE_OPTS[version]) | ||
| const toWrite = withOutboundTopicAlias(client, packet, version) |
There was a problem hiding this comment.
[Minor] · Design / altitude
Per-feature v5 PUBLISH logic on the universal serialize path.
write() has 13 call sites (pubrec.js:29,34; client.js:143,360,555; connect.js:500,567; ping.js:9; publish.js:84,93,99,138; unsubscribe.js:98; subscribe.js:272), and exactly two ever carry a publish: client.js:143 (deliver0) and client.js:555 (writeQoS). This file's own header comment calls it "the broker's universal (v3/v4 included) hot path" and goes out of its way to avoid a per-write allocation — then this adds a call + property load to every PINGRESP, PUBACK and SUBACK.
deliver0/writeQoS is already the "shape the outbound PUBLISH for this client" seam — no-local, dedupe, qos downgrade and authorizeForward all live there (client.js:123-176).
Why: correctness is fine either way; this is placement. It also entangles the commit-after-success fix above with the generic write path — at deliver0/writeQoS the write callback is already in hand, so committing after success is natural there.
There was a problem hiding this comment.
Still open as of this review: withOutboundTopicAlias is called unconditionally from write(), which is the universal v3/v4/v5 path, with the version check inside the callee rather than at the call site.
There was a problem hiding this comment.
Still open as of the round-7 review at 0d2f83d — withOutboundTopicAlias is still called from the universal write() path rather than behind a v5 branch.
When a client advertises a Topic Alias Maximum > 0 in its CONNECT (§3.1.2.11.5), the broker now assigns aliases on outbound PUBLISH to save bandwidth (§3.3.2.3.4): the first PUBLISH on a topic carries the full topic name + a Topic Alias, and subsequent PUBLISHes on that topic carry an empty topic + the alias. Per connection, bounded by the client's advertised maximum; server MAY, so it is off unless the client opts in. - write.js: `withOutboundTopicAlias` rewrites a v5 PUBLISH just before serialize. It returns a shallow clone when aliasing (never mutates the caller's packet, which persistence holds by reference for QoS>0 and must keep its real topic for resend). Non-publish / non-v5 / non-advertising clients hit a cheap early guard. - connect.js: parse the client's topicAliasMaximum into `client._outboundTopicAliasMaximum`; init the per-connection alias Map only when > 0. - Strategy: assign 1..max in first-seen order, never evict; once full, later new topics send the full name (no alias). Tests: register-then-reuse, not-advertised (no aliasing), and full-table fallback. Docs: topicAliasMaximum note clarifying the two directions. Note: the alias map is bounded by min(clientMax, distinct topics delivered) — a client advertising 65535 could grow it accordingly (self-inflicted, per connection); a broker-side cap can be a follow-up if wanted. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…est + cap follow-up - Add a reconnect test pinning the §3.3.2-11 invariant: outbound Topic Alias mappings must not survive a Network Connection. A fresh Client gets a fresh alias map, so the first PUBLISH after reconnect on a previously-aliased topic carries the full topic + a newly-assigned alias, never a bare alias. - Reference the tracked follow-up (moscajs#1124) for a broker-side outbound alias cap in the connect.js comment, per the review thread (deferred to keep this PR focused). Refs moscajs#840 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…cap, test coverage Blocker (connect.js): a duplicated Topic Alias Maximum in CONNECT decodes to an array (a §3.1.2.11.5 Protocol Error) that slipped between two type-inconsistent guards and crashed the connection on first delivery. Honor only a numeric value; an array now disables outbound aliasing for the connection instead of crashing. Major (write.js/constants.js): the outbound alias table was unbounded and sized by the client (measured ~19 MB/conn for 10k topics). Clamp the effective per-connection maximum to min(clientAdvertised, OUTBOUND_TOPIC_ALIAS_LIMIT=64); a configurable cap stays tracked in moscajs#1124. Major (run_compat.py / mqtt-compat.yml): drop test_server_topic_alias from EXPECTED_GAPS — this PR implements it and the compat run now unexpectedly passes. Make the unexpected-pass guard fail the job (SystemExit) so a stale gap list can't reach a green check again. Major (tests): add the two missing coverage lanes — an aliased QoS 1 message left un-acked and resent after reconnect carries the full topic (QoS>0 path), and a multi-subscriber fanout (two v5 subs with independent per-connection alias sequences; a v3/v4 sub that never receives a Topic Alias). Minors: reject non-string (Buffer) topics from aliasing; note the alias-commit-before-write ordering for the planned maximumPacketSize follow-up; rename the inbound map _topicAliases -> _inboundTopicAliases; add a waitFor helper replacing the busy-wait polls; wait on broker-side deregistration on reconnect; strictEqual for the aliasing-off assertions; drop redundant client.end() calls; move the outbound-alias behaviour to a connackSent callout in docs; README nit. Refs moscajs#840 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
7f43786 to
52a8a38
Compare
|
Addressed the review (rebased onto current Blocker · dup Topic Alias Maximum → crash — Major · unbounded alias table — clamped the effective per-connection max to Major · EXPECTED_GAPS — removed Major · test coverage — added the two missing lanes: an aliased QoS 1 message left un-acked and resent after reconnect (raw subscriber so PUBACK is under our control) carries the full topic; and a fanout with two v5 subscribers (independent per-connection alias sequences — the spread-not-mutate proof on the shared Minors — Buffer-topic guard ( Two I did not fold in — flagging for your call rather than guessing:
Full suite green, lint + tsd clean. |
robertsLando
left a comment
There was a problem hiding this comment.
deep-review round 2 — 52a8a38
Verdict: Ship with minor changes. Everything from round 1 is genuinely fixed — verified against the code, not the commit message.
| Round-1 finding | Status | Evidence |
|---|---|---|
| Blocker · dup Topic Alias Maximum | Fixed | connect.js:314-317 numeric guard. The new wire test really does emit 06 22 0005 22 0005 and re-parses to [5,5]; it would have thrown on 7f43786. |
| Major · unbounded alias table | Fixed | OUTBOUND_TOPIC_ALIAS_LIMIT = 64 at the single assignment site; grep shows one write, one read — no unclamped path. A smaller client value is still honored (min(2,64) = 2). |
Major · stale EXPECTED_GAPS |
Fixed | Entry removed, remaining four intact, ast.parse clean. SystemExit(1) propagates — heredoc not a pipe, no continue-on-error, and if: always() doesn't mask a step's own exit code. |
| Major · test coverage | Partial | Both lanes exist and their behavioural assertions are sound, but neither pins the clone — see the line comment. |
Minors (Buffer guard, ordering comment, rename, waitFor, dereg wait, strictEqual, docs callout, README) |
Fixed | Rename is complete: grep -rnw '_topicAliases' lib/ aedes.js types/ test/ docs/ → zero hits. waitFor throws past a 2 s deadline and replaced all seven alias-lane busy-waits. |
test/mqtt5.js 78/78, test/qos2.js 21/21. No rebase fallout — the extra files in 7f43786..52a8a38 are #1116 landing on mqttv5 (merge-base = 0f5fc3f).
Two Minors as line comments above. Your two questions are answered in a separate comment on the PR thread. Plus:
- [Nit]
.github/workflows/mqtt-compat.yml:102— the step is still named "Warn on unexpected passes" although it now fails the job. "Fail on unexpected passes". - [Nit]
waitForis used only in the alias lanes; ~15 rawwhile (…) await delay(5)busy-waits remain elsewhere intest/mqtt5.js(:602,:827,:860,:885, …), mostly from the rebase. Not this PR's scope, but the helper exists now and the file is inconsistent. - [FYI] The cap bounds entry count, not bytes. 64 never-evicted keys with topics up to 65535 B (unbounded unless
maximumPacketSizeis set) is ≈4.2 MB/connection worst case — vastly better than the ~4 GB ceiling before, but #1124 may want a byte budget rather than an entry count.
Coverage
Verified this round: round-1 fix confirmation (adversarial, per-item), regression hunt on all new code, rename completeness, workflow exit-code propagation, mqtt-packet property-decoding domain, and a mutation test of the clone invariant. npm test in full (lint + tsd + all suites) exceeded my 2-minute budget and was not run to completion — the 78/78 and 21/21 figures are node --test on test/mqtt5.js and test/qos2.js only. CI is green on all 17 checks.
|
Answering your two open questions (round-2 review is posted separately with the line-level items).
I told you only 2 of 13 The altitude argument also runs the other way from how I first put it. Topic aliasing is a pure wire-encoding concern: same message, different bytes. Duplicated-property class-fix — agreed, separate change. Please open the issue. I searched; there isn't one yet. Three things worth capturing in it so the scope is deliberate:
Nothing couples the two: this PR already closes the Blocker defensively, so the class-fix can land whenever. |
…discriminating clone test - connect.js: gate the outbound Topic Alias Maximum on `> 0` (the same predicate write.js uses via `!max`) instead of `typeof === 'number'`, so a negative value — reachable when a preConnect hook mutates the packet — yields 0 at BOTH sites and can't diverge into a bogus-max crash. Added a preConnect-negative regression test. - tests: rewrite the QoS 1 clone test so it actually pins the clone. The old one passed even against an in-place-mutation variant (QoSPacket copies topic by value). The new one exploits that aedes-packet reference-shares `properties`: publisher sends a User Property, an aliased QoS 1 message is left un-acked, the session reconnects advertising NO Topic Alias Maximum, and the resent PUBLISH must carry no stale topicAlias. Verified it fails on the mutating variant. - mqtt-compat.yml: rename the step "Warn on unexpected passes" -> "Fail on unexpected passes" (it now SystemExits). - Opened moscajs#1131 for the general duplicated-CONNECT-property validation (separate, base-branch change per review). Refs moscajs#840 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Thanks — round-2 items addressed in
[Nit] Renamed the step to "Fail on unexpected passes". Duplicated-property class-fix — opened #1131, capturing your three scoping points (base-branch behaviour change on FYI (byte budget vs entry count) and the remaining non-alias busy-waits — noted; the byte-budget point is worth adding to #1124, and the busy-wait cleanup is out of scope here now that Full suite green, lint + tsd clean. |
robertsLando
left a comment
There was a problem hiding this comment.
Deep review — round 3
Verdict: Needs work, but narrowly. No blockers; ~30 LOC of changes, not a rework.
Top 3 risks
- No broker-side kill switch — outbound aliasing is fully client-triggered and un-tunable.
- The alias table is bounded by count (64), not bytes, and never evicts — memory amplification plus permanent starvation on churning topic sets.
- The Paho compat job cannot fail on the regression it was added to catch.
Strengths
- The tests are genuinely discriminating, not tautological. Verified by mutation: reverting
withOutboundTopicAliasto mutate in place, and breaking thealiases.size >= maxboundary, each produced a failing test. - Non-alias overhead measured at <0.5% (1.8 ns/packet with no field, 3.5 ns with
max = 0, against ~1 us forwriteToStream). The!max-first predicate ordering is what buys that — keep it first if the guard grows. - The
EXPECTED_GAPSremoval is backed by a real passing run, not a silenced entry: CI run 29898530229 showstest_server_topic_alias: pass. - Two prior review rounds landed 13 of 15 threads.
Theme. Most of the Majors share one root cause: the resource policy is hardcoded and client-driven rather than an operator-owned broker option. Deferring that to #1124 is what turns a single design gap into four separate findings.
Finding outside the diff
[Minor] · Security — lib/client.js:404 close() releases neither _outboundTopicAliases nor _inboundTopicAliases, unlike the adjacent _parser._queue = null. Audited and this is not a leak — Client is a fresh instance per TCP connection and close() -> unregisterClient drops the broker's reference on every disconnect path (error, timeout, takeover, close), so both maps become collectible with the client. It only shortens the retention window when something still holds a closed client (conn.client backref, in-flight mqemitter callbacks, takeover races). Cheap to do while you are there.
Prior threads
lib/write.js:70— still open. Per-feature v5 logic sits on the universalwrite()hot path rather thandeliver0/writeQoS. The placement is defensible (it is the only choke point coveringclient.publish(), retained and queued sends) — but every other per-client outbound PUBLISH shaping (nl,rap,subscriptionIdentifier,messageExpiryInterval) lives in the delivery wrappers atsubscribe.js:225-243andconnect.js:530-537, including the same clone-never-mutate invariant. Worth stating the rationale where the other shaping lives so the split does not read as accidental.lib/handlers/connect.js:315— behaviour now matches (> 0 ? Math.min(...) : 0closes the-1case, pinned bytest/mqtt5.js:417-441). The thread's textual ask (one shared predicate rather than two expressions) is still literally true; the remaining live gap is theNumber.isIntegercomment below.
Audited clean
- Commit-before-write is safe as the in-code comment claims: every
write()error path (deliver0->_onError,writeQoS->emit('error')via the listener atclient.js:116,emptyQueue-> pipeline ->_nextBatch(err)) destroys the socket, and mqtt-packet validates topic/messageId/properties before emitting any bytes. No broker/client alias desync. The caveat about a future packet-dropping path is the right guard to keep. - No cross-client leakage:
resolveTopicAliasclears the publisher's inbound alias (publish.js:174) before fanout, andwrite.jsspreads bothpacketandpacket.propertiesrather than mutating the reference-shared objects. - Workflow security:
on: pull_request(notpull_request_target),permissions: contents: read, no secrets, pinned Paho ref. No new dependencies. - Pre-existing, not introduced here:
restoreSubsruns beforedoConnack, so a concurrently published message can be written — and now register an alias — before the CONNACK. Aliasing makes it more visible.
Coverage
Eight specialists run against a dedicated worktree at the PR head (8cb493c): correctness, security, performance, DRY/codebase-fit, design/API/backcompat, tests, operability, readability. None skipped. Repo-wide DRY and sibling-file context intact. Test findings are mutation-verified; performance figures are measured, not estimated.
| // Registered topic: send an empty topic + the alias. | ||
| return { ...packet, topic: '', properties: { ...packet.properties, topicAlias: known } } | ||
| } | ||
| if (aliases.size >= max) { |
There was a problem hiding this comment.
[Major] · Security / Performance / Correctness (flagged independently by three specialists)
The cap bounds entry count, not bytes, and entries are first-come with no eviction. Two distinct failure modes:
(a) Memory. An MQTT topic is up to 65535 bytes, so 64 never-evicted entries is ~4 MB of client-controlled per-connection state — measured 1.9-3.4 KB retained for a realistic full table, ~190-340 MB at 100k v5 connections, and far worse under attack. A client advertises topicAliasMaximum: 64, subscribes #, self-publishes 64 max-length topics, then idles (keepalive is client-chosen, maximumPacketSize defaults to unlimited) — transient upload converts into persistent retention for the connection's life. This is net-new surface: before this PR the broker held no per-connection outbound topic table at all. Note the asymmetry with the inbound table, which is admin-bounded via broker.topicAliasMaximum (default 0); this one is client-bounded.
(b) Starvation. MQTT 5's own request/response idiom uses unique reply topics (resp/<uuid>). A # subscriber — or any wildcard covering churning topics — fills all 64 slots with one-shot topics that never repeat, then aliases nothing for the rest of the connection. Each of those 64 also paid +3 bytes for a registration that never amortizes. The feature becomes pure overhead on exactly the busiest connections.
Fix: "never evict" is a design choice, not a spec constraint — §3.3.2.3.4 explicitly permits reassigning an alias by resending it with a new topic name, so eviction is legal and O(1). LRU on the least-recently-used slot, or registering a topic only on its second send, fixes (b) and raises hit rate; bounding total key bytes fixes (a). The constants.js:15-20 comment currently claims the count bound addresses the amplification vector — it addresses only half of it.
There was a problem hiding this comment.
Still open, and this round escalates it to a Blocker with numbers.
The table is bounded by entry count (64), never by bytes. validateTopic caps topic levels, not length, so a single topic can be ~64 KB, and the Map keys pin those strings for the whole connection when they would otherwise be garbage right after delivery.
Measured on this branch: 20 connections x 64 distinct 60 KB topics retained 146.9 MB (7.3 MB per connection). The identical run with outboundTopicAliasMaximum: 0 retained 0.3 MB. That extrapolates to roughly 7 GB at 1000 connections — and it is on by default.
A byte budget for the table would close it, or simply skip aliasing topics over a few hundred bytes: long topics are the least likely to recur, so they are the worst candidates for an alias slot anyway.
There was a problem hiding this comment.
Still open at 0d2f83d, and this round's security pass raised it again independently: aedes.js:95-97 floors outboundTopicAliasMaximum at 0 but sets no ceiling, unlike the sibling maxTopicLevels two lines below. Worth noting the framing in the PR body ("self-inflicted per connection") isn't quite right — the table is keyed by full topic strings chosen by whoever publishes, and one retained copy lands on every subscribed connection, so the broker pays N-times amplification for content a third party picks.
| if (Date.now() > deadline) throw new Error(`waitFor timed out: ${msg}`) | ||
| await delay(5) | ||
| } | ||
| } |
There was a problem hiding this comment.
[Minor] · DRY & Codebase Fit
waitFor is a third variant of a wait idiom that already exists twice over: this file inlines while (cond) await delay(5) roughly 18 times (e.g. :635, :860, :1054, :1362, :1492) plus a bespoke serverConnected poller at :1567, and shared async test utilities live in test/helper.js (withTimeout, checkNoPacket, nextPacketWithTimeOut).
Why: the real win here — a timeout instead of a 3-minute hang — does not reach any of the existing loops while it stays file-local.
Fix: put it in test/helper.js next to the other wait helpers. Migrating the raw loops can follow separately.
There was a problem hiding this comment.
Still open: waitFor is now the third wait idiom in this file, and the new test at line 246 uses the older while (!broker.clients[...]) await delay(5) form rather than the helper this PR adds.
There was a problem hiding this comment.
Still open at 0d2f83d — waitFor is still a third wait idiom alongside the inline while-loop the new tests use.
| @@ -233,6 +233,8 @@ Emitted when server sends an acknowledge to `client`. Please refer to the MQTT s | |||
|
|
|||
| For MQTT 3.1/3.1.1 the packet carries a `returnCode` (`0` = success). For MQTT 5.0 it instead carries a `reasonCode` (`0x00` = success; the v3/v4 return codes map to the equivalent v5 reason codes) and may carry a `properties` object advertising negotiated capabilities (e.g. `topicAliasMaximum`, `maximumPacketSize`, `receiveMaximum`, `serverKeepAlive`, `assignedClientIdentifier`, `sharedSubscriptionAvailable`). | |||
There was a problem hiding this comment.
[Nit] · Design / Docs
The outbound-alias callout lives under Event: connackSent, and the option bullet at line 55 links here — but the behaviour it describes concerns outbound PUBLISHes, not the CONNACK event. Optional: move it near Event: publish or give it its own section.
There was a problem hiding this comment.
Still open (nit): the callout sits under connackSent, which is not where a reader looks for outbound PUBLISH behaviour.
There was a problem hiding this comment.
Still open at 0d2f83d — the outbound-alias explainer is still under Event: connackSent rather than near the option it documents.
… + minors Major (flagged by three specialists): the outbound Topic Alias cap is now the `outboundTopicAliasMaximum` broker option (default 64, `0` disables) instead of a hardcoded constant — operator-tunable, in the established pattern (defaultOptions + Aedes types + docs bullet). The effective per-connection max is `min(client-advertised, broker option)`. Major: a caller-supplied `properties.topicAlias` on the public `client.publish()` is now treated as caller-owned — forwarded verbatim, not broker-aliased — so the broker table can't disagree with a hand-rolled one and misdeliver. Major (compat CI): `run_compat.py` now exits non-zero on any non-gap, non-skipped failure (and on an unexpected pass), so a regression in a test removed from EXPECTED_GAPS (like test_server_topic_alias) turns the check red instead of only lowering the reported percentage. Minors: - `Number.isInteger` on the advertised max (a fractional preConnect-injected value no longer slips a non-integer through the guard). - Declare `_outboundTopicAliasMaximum`/`_outboundTopicAliases` in the constructor (monomorphic hot-path shape; the `> 0 ⇒ Map` invariant is now local). - `close()` releases both alias maps. - Fanout test reframed to assert distinct per-connection alias numbering + v4 exclusion (the clone invariant is pinned by the QoS 1 poison test; it isn't observable on the QoS 0 fanout path). New tests: client above the broker cap is clamped; `outboundTopicAliasMaximum: 0` disables outbound aliasing. Refs moscajs#840 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Round-3 addressed in Major · broker option (flagged ×3) — landed here rather than deferring to #1124: Major · caller-supplied Major · compat CI can't go red — Minors — Fanout test [Minor] — you're right it wasn't discriminating, and digging in, it can't be on the QoS 0 fanout path: each subscriber's Deferred with reasoning:
Full suite green, lint + tsd clean. |
…existing gaps) The previous "exit non-zero on any non-gap failure" turned the Paho job red on ~6 pre-existing not-yet-implemented features (test_maximum_packet_size, test_offline_message_queueing, …) that aren't tracked in EXPECTED_GAPS. Narrow the gate to an EXPECTED_PASSES set — tests aedes implements whose Paho case is their only interop evidence (test_server_topic_alias, removed from EXPECTED_GAPS here) — so a regression there turns the check red while the WIP denominator stays counted in the percentage. Unexpected passes still fail. Refs moscajs#840 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The caller-owned topicAlias early-return in write.js was the only uncovered patch line. Add a test: a broker-side client.publish() with properties.topicAlias set is forwarded verbatim (topic kept, alias 3 not reassigned). Refs moscajs#840 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
robertsLando
left a comment
There was a problem hiding this comment.
Deep review — outbound Topic Alias
Verdict: Needs work — three Blockers, all in the new alias path.
Top 3 risks
- A client's Will properties can inject a Topic Alias that hijacks an alias slot in other clients' tables.
- A caller-set alias silently desynchronizes the broker's table, misdelivering later messages under the wrong topic.
close()leaves the max/map pair inconsistent, dropping in-flight QoS>0 writes and firing spuriousclientError.
Theme: all three share one root cause — properties.topicAlias is treated as broker-owned state, but it is remotely settable and never re-validated. A second theme is teardown: the alias maps are the only per-connection state in client.js that gets nulled, and nothing that reads them was updated to expect null.
Strengths. The non-aliasing path is genuinely cheap (one monomorphic read, then return) — the guard ordering was done right. The clone-not-mutate invariant holds for persistence and MQEmitter, so QoS>0 resend keeps its real topic. Direction separation between inbound (broker.topicAliasMaximum, §3.2.2.3.8) and outbound (§3.1.2.11.5) is correct. Per-connection lifetime is right — the map lives on Client, so it never survives a takeover. Tests are deterministic (no fixed sleeps) and several assert on wire-decoded bytes via raw sockets. The previous review round's 20 addressed threads landed real fixes.
MQTT 5.0 conformance
Mostly right, two real violations.
Correct: direction separation (§3.2.2.3.8 inbound vs §3.1.2.11.5 outbound), per-connection lifetime (§3.3.2.3.4, §4.1 — never crosses a takeover), and §4.4 resend safety (the spread preserves dup/messageId/qos/retain, the stored packet keeps its real topic, and the fresh post-reconnect table forces a full topic).
Violated: [MQTT-3.1.2-27] and [MQTT-3.1.2-26] on the caller-alias path (see the inline Blockers), plus the §3.1.2.11.5 duplicate-property Protocol Error handled leniently (inline Minor).
Compat workflow — up to date, and the PR undersells it
Verified by reading the full files, not just the diff: test_server_topic_alias is cleanly moved out of EXPECTED_GAPS["v5"] into the new EXPECTED_PASSES["v5"], and a failure now sets regressions → sys.exit(1) → red job. It is hard-gated, not merely re-counted. Both directions are checked (a stale gap that starts passing also fails the build), and no other list needed syncing — a repo-wide grep for EXPECTED_GAPS / EXPECTED_PASSES / test_server_topic_alias came back clean.
Actions are all on current majors (checkout@v7, setup-node@v6, setup-python@v6, upload-artifact@v7), PAHO_REF is pinned to a commit SHA, Node 24 matches ci.yml's newest matrix entry and engines.node: ">=20", and permissions / concurrency / timeout-minutes: 20 are all present. It runs on every PR with no path filter — broader than "PRs touching lib/", not narrower.
Findings outside the diff
These are real but anchor to unchanged lines, so they can't be inline comments:
tools/mqtt-compat/README.md:50-65— still says "Two categories are handled specially" and "Both lists inrun_compat.py". The new CI-gatingEXPECTED_PASSESset is undocumented, so a contributor following the README won't learn a must-pass gate exists or how to add to it. Verified: no occurrence ofEXPECTED_PASSESanywhere in the README.test/types/aedes.test-d.ts:26— setstopicAliasMaximumbut neveroutboundTopicAliasMaximum.tsdis the only compile-check of the new declaration, so the field can drift unexercised.lib/handlers/connect.js:331—client._will = packet.willstores the Will with its properties intact; this is the source half of the Will-hijack Blocker commented inline onlib/write.js.
Still-open threads from the previous round
31 unresolved threads reconciled against the PR head: 20 genuinely addressed (verified against the code, resolving those separately), 11 still open. Of those 11, two are escalated by this round with new evidence and I've replied on them directly:
lib/write.js:44— count-bound cap, no eviction. Now measured: 20 connections × 64 distinct 60 KB topics retained 146.9 MB (7.3 MB/conn) vs 0.3 MB withoutboundTopicAliasMaximum: 0. ~7 GB at 1000 connections, on by default.lib/write.js:60— allocation per aliased publish. Now benchmarked: ~2× serialize cost, plus a suspected polymorphic-IC tax on non-aliasing clients.
The rest still stand as written: lib/write.js:77 (v5-only logic on the universal write() path), docs/Aedes.md:55/:237/:235 (the connackSent callout still says outbound aliasing is "automatic, no option" and points at #1124 for a configurable cap — while line 56 of the same file documents the option this PR ships; three specialists flagged this independently), lib/constants.js:21 (no test advertises a maximum above 64), test/mqtt5.js:382 (full-table test never re-verifies a cached alias), test/mqtt5.js:384 (no QoS 2 resend-alias twin), test/mqtt5.js:56 (waitFor is now the third wait idiom), test/mqtt5.js:321 (comment cites the wrong write.js lines).
Coverage
Dimensions run: correctness, security, performance, DRY/codebase-fit, design/API/backcompat, tests, operability, readability, MQTT-5 spec conformance, compat-workflow freshness. Readability came back essentially clean. Review ran against a worktree at the PR head (5e02cf5), so repo-wide checks had full context. Both Blocker claims were re-verified by hand against lib/write.js, lib/handlers/connect.js:331, lib/handlers/publish.js:176, and lib/client.js:430-431.
| // INTEGER: mqtt-packet decodes a duplicated property to an array and a preConnect | ||
| // hook can set a fractional/negative value on the mutable packet; the `> 0` | ||
| // integer gate matches write.js's `!max` so a bad value yields 0 at both sites. | ||
| const advertisedTopicAliasMax = packet.properties?.topicAliasMaximum |
There was a problem hiding this comment.
[Minor] · MQTT 5.0 conformance
A repeated Topic Alias Maximum in CONNECT — which mqtt-packet decodes as an array — is silently coerced to "aliasing off".
Why: §3.1.2.11.5 states that including it more than once is a Protocol Error, and §4.13 requires CONNACK reason code 0x82 followed by a close. test/mqtt5.js:591 now pins the non-conformant outcome (reasonCode === 0), so the deviation is locked in by a test. Blast radius is small, but it's worth being deliberate about.
Fix: either treat a non-integer Topic Alias Maximum as a Protocol Error, or document this as a deliberate lenient-parse policy shared with the other v5 properties — the current state reads as an oversight rather than a choice.
There was a problem hiding this comment.
Still open at 0d2f83d — a duplicated Topic Alias Maximum property still silently disables aliasing instead of returning Protocol Error 0x82.
| run: kill "${BROKER_PID}" 2>/dev/null || true | ||
|
|
||
| - name: Warn on unexpected passes | ||
| - name: Fail on unexpected passes |
There was a problem hiding this comment.
[Minor] · Operability
This "Fail on unexpected passes" step re-derives the same unexpected_pass condition that run_compat.py:384 (if regressions or xpass: sys.exit(1)) already fails on. By the time it runs, the job is red from the earlier "Run compatibility suites" step for the same test names.
Why: not incorrect, just dead weight — but it has an asymmetric side-effect. The regression path (an EXPECTED_PASSES test such as test_server_topic_alias starting to fail) surfaces only as a plain stderr line, with no ::error annotation, while the xpass path gets one. Verified the job still fails correctly either way (run: steps use bash -eo pipefail, so the script's exit code propagates).
Fix: drop the now-redundant step, or extend it to emit a ::error annotation for regressions too.
There was a problem hiding this comment.
Still open at 0d2f83d — the workflow's inline "fail on unexpected passes" step still re-derives from result.json what run_compat.py already signals via its exit code. Two places to keep in sync for one rule.
| for r in reports for t in r["results"] | ||
| if t["name"] in EXPECTED_PASSES.get(r["protocol"], set()) | ||
| and t["status"] != "pass"] | ||
| if regressions: |
There was a problem hiding this comment.
[FYI] · Operability
EXPECTED_PASSES treats any non-pass status as a regression, including the harness's own timeout status. A transient Paho subprocess timeout therefore turns the check red with no retry.
This reads as intentional — a hard gate is the point — and I'd leave it. Flagging only so the flake surface is known before it fires on someone else's unrelated PR.
There was a problem hiding this comment.
Still open at 0d2f83d (noted as FYI, no change was requested). Flagging only because this round found a related gap: EXPECTED_PASSES is membership-only, so if test_server_topic_alias ever stops appearing in results — a Paho rename on a PAHO_REF bump, or a load failure — regressions is empty and the job goes green, silently dropping the one hard gate protecting this feature.
… surface Blockers: - Will-property alias hijack: a client could set will.properties.topicAlias; resolveTopicAlias only strips it off a live inbound PUBLISH, never off a Will, so on disconnect the Will fanned out with the attacker's alias — hijacking an alias slot in OTHER clients' tables (cross-client misdelivery / forced 0x94). Now the write path scrubs any packet-carried topicAlias (Will, retained, caller) before emitting, and the Will is also scrubbed at storage (the inbound-PUBLISH analogue). - Caller-alias desync: forwarding a caller-set topicAlias verbatim (my prior-round fix) let the broker's table diverge from the client's and misdeliver, and emitted aliases a client never negotiated ([MQTT-3.1.2-26/27]). topicAlias is broker-owned on the outbound path: a caller value is now always stripped and the broker applies its own aliasing. - close() map/max inconsistency: nulling the alias maps while `connected` was still true let an in-flight deliver hit aliases.get() on null (TypeError -> spurious clientError + dropped QoS>0 write). Now `_outboundTopicAliasMaximum` is zeroed in the same statement (write.js short-circuits on !max), and the inbound map is no longer nulled at all (no `!max` guard to hide behind; collected with the Client like subscriptions/duplicates). Major (design/backcompat): `outboundTopicAliasMaximum` now defaults to 0 (opt-in), matching the inbound `topicAliasMaximum` sibling — enabling it changes wire behaviour and adds per-connection memory, so it must not turn on silently on upgrade. This also resolves the memory concern (off by default) and the fail-open input handling: a non-integer / negative value now coerces to 0 (disabled), never to an enabled default. Tests: caller alias stripped (enabled + disabled/[MQTT-3.1.2-27]); Will delivered to subscribers gets a broker alias, never the client-supplied one; retained flush-on-subscribe is aliased and registers the alias. Existing outbound tests updated to enable the now-opt-in option. Docs: the connackSent callout no longer claims "automatic, no option"; option default corrected to 0; README documents the EXPECTED_PASSES CI gate; test-d exercises the new option. Refs moscajs#840 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Making outboundTopicAliasMaximum opt-in (default 0) turned off the feature for the Paho compat broker, so the EXPECTED_PASSES-gated test_server_topic_alias regressed to fail (red CI). Enable it on tools/mqtt-compat/broker.js (65535) alongside the inbound topicAliasMaximum, so the compat run measures real broker ability rather than a disabled feature. Verified locally that the compat broker now aliases outbound PUBLISHes. Refs moscajs#840 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Round-4 addressed in Blockers
Majors
Deferred, with reasoning
Full suite green, lint + tsd clean, compat gate green ( |
robertsLando
left a comment
There was a problem hiding this comment.
Round-7 deep review — outbound (broker-assigned) Topic Alias
Verdict: Ship with minor changes. No blockers.
Top 3 risks
- The published
.d.tssays the feature defaults to on with 64 slots when it defaults to0— the opposite of the opt-in posture the whole PR rests on. - A non-integer inbound alias bypasses the range check and grows the inbound map without bound (
publish.js:153). - The compat harness's new hard gate is membership-only and goes green if the test simply disappears.
Strengths. The shallow-clone-vs-persistence invariant genuinely holds — verified that aedes-packet shares properties by reference across the fanout, and every aliasing branch clones both levels, so the packet persistence holds keeps its real topic. Per-connection scoping, 1..max numbering, reconnect reset and close() teardown all check out against §3.3.4; the cleanStart=false + un-acked-QoS1 case is pinned by a real test. Tests are live-socket and byte-level, not tautologies. The Client constructor initialises both new fields unconditionally, so there's no hidden-class split and v3/v4 clients pay one property read with zero allocation.
Measured interop. Downloaded the mqtt-compat-result artifacts: head 0d2f83d (run 33058791964) is v5 65% (17/26); the sibling feat/v5-enhanced-auth branch on the same base (run 33061639309) is v5 62% (16/26). Exactly one test differs — test_server_topic_alias ❌→✅. v3 is 100% (10/10) on both. Nothing regressed: test_client_topic_alias sends TopicAliasMaximum = 0 and no other v5 test sets the property, so broker.js's outboundTopicAliasMaximum: 65535 has zero blast radius. Good result.
Themes. (a) The type/doc surface has drifted from the code in four places. (b) The feature has no operator-facing signal at all.
Prior threads. Reconciled all 22 unresolved threads against this head: 9 were genuinely fixed and I've resolved them; 13 are still open and I've replied on each with what this round found. The biggest carry-overs, all confirmed again independently this round: the unbounded/never-evicting alias table (write.js:59), the two allocations per aliased publish (write.js:76), and the complete absence of an operator surface (client.js:44) — on that last one, when the table fills, write.js silently and permanently falls back to full topic names for the rest of the connection, dropping the entire benefit the operator enabled the feature for, with no event, counter or log. publish.js:130 emits clientError for every other notable v5 outcome, so it also breaks a local convention.
Not anchored to a line — docs/Client.md:133: withOutboundTopicAlias now strips properties.topicAlias from every v5 publish, including when outboundTopicAliasMaximum is 0, so callers of the documented public client.publish() who set it themselves silently lose it. Spec-justified, but it's an undocumented contract change on a public method.
FYI — 6 of the 10 v5 non-passes (test_will_message, test_will_delay, test_maximum_packet_size, test_offline_message_queueing, test_publication_expiry, test_server_keep_alive) are in neither EXPECTED_GAPS nor HARNESS_LIMITED: pre-existing, ungated, unflagged.
Coverage. Eight lanes ran against a worktree at 0d2f83d: correctness, security, performance, DRY/codebase-fit, design/API/backcompat, tests, operability, and MQTT-5 spec + Paho interop. Compat numbers above are measured from CI artifacts, not estimated.
| // Coerce a non-integer / negative value to 0 (disabled), not to the default: | ||
| // "bad value" must never mean "silently enabled". A string '0' from env/JSON | ||
| // config, -1, or 0.5 all disable outbound aliasing rather than turning it on. | ||
| this.outboundTopicAliasMaximum = Number.isInteger(opts.outboundTopicAliasMaximum) && opts.outboundTopicAliasMaximum > 0 |
There was a problem hiding this comment.
[Minor] · Design / API
Third distinct validation style for a numeric option in this file: outboundTopicAliasMaximum silently coerces bad values to 0, the sibling topicAliasMaximum (line 94) takes opts raw, and maxTopicLevels clamps. The same "positive integer, else 0" gate is also written at connect.js:316, with write.js:47's !max and client.js:443 tied to it by prose.
Why: A realistic '64' from env or JSON silently disables the feature with no signal, and docs/Aedes.md:56 doesn't mention it. Four sites hold one invariant together with comments.
Fix: At minimum document that a non-integer or negative value disables it; better, extract one positiveInt() into lib/utils.js (alongside ackProperties/armLongTimer) and use it at both parse sites.
| // hook can set a fractional/negative value on the mutable packet; the `> 0` | ||
| // integer gate matches write.js's `!max` so a bad value yields 0 at both sites. | ||
| const advertisedTopicAliasMax = packet.properties?.topicAliasMaximum | ||
| client._outboundTopicAliasMaximum = packet.protocolVersion === 5 && Number.isInteger(advertisedTopicAliasMax) && advertisedTopicAliasMax > 0 |
There was a problem hiding this comment.
[Minor] · DRY & Codebase Fit
The other negotiated CONNECT limits set just above (client.maximumPacketSize, client.receiveMaximum) are public and declared in types/client.d.ts:38,43; the negotiated outbound alias max is _-prefixed and undeclared. Separately, broker-side clamping of a client-requested v5 value has a precedent as a broker method (aedes.js:356 clampSessionExpiry); this one is inlined in the handler.
Why: Same family of values, gratuitously different visibility — plugins and tests can inspect one and not the other.
Fix: Either expose it consistently or note why it differs.
Majors: - types/instance.d.ts: the outboundTopicAliasMaximum comment said "(default: 64)" but the option defaults to 0 (opt-in). Corrected to "(default: 0)" — 5 lanes flagged this one-word IDE-hover contradiction. - publish.js resolveTopicAlias: reject a non-integer inbound alias with Number.isInteger. A duplicated Topic Alias property decodes to an array; `[1,2] < 1`/`> max` are both false via NaN, so it slipped the range check and .set() a fresh array key per PUBLISH — an unbounded inbound-map DoS bypassing topicAliasMaximum. The inbound sibling of the outbound guard. Test added (rejected 0x94, map does not grow). - run_compat.py: the EXPECTED_PASSES gate was membership-only, so an expected-pass test VANISHING (Paho rename / load failure) went green. Now also fail when an expected-pass name is absent from the evaluated set, or is simultaneously listed as a gap/harness- limited. Minors: - write.js: an empty-topic packet carrying a caller alias is left untouched (stripping would emit a zero-length topic with no alias — malformed [MQTT-3.3.2-1]); stripTopicAlias uses `= undefined` not `delete` (avoids the dictionary-mode deopt, matching publish.js); the commit-before-write comment now names the real stream.destroy → conn 'error' → _onError path (writeToStream doesn't throw). - aedes.js: clamp outboundTopicAliasMaximum to 65535 (Two Byte Integer), matching the maxTopicLevels precedent. - client.js: .clear() the inbound alias map symmetrically on close (kept a Map — safe). - connect.js: document the duplicated-Topic-Alias-Maximum lenient parse as deliberate (degrade a MAY optimisation rather than refuse an otherwise-valid CONNECT). - Tests: QoS1 poison test asserts the resent topic too; full-table test re-verifies a cached alias still resolves; stale write.js line citation fixed; §3.3.2.3.4 citation. - README: the unexpected-pass path fails the build (was "emits a CI warning"). Refs moscajs#840 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Round-7 addressed in Majors
Minors
Deliberate / deferred
Full suite green, lint + tsd clean, all CI checks green. |
robertsLando
left a comment
There was a problem hiding this comment.
Round-8 deep review — outbound (broker-assigned) Topic Alias
Reviewed at 4d8d41d in an isolated worktree, across 8 parallel lenses (correctness, security, MQTT-5 conformance, performance, DRY, design/API, tests, operability) plus a prior-thread reconciliation pass.
Verdict: Needs work — narrowly. No Blockers; ~40 LOC plus two test fixes, not a rework.
Top 3 risks
- Two tests that exist to protect the load-bearing clone contract were mutation-proven not to catch it breaking.
- A caller-supplied
topicAliasescapes the scrub on two reachable packet shapes, so the broker emits an alias the client never negotiated ([MQTT-3.1.2-27]). - The alias table is bounded by entry count, not bytes — open since round 5, and the amplification is worse than previously quantified.
Strengths. The spec work is genuinely correct: absent is treated identically to 0, allocation starts at 1, per-connection lifetime survives takeover, and QoS resend keeps the real topic. The Paho test_server_topic_alias flip is substantiated — I fetched client_test5.py at the pinned PAHO_REF 9d7bb80, and the test has no early return: three phases pinning the register/reuse pair, the absent case, and the explicit 0 case. mqtt-packet 9.0.2 behaviour was verified against source (topicAlias id 35 int16; getProperties skips undefined; writeToStream.publish() accepts topic: ''), not assumed.
Theme. The scrub half of this feature — always-on, independent of the option — is under-documented and under-tested relative to the aliasing half, which is opt-in and well covered.
Carry-over
12 of the 25 open threads are still unaddressed. I re-verified each against the head rather than trusting isOutdated. The one that moved is lib/write.js:65 (byte-unbounded table): the amplification is larger than round 5 quantified, because the table is filled by what the broker delivers — one attacker publishing 64 distinct 64 KB topics into a wildcard subscription fills the table of every v5 subscriber at once. Roughly 10,000x amplification: ~4 MB of attacker traffic against ~40 GB retained across 10k subscribers. At the 65535 ceiling that tools/mqtt-compat/broker.js:22 actually sets, it is ~4.3 GB per connection. Detail on the thread.
The other 13 threads are genuinely fixed; I am resolving those.
Not anchorable to the diff
[Minor] - Security / DRY / Spec (flagged independently by three lenses) - aedes.js:94, outside this diff. The inbound topicAliasMaximum still takes opts raw while its new sibling three lines below gets Number.isInteger + Math.min(..., 65535). The value is advertised as an int16 CONNACK property, so 65536, Infinity, or a JSON-config string makes mqtt-packet stream.destroy() every v5 CONNECT — a self-inflicted v5 outage. Infinity additionally lifts the alias > max bound in resolveTopicAlias. Pre-existing, but the asymmetry is newly visible now that one of the two adjacent options is hardened and the other is not.
Coverage
Lenses run: correctness, security, MQTT 5.0 conformance, performance, DRY/codebase-fit, design/API, tests, operability. Suite executed at the head: 86 pass. Readability was not run — tests and DRY covered the large new file, and no non-test source file cleared the size threshold.
| // ACK the first (full-topic) delivery so only the aliased one is left pending. | ||
| const first = rx1.find(p => p.cmd === 'publish') | ||
| raw1.write(generate({ cmd: 'puback', messageId: first.messageId }, { protocolVersion: 5 })) | ||
| await pub.publishAsync('o/q1', 'two', { qos: 1, properties: props }) |
There was a problem hiding this comment.
[Major] - Tests
This test passes with withOutboundTopicAlias mutated to write in place on both branches — I ran it.
Two independent maskings: (1) aedes-packet's Packet constructor copies topic by value on every wrap, so the topic assertion holds regardless of write.js's clone choice; (2) the second connection reconnects with aliasing disabled, so the off-path stripTopicAlias strips the poisoned alias before it could reach the wire — the very defense this test is meant to check independently of.
Why: the shallow-clone contract that this PR's own comments call load-bearing is currently unprotected. An in-place-mutation regression ships undetected.
Fix: have the second connection reconnect still advertising a max, so the disabled-path stripping cannot mask the bug — or assert on object identity / the persisted packet rather than on final wire bytes after a masking code path.
| const { connect } = await createServerAndConnect(t, { brokerOptions: { outboundTopicAliasMaximum: 5 } }) | ||
| // The subscriber advertises a Topic Alias Maximum, so the broker may alias. | ||
| const sub = connect({ clientId: 'oalias-sub', properties: { topicAliasMaximum: 5 } }) | ||
| await once(sub, 'connect') |
There was a problem hiding this comment.
[Major] - Tests
This test passes with all three reset lines deleted from lib/client.js close() (_outboundTopicAliasMaximum = 0, _outboundTopicAliases = null, _inboundTopicAliases.clear()) — I ran it.
A full disconnect-then-reconnect always builds a brand-new Client, so the reset-on-close path is never exercised; the new object simply supersedes the old state.
Why: the scenario the PR's comments explicitly worry about — an in-flight deliver0/deliverQoS from a closing client, or a live takeover where the old Client is still referenced — is not what this tests, and no test would catch a regression there.
Fix: force a genuine takeover (connect a duplicate clientId while the first connection is still open, not after end() and deregistration), and/or keep a reference to the old Client and assert close() actually cleared its alias state.
| // Only alias non-empty *string* topics: a Buffer topic (reachable via | ||
| // client.publish(), which skips topic validation) would key the Map by object | ||
| // identity, so an equal-but-distinct Buffer never matches — burning a fresh | ||
| // alias slot per publish until the table is permanently full. |
There was a problem hiding this comment.
[Major] - MQTT 5.0 conformance / Correctness (two lenses)
The "aliasing off / not applicable" passthrough only scrubs a caller-supplied properties.topicAlias when the topic is a non-empty string. Two shapes escape and are written verbatim to a v5 client:
- a Buffer topic +
topicAlias: n—callerAliasrequirestypeof === 'string', so this falls intoreturn packet topic: ''+topicAlias: n— documented as intentional
Both are reachable through the public client.publish(), and both are emitted even when the feature is off (the default).
Spec: [MQTT-3.1.2-27] — "If Topic Alias Maximum is absent or zero, the Server MUST NOT send any Topic Aliases to the Client." With aliasing on, the empty-topic form additionally shares one namespace with the broker's own table, so the receiver resolves the alias to the wrong topic ([MQTT-3.3.2-11]/[MQTT-3.3.2-13]) — or, if unregistered, answers 0x94 and drops.
The oalias-27-sub test proves this path is considered reachable; it just doesn't cover these two shapes.
Fix: in the passthrough branch, scrub or fail the write on any caller alias the broker did not assign. An empty-topic packet is unsendable either way, so failing the write is strictly better than a wire violation.
| // Registered topic: send an empty topic + the alias (overwrites any caller alias). | ||
| return { ...packet, topic: '', properties: { ...packet.properties, topicAlias: known } } | ||
| } | ||
| if (aliases.size >= max) { |
There was a problem hiding this comment.
[Major] - Operability
When a connection's outbound alias table fills (aliases.size >= max), the broker silently reverts to full-topic PUBLISHes for that client. No event, log, or counter marks it.
Why: this codebase already has the convention for exactly this case. sessionLimitReached exists because, per its own doc (docs/Aedes.md:256), "without it the guard would degrade behavior silently"; rejectPacketTooLarge (lib/client.js:399) emits clientError explicitly as telemetry before acting. Outbound-alias exhaustion breaks that convention — an operator cannot tell that a client's table filled and the connection quietly lost its bandwidth optimization, nor size outboundTopicAliasMaximum for the workload.
Fix: emit a one-shot signal the first time a connection's table fills, latched the way _rejectingPacketTooLarge is.
|
|
||
| For MQTT 3.1/3.1.1 the packet carries a `returnCode` (`0` = success). For MQTT 5.0 it instead carries a `reasonCode` (`0x00` = success; the v3/v4 return codes map to the equivalent v5 reason codes) and may carry a `properties` object advertising negotiated capabilities (e.g. `topicAliasMaximum`, `maximumPacketSize`, `receiveMaximum`, `serverKeepAlive`, `assignedClientIdentifier`, `sharedSubscriptionAvailable`). | ||
|
|
||
| > __MQTT 5.0 outbound Topic Alias:__ opt-in via the [`outboundTopicAliasMaximum`](#new-aedesoptions) broker option (default `0` = off). When it is set __and__ a v5 client advertises its own `Topic Alias Maximum` in CONNECT, the broker assigns Topic Aliases on the PUBLISHes it sends to that client — full topic + alias on first use of a topic, empty topic + the same alias thereafter — to save bandwidth. Mappings are per Network Connection (reset on reconnect, never carried across, per §3.3.2.3.4) and the per-connection table never evicts. The number of aliases assigned is `min(client's advertised maximum, outboundTopicAliasMaximum)`. Full documentation is under the [`outboundTopicAliasMaximum`](#new-aedesoptions) option. |
There was a problem hiding this comment.
[Major] - Design / API
"Opt-in ... default 0 = off" is not accurate.
lib/write.js:46-58 strips a caller-supplied properties.topicAlias from every v5 non-empty-topic PUBLISH, and lib/handlers/connect.js:346-348 deletes topicAlias from every Will — both regardless of the option. test/mqtt5.js:596 asserts this default-on behaviour, so it is intended, just undocumented.
Why: the back-compat claim doesn't hold. An embedder who never sets outboundTopicAliasMaximum and today calls client.publish({ topic: 'x', properties: { topicAlias: 7 } }) silently loses the alias on upgrade.
Fix: document the always-on scrub — outbound topicAlias is broker-owned, and a caller- or Will-supplied value is dropped — and reword "off" to "no aliases assigned".
| client._outboundTopicAliasMaximum = packet.protocolVersion === 5 && Number.isInteger(advertisedTopicAliasMax) && advertisedTopicAliasMax > 0 | ||
| ? Math.min(advertisedTopicAliasMax, client.broker.outboundTopicAliasMaximum) | ||
| : 0 | ||
| if (client._outboundTopicAliasMaximum > 0) { |
There was a problem hiding this comment.
[Major] - DRY / Codebase fit
_outboundTopicAliases is the only per-connection collection in the codebase allocated outside a constructor (grep "new Map()" gives lib/client.js:39, aedes.js:136/140/144, and this line).
Its direct sibling _inboundTopicAliases is allocated once at lib/client.js:39 and merely .clear()d at teardown. The outbound one is declared null at lib/client.js:45, allocated here, and nulled at lib/client.js:440-441.
Why: the resulting "max > 0 implies the map is a Map" invariant is hand-maintained across four sites with long explanatory comments (lib/client.js:41-45, lib/client.js:430-441, here, and lib/write.js:49-59) — a shape the repo uses nowhere else. Any future write path that reads the map without the !max guard breaks it.
Fix: allocate unconditionally in the Client constructor like the inbound sibling, make _outboundTopicAliasMaximum the sole gate, and make teardown .clear(). The three invariant comments then collapse.
| proto = r["protocol"] | ||
| evaluated = {t["name"] for t in r["results"]} | ||
| for t in r["results"]: | ||
| if t["name"] in EXPECTED_PASSES.get(proto, set()) and t["status"] != "pass": |
There was a problem hiding this comment.
[Minor] - Correctness
The EXPECTED_PASSES gate only iterates protocols present in reports. A protocol that never ran — --protocols v3, or a report that failed to materialize — has its expected-pass entries checked zero times, and the script exits 0.
Why: this is precisely the silent-green failure this block's own comment claims to close ("would leave regressions empty and the job green"). The absent check added last round catches a vanished test, not a vanished protocol.
Fix: diff EXPECTED_PASSES.keys() against the protocols actually reported, and fail on any that is missing.
| # gap list — an unexpected pass — fails too. | ||
| # | ||
| # Membership isn't enough: an EXPECTED_PASSES test that VANISHES from the results | ||
| # (a Paho rename on a PAHO_REF bump, a load failure) would leave `regressions` |
There was a problem hiding this comment.
[Minor] - Operability
The regression gate does fail the job (sys.exit(1)), but the reason only reaches a plain print(..., file=sys.stderr) — never a ::error annotation, unlike the adjacent xpass check which does get one. The regressed row in report.md and the sticky PR comment render as an indistinguishable failure marker, with nothing separating a hard-gated regression from an ordinary WIP gap.
Why: an operator seeing the red check has to dig through raw logs to find REGRESSION (expected-pass gate): .... The two artifacts people actually look at give no clue why it is red.
Fix: emit a ::error annotation for regressions, mirroring the xpass block, and mark regressed EXPECTED_PASSES rows distinctly in render_markdown.
| trustedProxies?: string[]; | ||
| // MQTT 5.0 broker limits, advertised in CONNACK. | ||
| topicAliasMaximum?: number; // max inbound topic alias; 0 disables (default: 0) | ||
| outboundTopicAliasMaximum?: number; // broker-side cap on outbound topic aliases per connection; opt-in, 0 disables (default: 0) |
There was a problem hiding this comment.
[Minor] - Design / API
Neither this declaration nor docs/Aedes.md:56 states that aedes.js:101-103 silently coerces a non-integer or negative value to 0 (disabled) and clamps the top to 65535.
Why: outboundTopicAliasMaximum: '64' arriving from env or JSON config silently disables the feature with no signal. The repo's own convention is to state clamping inline — types/instance.d.ts:87 says "clamped to [1, 100]".
Fix: add the coerce-to-0 and 65535-ceiling sentence to both the type comment and the doc.
| // Topic Alias (0x23) among them. A Topic Alias is meaningless on a stored Will | ||
| // (it names an entry in a since-gone connection's alias table) and, once the Will | ||
| // fans out to other subscribers, an attacker-chosen value would hijack an alias | ||
| // slot in their tables. The write path scrubs it defensively, but drop it at the |
There was a problem hiding this comment.
[Nit] - Correctness
delete client._will.properties.topicAlias uses delete, while both sibling scrubs (lib/handlers/publish.js:180, lib/write.js:29) deliberately assign undefined to avoid the dictionary-mode transition on a reference-shared properties object.
Behaviourally identical here; inconsistent with the rule the other two sites state explicitly.
…, test teeth - write.js: scrub a caller-supplied topicAlias on EVERY topic shape (Buffer and empty-string topics escaped the old non-empty-string-only gate), so the broker never emits an alias it did not assign [MQTT-3.1.2-27]. - aedes.js: clamp inbound topicAliasMaximum like its outbound sibling (Number.isInteger + Math.min(65535)); a bad value disables it rather than advertising an int16 mqtt-packet can't encode or an Infinity bound. - test(poison): the wire can never reveal a properties-poison after the write.js scrub (aliasing-on reconnect overwrites topicAlias, aliasing-off scrubs it), so read the stored packet straight out of persistence — the decisive clone check — and reconnect advertising a max for the realistic resume. Mutation-verified. - test(takeover): replace the reset-on-reconnect tautology (an independent connection trivially has its own map) with a genuine same-id session takeover. - test: cover the inbound topicAliasMaximum clamp (string/fraction disable it, over-range clamps to 65535). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…s, compat gate Completes the round-8 review beyond the top-3 (already in 0a62365): - [Major] write.js:67 — alias-table exhaustion was silent. Emit a one-shot `outboundTopicAliasExhausted` (client, { max }) event the first time a new topic can't be aliased, matching the sessionLimitReached / rejectPacketTooLarge telemetry convention. Latched per connection; typed + tsd + docs + test. - [Major] connect.js:327 / client.js — `_outboundTopicAliases` was the only per-connection Map allocated outside a constructor. Allocate it once in the Client ctor (like _inboundTopicAliases) and `.clear()` it at teardown, dropping the hand-maintained "max > 0 ⇒ Map" invariant; write()'s hot path now always reads a Map. `_outboundTopicAliasMaximum` stays the on/off switch. - [Major] Aedes.md:237 — documented that the caller-alias scrub (and Will scrub) run regardless of outboundTopicAliasMaximum; the option gates alias ASSIGNMENT, not the broker-owns-the-namespace scrub. - [Minor] types/instance.d.ts + Aedes.md — documented the coerce-to-0 / clamp-to-65535 behaviour for both alias-maximum options. - [Minor] run_compat.py — the expected-pass gate skipped a protocol that never produced a report (a --protocols subset / crash); now fails on a wholly-absent gated protocol, and emits a ::error annotation for regressions like the unexpected-pass step does. - [Nit] connect.js:344 — Will topicAlias scrub uses `= undefined`, not `delete`, matching the two sibling scrubs (avoids the dictionary-mode deopt). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Round-8 addressed —
|
Second of the next #821 batch, per the agreed sequence — the other low-blast-radius win. Broker-core only, cluster-safe, no persistence. Flips
test_server_topic_aliasfrom anEXPECTED_GAPtoward a pass.What
Inbound topic aliases (client→broker) already work; this adds the reverse (broker→client). When a client advertises a Topic Alias Maximum > 0 in its CONNECT (§3.1.2.11.5), the broker assigns aliases on outbound PUBLISH (§3.3.2.3.4):
Per connection, bounded by the client's advertised max. Server MAY, so it's entirely off unless the client opts in.
How
write.js—withOutboundTopicAliasrewrites a v5 PUBLISH just before serialize. It returns a shallow clone when aliasing, so the caller's packet — which persistence holds by reference for QoS > 0 and must keep its real topic for resend after reconnect — is never mutated. Non-publish / non-v5 / non-advertising clients hit a cheap early guard (no allocation).connect.js— parse the client'stopicAliasMaximumintoclient._outboundTopicAliasMaximum; allocate the per-connection aliasMaponly when > 0.1..maxin first-seen order, never evict; once the table is full, later new topics send the full name (no alias). (Eviction/LRU is a possible future optimization.)Tests
Register-then-reuse (full topic + alias, then empty topic + same alias); not-advertised (no aliasing); full-table fallback to the full topic.
Note
The alias map is bounded by
min(clientMax, distinct topics delivered)— a client advertising65535could grow it accordingly (self-inflicted, per connection). A broker-side cap option could be a follow-up if you'd like one.369 unit tests + lint + tsd green.
🤖 Generated with Claude Code