Skip to content

logging: root every layer's XLOG category (standalone dep maps + picoquic bridge + smoke test) - #312

Open
gmarzot wants to merge 8 commits into
mainfrom
standalone-xlog-dep-category-roots
Open

logging: root every layer's XLOG category (standalone dep maps + picoquic bridge + smoke test)#312
gmarzot wants to merge 8 commits into
mainfrom
standalone-xlog-dep-category-roots

Conversation

@gmarzot

@gmarzot gmarzot commented Jul 3, 2026

Copy link
Copy Markdown
Collaborator

Makes every per-layer --logging selector work, and adds a regression test. Three parts, one theme — each XLOG layer is rooted at its natural category name.

Problem

Every bundled dependency compiles with an absolute __FILE__, so folly derives categories like home.runner.work.moxygen...MoQSession. Result: --logging=moxygen=DBG4, quic=WARN, fizz=DBG1, quic.picoquic=DBG3, etc. match nothing — only the root level (--logging=DBG4) works. The per-layer operator UX documented in moqx docs/logging.md and the PicoQuicXLogSink header is effectively dead.

Fix — three complementary pieces

1. C++ dependencies (standalone/CMakeLists.txt). We don't own their CMakeLists, so root the categories from the consumer side: before each dep's add_subdirectory, add -fmacro-prefix-map to strip its FetchContent source root from __FILE__. folly then derives folly.*, fizz.*, wangle.*, quic.mvfst.*, proxygen.*, moxygen.*.

-fmacro-prefix-map (not -ffile-prefix-map) is deliberate: rewrites only __FILE__/macros (debug-info paths intact), stacks across deps without macro-redefinition conflicts, and shortens __FILE__ before folly's own (ineffective-when-embedded) FOLLY_XLOG_STRIP_PREFIXES runs — neutralizing that latent bug too. mvfst is mapped under quic/mvfst/ (the documented category) so it doesn't collide with picoquic at a bare quic.* root. The moxygen map roots moxygen.* immediately, independent of the upstream moxygen fix (facebookexperimental#207).

2. picoquic bridge (PicoQuicXLogSink.cpp). The sink used bare XLOG(), so its events landed under openmoq.transport.pico.* instead of the advertised quic.picoquic.*. Add XLOG_SET_CATEGORY_NAME("quic.picoquic") — the right model for a bridge: it names its category explicitly rather than deriving it from __FILE__, so it needs no prefix-map.

3. Regression test (standalone/test/xlog_category_smoke.sh, CTest xlog_category_roots). Drives a short date-server/text-client session and asserts, per layer, that its selector enables that layer's DBG-level lines and excludes the others. Matches only V-prefixed (DBG) lines so INFO/WARN/ERR can'"'"'t mask a broken category; layers still on the glog backend are skipped, not failed. Validated against a pre-fix build: it correctly FAILs moxygen scoping while sanity-detecting the same lines at the root level.

These categories bake into the CI-built artifacts, so downstream consumers (e.g. the moqx relay) inherit the clean roots automatically.

Relationship to #207

facebookexperimental#207 fixes only moxygen.* (moxygen'"'"'s own CMakeLists). This PR covers the other five layers plus picoquic plus the test; the one moxygen map here is redundant-but-harmless once #207 lands and syncs.

Test plan

  • CI standalone build (deps on XLOG via MOXYGEN_LOGGING_FOLLY_XLOG=ON) runs ctest -R xlog_category_roots.
  • Manual: --logging=moxygen.MoQSession=DBG4 scopes to moxygen; --logging=quic.mvfst=DBG1 to transport; --logging=quic.picoquic=DBG3 to the picoquic bridge.

This change is Reviewable

@gmarzot gmarzot self-assigned this Jul 9, 2026
@gmarzot gmarzot changed the title standalone: root each dependency's XLOG categories at its natural name (+ smoke test) logging: root every layer's XLOG category (standalone dep maps + picoquic bridge + smoke test) Jul 9, 2026
@gmarzot
gmarzot requested a review from afrind July 21, 2026 14:01
gmarzot added 8 commits July 21, 2026 10:02
Fix per-layer --logging selectors (fizz=, wangle=, quic.mvfst=, proxygen=,
folly=, moxygen=) in the standalone/bundled build. Previously every bundled
dependency compiled with an absolute __FILE__, so folly derived categories like
home.runner.work.moxygen...MoQSession — meaning moxygen=DBG4, quic=WARN, etc.
matched nothing and only the root level or the literal absolute path worked.

The dependency CMakeLists are not ours to edit, so root the categories from the
consumer side: add -fmacro-prefix-map before each dep's add_subdirectory to
strip its FetchContent source root from __FILE__ at compile time. folly then
derives folly.*, fizz.*, wangle.*, quic.mvfst.*, proxygen.*, moxygen.*.

-fmacro-prefix-map (not -ffile-prefix-map) is deliberate: it rewrites only
__FILE__/macros and leaves debug-info paths intact, multiple maps stack without
macro-redefinition conflicts, and it shortens __FILE__ before folly's own
CMAKE_SOURCE_DIR-scoped (and therefore ineffective when embedded)
FOLLY_XLOG_STRIP_PREFIXES runs, neutralizing that latent bug too. mvfst is
mapped under quic/mvfst/ (the documented category) so it does not collide with
picoquic at a bare quic.* root. The moxygen map roots moxygen.* in this build
immediately, independent of the upstream moxygen strip-prefix fix.

These categories bake into the CI-built artifacts, so downstream consumers
(e.g. the moqx relay) inherit the clean roots without further changes.
Guards the -fmacro-prefix-map dependency-category rooting. The category is a
compile-time property of each dep's __FILE__, and the deps are not ours, so a
faithful check must exercise the built binaries: test/xlog_category_smoke.sh
drives a short date-server/text-client session and, for each layer on the XLOG
backend, asserts its selector (moxygen=, fizz=, quic.mvfst=, ...) enables that
layer's DBG lines and excludes the others. It matches only DBG-level ('V') lines
so INFO/WARN/ERR (which appear regardless of the selector) cannot mask a broken
category. Layers still on the glog backend are skipped, not failed.

Wired as CTest 'xlog_category_roots' under BUILD_TESTS AND BUILD_SAMPLES.

Validated against a pre-fix build: the test correctly FAILS moxygen scoping
(absolute-path category) while sanity-detecting the same lines at the root
level, confirming it catches the regression this change prevents.
…egory

The PicoQuicXLogSink header and operator docs advertise the quic.picoquic.*
category root (`--logging=quic.picoquic=DBG3`), but the sink used bare XLOG(),
so its events derived their category from __FILE__ and actually landed under
openmoq.transport.pico.* — meaning quic.picoquic=... matched nothing.

Add XLOG_SET_CATEGORY_NAME("quic.picoquic") at global scope so every XLOG in the
TU uses that category regardless of file path. This is the right model for a
bridge: it names its category explicitly rather than deriving it, and so is
independent of the dependency prefix-maps in this same change.
…oquic

Expand the category smoke test from three layers to all advertised ones:
- mvfst session (moqdateserver/moqtextclient) now covers moxygen, fizz,
  quic.mvfst, wangle, proxygen, and folly;
- a new, guarded picoquic session (pico_evb_relay_server/pico_evb_text_client,
  minted a throwaway openssl cert) covers quic.picoquic. It asserts the sink's
  lines are selected by quic.picoquic= AND excluded by openmoq.transport.pico=,
  which directly proves the XLOG_SET_CATEGORY_NAME override took effect.

The scoping check is now one session per layer (own marker present, every other
layer's marker absent) instead of an N*N matrix. Layers not on the XLOG backend
or not exercised by a session are skipped, not failed, so the test stays honest
across pin advances and BUILD_PICOQUIC being off. CTest passes the pico binaries
only when those targets exist. Validated against a pre-fix build: moxygen still
correctly FAILs while the glog-backend layers skip.
…ner dir

CI showed moxygen=DBG9 selected nothing while fizz= and quic.mvfst= worked. The
standalone build adds the INNER dir (<repo>/moxygen) as the subdirectory, and
the map computed _MOXYGEN_ROOT as <repo>/moxygen too — so it stripped
<repo>/moxygen/ and left a bare MoQSession.cpp, i.e. category "MoQSession",
which moxygen= never matches.

Strip the repo root (<repo>) instead, leaving moxygen/MoQSession.cpp ->
moxygen.MoQSession (and openmoq/... -> openmoq.*).
The picoquic pass skips because installPicoQuicXLogSink() is unwired (#318), not
because of a test fault. Cite the tracking issue so the SKIP is self-explanatory;
the pass auto-activates once the sink is installed.
Temporary: prints the DBG-level source files each session emits at root DBG9,
and forces a non-zero exit so ctest --output-on-failure surfaces it in the CI
log. Deps-on-XLOG only shows in CI (local .scratch is glog), so this reveals
which files wangle/proxygen/folly actually emit — to pick correct markers.
Reverted in the next commit.
Restore the normal exit. The CI inventory confirmed moxygen/fizz/quic.mvfst are
actively verified, and that wangle/proxygen/folly SKIP for structural reasons,
not bad markers:
  - proxygen is still on the glog backend (no XLOG option wired) — proxygen= is
    inert until it moves to XLOG;
  - wangle's Acceptor/ConnectionManager only fire behind a TCP HTTP acceptor
    (e.g. the moqx admin server), not the MoQ-over-QUIC path here;
  - folly's own XLOG is sparse.
Documented inline so the SKIPs are self-explanatory; they auto-activate if a
future build/session exercises those layers. Also added RecordLayer.cpp to the
fizz marker (seen in the inventory).
@gmarzot
gmarzot force-pushed the standalone-xlog-dep-category-roots branch from 4a151da to a5bbdd7 Compare July 21, 2026 14:02
@gmarzot
gmarzot requested a review from michalhosna July 26, 2026 13:54

@afrind afrind left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@afrind reviewed 3 files and all commit messages, and made 3 comments.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on gmarzot and michalhosna).


standalone/CMakeLists.txt line 476 at r1 (raw file):

    # stacks across deps without macro-redefinition conflicts, and shortens
    # __FILE__ *before* folly's own (CMAKE_SOURCE_DIR-scoped, thus ineffective
    # here) FOLLY_XLOG_STRIP_PREFIXES runs — neutralizing that latent bug too.

Same here. I recently learned that claude needs to "talk to itself" in the comments in order to write good code, and needs either a pre-tool hook or a second pass to clean this up to something humans actually want.


standalone/CMakeLists.txt line 586 at r1 (raw file):

moxygen_install_headers(moxygen ${CMAKE_CURRENT_SOURCE_DIR}/../moxygen)

# Regression guard for the per-layer XLOG category rooting above: drives short

Does this belong here in the standalone/CMakeLists.txt top level? Maybe it's better in a sub-file (under test?)


moxygen/openmoq/transport/pico/PicoQuicXLogSink.cpp line 26 at r1 (raw file):

// overriding the file-path-derived default (openmoq.transport.pico.*). This is
// the semantic root operators target (`--logging=quic.picoquic=DBG3`), and it
// keeps picoquic independent of the build-side prefix-maps: unlike the C++ deps,

Consider whether we can shorten this comment a bit?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants