logging: root every layer's XLOG category (standalone dep maps + picoquic bridge + smoke test) - #312
logging: root every layer's XLOG category (standalone dep maps + picoquic bridge + smoke test)#312gmarzot wants to merge 8 commits into
Conversation
Fix per-layer --logging selectors (fizz=, wangle=, quic.mvfst=, proxygen=, folly=, moxygen=) in the standalone/bundled build. Previously every bundled dependency compiled with an absolute __FILE__, so folly derived categories like home.runner.work.moxygen...MoQSession — meaning moxygen=DBG4, quic=WARN, etc. matched nothing and only the root level or the literal absolute path worked. The dependency CMakeLists are not ours to edit, so root the categories from the consumer side: add -fmacro-prefix-map before each dep's add_subdirectory to strip its FetchContent source root from __FILE__ at compile time. folly then derives folly.*, fizz.*, wangle.*, quic.mvfst.*, proxygen.*, moxygen.*. -fmacro-prefix-map (not -ffile-prefix-map) is deliberate: it rewrites only __FILE__/macros and leaves debug-info paths intact, multiple maps stack without macro-redefinition conflicts, and it shortens __FILE__ before folly's own CMAKE_SOURCE_DIR-scoped (and therefore ineffective when embedded) FOLLY_XLOG_STRIP_PREFIXES runs, neutralizing that latent bug too. mvfst is mapped under quic/mvfst/ (the documented category) so it does not collide with picoquic at a bare quic.* root. The moxygen map roots moxygen.* in this build immediately, independent of the upstream moxygen strip-prefix fix. These categories bake into the CI-built artifacts, so downstream consumers (e.g. the moqx relay) inherit the clean roots without further changes.
Guards the -fmacro-prefix-map dependency-category rooting. The category is a
compile-time property of each dep's __FILE__, and the deps are not ours, so a
faithful check must exercise the built binaries: test/xlog_category_smoke.sh
drives a short date-server/text-client session and, for each layer on the XLOG
backend, asserts its selector (moxygen=, fizz=, quic.mvfst=, ...) enables that
layer's DBG lines and excludes the others. It matches only DBG-level ('V') lines
so INFO/WARN/ERR (which appear regardless of the selector) cannot mask a broken
category. Layers still on the glog backend are skipped, not failed.
Wired as CTest 'xlog_category_roots' under BUILD_TESTS AND BUILD_SAMPLES.
Validated against a pre-fix build: the test correctly FAILS moxygen scoping
(absolute-path category) while sanity-detecting the same lines at the root
level, confirming it catches the regression this change prevents.
…egory
The PicoQuicXLogSink header and operator docs advertise the quic.picoquic.*
category root (`--logging=quic.picoquic=DBG3`), but the sink used bare XLOG(),
so its events derived their category from __FILE__ and actually landed under
openmoq.transport.pico.* — meaning quic.picoquic=... matched nothing.
Add XLOG_SET_CATEGORY_NAME("quic.picoquic") at global scope so every XLOG in the
TU uses that category regardless of file path. This is the right model for a
bridge: it names its category explicitly rather than deriving it, and so is
independent of the dependency prefix-maps in this same change.
…oquic Expand the category smoke test from three layers to all advertised ones: - mvfst session (moqdateserver/moqtextclient) now covers moxygen, fizz, quic.mvfst, wangle, proxygen, and folly; - a new, guarded picoquic session (pico_evb_relay_server/pico_evb_text_client, minted a throwaway openssl cert) covers quic.picoquic. It asserts the sink's lines are selected by quic.picoquic= AND excluded by openmoq.transport.pico=, which directly proves the XLOG_SET_CATEGORY_NAME override took effect. The scoping check is now one session per layer (own marker present, every other layer's marker absent) instead of an N*N matrix. Layers not on the XLOG backend or not exercised by a session are skipped, not failed, so the test stays honest across pin advances and BUILD_PICOQUIC being off. CTest passes the pico binaries only when those targets exist. Validated against a pre-fix build: moxygen still correctly FAILs while the glog-backend layers skip.
…ner dir CI showed moxygen=DBG9 selected nothing while fizz= and quic.mvfst= worked. The standalone build adds the INNER dir (<repo>/moxygen) as the subdirectory, and the map computed _MOXYGEN_ROOT as <repo>/moxygen too — so it stripped <repo>/moxygen/ and left a bare MoQSession.cpp, i.e. category "MoQSession", which moxygen= never matches. Strip the repo root (<repo>) instead, leaving moxygen/MoQSession.cpp -> moxygen.MoQSession (and openmoq/... -> openmoq.*).
The picoquic pass skips because installPicoQuicXLogSink() is unwired (#318), not because of a test fault. Cite the tracking issue so the SKIP is self-explanatory; the pass auto-activates once the sink is installed.
Temporary: prints the DBG-level source files each session emits at root DBG9, and forces a non-zero exit so ctest --output-on-failure surfaces it in the CI log. Deps-on-XLOG only shows in CI (local .scratch is glog), so this reveals which files wangle/proxygen/folly actually emit — to pick correct markers. Reverted in the next commit.
Restore the normal exit. The CI inventory confirmed moxygen/fizz/quic.mvfst are
actively verified, and that wangle/proxygen/folly SKIP for structural reasons,
not bad markers:
- proxygen is still on the glog backend (no XLOG option wired) — proxygen= is
inert until it moves to XLOG;
- wangle's Acceptor/ConnectionManager only fire behind a TCP HTTP acceptor
(e.g. the moqx admin server), not the MoQ-over-QUIC path here;
- folly's own XLOG is sparse.
Documented inline so the SKIPs are self-explanatory; they auto-activate if a
future build/session exercises those layers. Also added RecordLayer.cpp to the
fizz marker (seen in the inventory).
4a151da to
a5bbdd7
Compare
afrind
left a comment
There was a problem hiding this comment.
@afrind reviewed 3 files and all commit messages, and made 3 comments.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on gmarzot and michalhosna).
standalone/CMakeLists.txt line 476 at r1 (raw file):
# stacks across deps without macro-redefinition conflicts, and shortens # __FILE__ *before* folly's own (CMAKE_SOURCE_DIR-scoped, thus ineffective # here) FOLLY_XLOG_STRIP_PREFIXES runs — neutralizing that latent bug too.
Same here. I recently learned that claude needs to "talk to itself" in the comments in order to write good code, and needs either a pre-tool hook or a second pass to clean this up to something humans actually want.
standalone/CMakeLists.txt line 586 at r1 (raw file):
moxygen_install_headers(moxygen ${CMAKE_CURRENT_SOURCE_DIR}/../moxygen) # Regression guard for the per-layer XLOG category rooting above: drives short
Does this belong here in the standalone/CMakeLists.txt top level? Maybe it's better in a sub-file (under test?)
moxygen/openmoq/transport/pico/PicoQuicXLogSink.cpp line 26 at r1 (raw file):
// overriding the file-path-derived default (openmoq.transport.pico.*). This is // the semantic root operators target (`--logging=quic.picoquic=DBG3`), and it // keeps picoquic independent of the build-side prefix-maps: unlike the C++ deps,
Consider whether we can shorten this comment a bit?
Makes every per-layer
--loggingselector work, and adds a regression test. Three parts, one theme — each XLOG layer is rooted at its natural category name.Problem
Every bundled dependency compiles with an absolute
__FILE__, so folly derives categories likehome.runner.work.moxygen...MoQSession. Result:--logging=moxygen=DBG4,quic=WARN,fizz=DBG1,quic.picoquic=DBG3, etc. match nothing — only the root level (--logging=DBG4) works. The per-layer operator UX documented in moqxdocs/logging.mdand thePicoQuicXLogSinkheader is effectively dead.Fix — three complementary pieces
1. C++ dependencies (
standalone/CMakeLists.txt). We don't own their CMakeLists, so root the categories from the consumer side: before each dep'sadd_subdirectory, add-fmacro-prefix-mapto strip its FetchContent source root from__FILE__. folly then derivesfolly.*,fizz.*,wangle.*,quic.mvfst.*,proxygen.*,moxygen.*.-fmacro-prefix-map(not-ffile-prefix-map) is deliberate: rewrites only__FILE__/macros (debug-info paths intact), stacks across deps without macro-redefinition conflicts, and shortens__FILE__before folly's own (ineffective-when-embedded)FOLLY_XLOG_STRIP_PREFIXESruns — neutralizing that latent bug too.mvfstis mapped underquic/mvfst/(the documented category) so it doesn't collide with picoquic at a barequic.*root. Themoxygenmap rootsmoxygen.*immediately, independent of the upstream moxygen fix (facebookexperimental#207).2. picoquic bridge (
PicoQuicXLogSink.cpp). The sink used bareXLOG(), so its events landed underopenmoq.transport.pico.*instead of the advertisedquic.picoquic.*. AddXLOG_SET_CATEGORY_NAME("quic.picoquic")— the right model for a bridge: it names its category explicitly rather than deriving it from__FILE__, so it needs no prefix-map.3. Regression test (
standalone/test/xlog_category_smoke.sh, CTestxlog_category_roots). Drives a short date-server/text-client session and asserts, per layer, that its selector enables that layer's DBG-level lines and excludes the others. Matches onlyV-prefixed (DBG) lines so INFO/WARN/ERR can'"'"'t mask a broken category; layers still on the glog backend are skipped, not failed. Validated against a pre-fix build: it correctly FAILsmoxygenscoping while sanity-detecting the same lines at the root level.These categories bake into the CI-built artifacts, so downstream consumers (e.g. the moqx relay) inherit the clean roots automatically.
Relationship to #207
facebookexperimental#207 fixes only
moxygen.*(moxygen'"'"'s own CMakeLists). This PR covers the other five layers plus picoquic plus the test; the onemoxygenmap here is redundant-but-harmless once #207 lands and syncs.Test plan
MOXYGEN_LOGGING_FOLLY_XLOG=ON) runsctest -R xlog_category_roots.--logging=moxygen.MoQSession=DBG4scopes to moxygen;--logging=quic.mvfst=DBG1to transport;--logging=quic.picoquic=DBG3to the picoquic bridge.This change is