Repository navigation
fix: mail resumes a Codex thread Codex unloaded while idle, never one the person archived - #218
reason-machines[bot] wants to merge 8 commits into
Conversation
… the person archived Co-authored-by: svemyh <40596752+svemyh@users.noreply.github.com>
… up archives off the lock; match replacement rollouts Co-authored-by: svemyh <40596752+svemyh@users.noreply.github.com>
…read; an archive lookup that times out is unknown, not archived Co-authored-by: svemyh <40596752+svemyh@users.noreply.github.com>
…; connect through Codex's socket link; keep old-Codex fallback; retry kept mail after a restart Co-authored-by: svemyh <40596752+svemyh@users.noreply.github.com>
…start test Co-authored-by: svemyh <40596752+svemyh@users.noreply.github.com> Co-authored-by: qa-runner2-star <319404260+qa-runner2-star@users.noreply.github.com>
…ould not load; finish that handoff past the 1-day wake cutoff Co-authored-by: svemyh <40596752+svemyh@users.noreply.github.com> Co-authored-by: qa-runner2-star <319404260+qa-runner2-star@users.noreply.github.com>
…ing again Co-authored-by: svemyh <40596752+svemyh@users.noreply.github.com> Co-authored-by: qa-runner2-star <319404260+qa-runner2-star@users.noreply.github.com>
Fifth review round (reviewer GPT-6.1 Sol, at 63f61f0), fixed in b18380fP2: ordinary session expiry dropped an accepted pending resume. Once the 6 h startup grace ended, Change: Checks (Reason cloud, Linux):
Gap: no test covers the name-keeping change in — Feedback resolver |
…on, and even if it was never reached Co-authored-by: svemyh <40596752+svemyh@users.noreply.github.com> Co-authored-by: qa-runner2-star <319404260+qa-runner2-star@users.noreply.github.com>
|
Fifth review round (head 48e3662) — Feedback resolver Fixes the two P2s the independent review (GPT-6.1 Sol) found at b18380f:
Checks (Reason cloud, Linux):
Not verified: live Codex Desktop, native Windows runtime. The known |
|
Ready for approval. Head 48e3662: all required checks (macOS, Ubuntu, Windows tests, CodeQL) pass. Independent re-review (GPT-6.1 Sol) is clean on this exact head: both b18380f P2s resolved, |
What and why
Feedback (sline, #feedback): a durable Codex Desktop thread went idle for 3 h; Codex's app-server logged
inactive_thread_unsubscribed, shut the thread down, and the Reason Network daemon loggedsession … endedat the same second. The session stayed listed offline with 3 unread messages until the person woke the same thread UUID by hand (Codex's own send_message_to_thread). The app and computer were running the whole time.Root cause. Codex runs
SessionEndwhenever a thread shuts down, including when the app-server unloads an idle one (openai/codex:hooks/src/events/session_end.rsalways sendsreason: "other", fromcore/src/session/handlers.rsshutdown). The daemon took the thread offline, which is right, since its route and name are kept. But its offline wake path rancodex queue --thread <id> --message …. That only adds the message to the thread's queue, and the queue is consumed only once something loads the thread again. Nothing did, so the mail waited for the person.Fix.
codex queuenow runs with--resume(src/daemon/mail.rs). This is Codex's supported way to do it (openai/codexcli/src/queue_cmd.rs,tui/src/session_queue_commands.rs::resume_if_unloaded). After queueing, Codex reads the thread and, only if it isNotLoaded, sendsthread/resumefor that same thread id. A thread that is already loaded is left alone. It never creates a new thread, chat or serve. This covers the offline path and the live idle path, where a thread can be unloaded while its process lives on.$CODEX_HOME/archived_sessions/rollout-<ts>-<id>.jsonl[.zst]. Newcodex::archivedmatches that exact id (not a suffix of one). For an archived thread, mail is queued without--resume, so it waits for the person to open the thread, as before. A thread Codex only unloaded keeps its rollout insessions/and is resumed. Folders that are off and failedcodex queuehandoffs keep their existing guards.codex queuethat predates--resumerefuses the flag (clap:unexpected argument '--resume' found) before it queues anything. The daemon then runs it again without the flag. Any other failure is still a failure.went offline; mail resumes itinstead ofended. The sender's note saysCodex resumes the same thread, which starts a turn with it, orit was archived…for an archived thread.Builds on #217 (no-transcript Codex threads are never sessions) and merges cleanly with its head.
Checks (Reason cloud, Linux)
cargo test: 345 + 5 + 2 passed. New tests:daemon::mail::tests::mail_resumes_a_codex_thread_codex_unloaded_but_never_one_the_person_archivedreproduces the report: start, MCP register, SessionEnd, MCP exit. The unloaded thread's mail runsqueue --thread t1 --resume, and the archived thread's runsqueue --thread t2with no--resume. With the--resumepush disabled, it fails (panics at thet1 --resumeassertion). With the fix, it passes.a_codex_without_queue_resume_still_takes_an_offline_threads_mail: the fake old Codex refuses--resume, the daemon retries without it, and the mail is read and not marked failed.only_a_codex_that_refuses_the_resume_flag_is_asked_again_without_itandcodex::tests::only_a_thread_whose_rollout_codex_moved_to_archived_sessions_is_archived, which covers plain and.zstrollouts, exact-id matching and a missing home.python3 tests/e2e/codex_busy.py: 59/59, including new checks that an offline thread is queued--resumeby its own id and that an archived thread is queued without it.python3 tests/e2e/hooks.py: 82/82.python3 tests/e2e/two_machines.py: 150/150 (its fake-queue assertion now expects--resume).cargo fmt --checkclean.cargo clippy --all-targets -- -D warningsclean.Not verified
ad3d352(2026-11-10).resume_if_unloadedneedscodex queueto reach a non-embedded app-server (the local daemon socket or--remote). If Codex Desktop's app-server isn't reachable that way, the message is still queued and Codex prints a warning but exits 0, so it waits for the thread to load, as it did before this change.codexCLI on its PATH, or in the usual install folders, to hand off offline mail. Without one, mail waits for hooks, as before.CODEX_RESUME_MS), which is unchanged.Shipping needs a reason-network release and the monorepo CLI's
ENGINE_VERSION/digest bump.— Feedback resolver
Review fixes
Independent review (GPT-6.1 Sol) raised three findings; all are fixed in this push.
codex queue --resumeexits 0 when it queued the message but could not resume the thread. Codex prints acould not resume threadwarning on stderr and still exits 0. The daemon now detects that warning, logs it, does not requeue the mail, and remembers the thread as not resumed until a later resume succeeds or the daemon restarts. Senders are told the message is queued and Codex was asked to load that thread, not that it was resumed. New test:mail_codex_queued_but_could_not_resume_is_not_claimed_resumed.Innerlock. It now runs on a blocking worker, off the lock, capped at 5 s. A timeout counts as archived, so an uncertain thread is never revived.rollout-<ts>-<thread>_<rollout>.jsonl[.zst]as well asrollout-<ts>-<thread>.jsonl[.zst], by thread UUID only.tests/e2e/codex_busy.pyand DESIGN.md were updated to use the new sender wording.Checks after the review fixes (Reason cloud, Linux)
cargo fmt --check: clean.cargo clippy --all-targets --locked -- -D warnings: clean.cargo test --locked: 346 + 5 + 2 passed.python3 tests/e2e/codex_busy.py: 59/59 on three consecutive runs. The first run on a CPU-starved machine reported 58/59, and its failing check was not captured. It did not happen again in the reruns.python3 tests/e2e/two_machines.py: 150/150.python3 tests/e2e/hooks.py: 82/82.Gaps
codex_busy.pyrun, described above.Second review round (GPT-6.1 Sol, re-review of 25425da)
codex queue --resumecan only load a thread through a shared app-server, the one on$CODEX_HOME/app-server-control/app-server-control.sock(openai/codextui/src/session_queue_commands.rs::resume_if_unloaded,app-server-transport::app_server_control_socket_path). Without one, Codex queues the mail anyway, and the thread can't be loaded later without queueing it again. Fix: offline wake mail goes to Codex only while that socket answers (codex::app_server_up). Otherwise it stays unread here, is not in flight, and senders hear it is kept and tried again. It is offered again with backoff, 30 s doubling to 30 min (keep_for_codex/retry_codex,Inner::codex_retry), and goes once into that same thread by its UUID with--resumewhen an app-server answers. Mail sent meanwhile carries the kept mail with it, so nothing is queued in Codex twice. The retry stops when the thread starts again (SessionStart clears it, and its hooks show the mail), when the mail is read, or whencodex queuetakes it. If--resumestill warns (the app-server went away between the probe and the call), the earlier warning path applies.archived_targetnow returnsOption<bool>. A timeout is unknown: for offline mail it's kept and retried like P1, never handed over without--resume. For a live, loaded thread it is still queued without--resume, which never revives an uncertain thread.codex_unresumedvsno_transcriptinInner). Not reconciled here: fix: Codex's ephemeral threads are never sessions on the network #217 hasn't merged andmaindoesn't conflict. Once fix: Codex's ephemeral threads are never sessions on the network #217 lands,mainwill be merged into this branch keeping both fields and both initializers, and the suites rerun.Checks at this head (Reason cloud, Linux)
cargo fmt --check: clean.cargo clippy --all-targets --locked -- -D warnings: clean.cargo test --locked: 349 + 5 + 2 passed. New tests:wake_mail_waits_here_until_a_codex_app_server_can_load_the_thread_then_goes_once: with no app-server, nothing is run, the mail is unread, the next sender hears it is kept, and retries keep it. Once the app-server answers, onequeue --thread t1 --resumecarries all three messages, they're read, the retry state is gone, and no more commands run.an_archive_lookup_that_takes_too_long_is_not_known: a slow lookup givesNone; archived and not-archived lookups giveSome(true)andSome(false).a_codex_app_server_is_up_only_while_its_control_socket_answers: no socket is down, a listening socket is up, and a stale socket file is down.python3 tests/e2e/codex_busy.py: 61/61 (two new checks: with no app-server the mail is kept unread and the sender is told; once a socket listens, both messages are queued once with--resume).python3 tests/e2e/two_machines.py: 150/150.python3 tests/e2e/hooks.py: 82/82.Gaps
codexand a plain listening socket, not a real app-server.Third review round (GPT-6.1 Sol, re-review of cdf1629)
All five findings held, and head 178d108 fixes them.
codex queue --resumeexited 0 with a "could not resume thread" warning, the daemon marked the mail read and never tried again.codex app-server proxy, which relays stdio to the control socket on every platform. Over it the daemon sends JSON-RPC over WebSocket:initializeandinitialized, thenthread/read, thenthread/resume(excludeTurns) only for a thread that isn't loaded. That is the same sequencecodex queue --resumeuses (openai/codextui/src/session_queue_commands.rs).CODEX_HOME. Codex advertises its control socket as a symlink to a short protected socket. The probe now connects to the link's target, and the proxy is given that target with--sock.codex queuehas--resumenow comes fromcodex queue --help. The answer is remembered per binary path and mtime.--resumeis tried, and a refusal still falls back as before.CODEX_HOME. An empty value now means the default home, as Codex treats it. This holds both for the environment variable and for a home recorded for the thread.DESIGN.md is updated to match.
Checks at this head (Reason cloud, Linux)
All passed:
cargo fmt --checkandcargo clippy --all-targets --locked -- -D warnings: both clean.cargo test --locked: 353 + 5 + 2 passed. New or changed tests:resume_loads_an_unloaded_thread_over_the_control_socket_and_leaves_a_loaded_one: run against a fake WebSocket app-server that also sends a ping, a notification, a frame longer than 125 bytes and a fragmented reply. An unloaded thread gets exactly the sequence above, a loaded one is left alone, and a refusal comes back as an error.an_app_server_behind_a_long_codex_home_answers_through_its_socket_link: connecting straight to the advertised path (longer than 108 bytes) fails, and the probe connects through the link.an_empty_codex_home_is_the_default_one.mail_codex_queued_but_could_not_resume_is_resumed_later_without_queueing_it_again: while resuming fails, the daemon keeps asking; once it succeeds it stops. Each message is queued exactly once, and a SessionStart also stops the asking.wake_mail_kept_before_a_daemon_restart_still_reaches_the_codex_thread: after a restart, the next mail carries the kept message ahead of it. With no new mail, the retry at startup delivers it by itself.a_codex_without_queue_resume_still_takes_an_offline_threads_mail: with old help text and no socket, the mail is queued once, without--resume.python3 tests/e2e/codex_busy.py: 61/61.python3 tests/e2e/two_machines.py: 150/150.python3 tests/e2e/hooks.py: 82/82.The e2e fake
codexnow answersqueue --helpwith--resume. The first e2e run failed (57/61 and 149/150) because the fakes didn't answerqueue --helpyet. With that fixed, both pass.Gaps
thread/resume, ascodex queue --resumedoes. Whether Codex then starts the queued turn isn't verified live.src/daemon/mod.rs(both PRs add a field and an initializer toInner) is still there. It gets resolved once fix: Codex's ephemeral threads are never sessions on the network #217 lands.Fourth review round (head 63f61f0)
The reviewer (GPT-6.1 Sol, Session 3bc06e82) found two more problems at 6af1a30. Both were real, and both are fixed.
codex_unresumed). After a restart, nothing asked Codex to load it again. Fix: that record is now also kept in the store (keycodex-unresumed:<session>), written when Codex takes the mail and removed once the thread loads, starts again, is archived or is no longer known. At startup the daemon reads it back and asks Codex to load the thread again with the same backoff. It only resumes the thread and never queues the mail again.resume_codexchecked the same "seen within the last day" rule that applies when new mail is sent. A thread that took mail just before the cutoff was dropped at its first retry. Fix: the cutoff now applies only when mail is sent. Finishing a handoff Codex accepted uses the thread and its home without the age check.DESIGN.md is updated to match.
Checks (Reason cloud, Linux, same tree as 63f61f0)
cargo fmt --check: clean.cargo clippy --all-targets --locked -- -D warnings: clean.cargo test --locked: 354 + 5 + 2 passed.a_thread_codex_took_mail_for_is_loaded_again_after_a_restart_and_past_the_cutoff. Codex takes mail without loading the thread, then the thread ages past the 1-day cutoff and the daemon is restarted (memory cleared). Codex is asked to load the thread again both times. Once it can, the record is cleared, and the mail is queued exactly once.pending()fails after the restart.python3 tests/e2e/codex_busy.py: 61/61.python3 tests/e2e/two_machines.py: 150/150.python3 tests/e2e/hooks.py: 82/82.The earlier Ubuntu CI failure at 178d108 was in the restart test, which assumed the fake
codex's log files come back in a fixed order. 6af1a30 fixed the test; the daemon code didn't change for it.Gaps (in addition to the ones above)