Handle MemoryError in submission pipeline

[tylerbessire]

Summary

• catch MemoryError in solve_with_budget and fall back to best-so-far prediction
• track per-task memory failures and run gc.collect() to free resources
• add regression test exercising MemoryError path and document progress marker

Testing

• python -m py_compile arc_submit.py
• pytest tests/test_solve_with_budget_memory.py -q
• pytest test_beam_search_fix.py -q

---

https://chatgpt.com/codex/tasks/task_e_68c5048f2e18832297e8b1c0856171a5

Summary by CodeRabbit

• New Features
  • Improved robustness: tasks that run out of memory now fail gracefully, fall back to the best-known result, and proceed without crashing.
  • Enhanced reporting: per-task logs indicate memory error status, and the final summary includes a memory error count.
  • Extended metadata now includes a memory error flag for clearer run diagnostics.
• Documentation
  • Added an update entry describing memory error handling behavior and test results.
• Tests
  • Added unit tests validating memory error fallback behavior and metadata flags.

[coderabbitai]

Caution

Review failed

The pull request is closed.

Walkthrough

Adds MemoryError handling and instrumentation to arc_submit.solve_with_budget, extending metadata and logs, invoking gc per task, and tracking memory error counts. Documents the change in AGENTS.md and introduces a unit test validating fallback to best_so_far and memerror flag behavior.

Changes

Cohort / File(s)	Summary
Documentation update AGENTS.md	Adds a progress entry noting MemoryError handling in the submission script, with date, test invocation, and brief notes.
Core submission logic arc_submit.py	Extends solve_with_budget to catch MemoryError, set meta.memerror, fall back to solver.best_so_far(task), run gc.collect() per task, log memerror per task, and aggregate memory error counts in final output. No public signatures changed.
Unit tests tests/test_solve_with_budget_memory.py	Adds a test ensuring MemoryError triggers memerror=True, timeout=False, and output falls back to best_so_far using a DummySolver.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor Runner as Main/CLI
  participant Submit as solve_with_budget
  participant Solver as Solver
  participant GC as gc

  Note over Runner,Submit: Per task execution
  Runner->>Submit: solve_with_budget(task, budget, solver)
  Submit->>GC: gc.collect()
  Submit->>Solver: solve_task_two_attempts(task, budget)
  alt Success
    Solver-->>Submit: result
    Submit->>Runner: outputs, meta{timeout: false, memerror: false}
  else Timeout
    Solver-->>Submit: best_so_far(task)
    Submit->>Runner: outputs from best_so_far, meta{timeout: true, memerror: false}
  else MemoryError (new)
    Solver--x Submit: MemoryError
    Submit->>Solver: best_so_far(task)
    Solver-->>Submit: fallback result
    Submit->>Runner: outputs from best_so_far, meta{timeout: false, memerror: true}
  end
  Note right of Submit: Runner aggregates mem_error_count and logs per-task flags

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

I nibbled through the memory hedge, hop-hop, no fear,
When bytes went poof—no crash, I steered clear.
Best-so-far carrots in my little cache,
GC sweeps the garden, tidy in a flash.
Flags up: memerror! but onward I quest—
With budgeted bounces, I bring back the best. 🥕🐇

✨ Finishing touches

• 📝 Generate Docstrings

🧪 Generate unit tests

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch codex/fix-memoryerror-during-submission-process

---

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d609fc6 and d483b12.

📒 Files selected for processing (3)

• AGENTS.md (1 hunks)
• arc_submit.py (3 hunks)
• tests/test_solve_with_budget_memory.py (1 hunks)

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}