Skip to content

Stabilize and refactor autotest execution - #334

Draft
ezhk wants to merge 1 commit into
mainfrom
agent/refactor-autotest-runner
Draft

ezhk wants to merge 1 commit into
mainfrom
agent/refactor-autotest-runner

Conversation

@ezhk

@ezhk ezhk commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Summary

  • replace run-tests.sh and autotest/yanet-autotest-run.py with one typed run-tests.py entry point
  • run memory-bounded autotest batches with configurable jobs, batch size, and CPU spacing
  • add an adaptive interactive status bar and stable CI progress output
  • update Meson installation, documentation, and the Docker image workflow to use the unified runner
  • make firewall synchronization and route-tunnel fixtures tolerate their expected asynchronous packets

Root cause

Repeated and parallel autotest runs could leave periodic firewall synchronization packets crossing step boundaries. Those valid delayed packets were then reported as unknown, while short receive windows could report expected packets as missing. Running every fixture in a separate container also amplified memory usage.

The new runner groups fixtures into isolated, core-remapped worker containers, retries failed shared batches individually, and retains bounded logs. The affected fixtures now explicitly expect synchronization packets and use longer receive limits. When a build prefix is used, the runner also exposes prefixed YANET binaries to child commands because yanet-autotest invokes yanet-cli internally.

Impact

Local runs use two workers, four fixtures per worker batch, and three cores per worker by default. All important configuration is available through command-line arguments. GitHub Actions remains compatible through the installed run-tests.py autotest-runner command.

Validation

  • ./run-tests.py autotest --jobs 1 --batch-size 1 --pattern 051_firewall_keepstate_with_sync_unicast
  • smoke runs for isolated fixtures 001 and 002
  • synthetic pass, failure, retry, TTY, and redirected-output checks
  • Python syntax and function-docstring validation
  • git diff --cached --check

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant