Skip to content

chore: fix flaky coverage tests#6986

Merged
LesnyRumcajs merged 1 commit intomainfrom
fix-flaky-coverage
Apr 30, 2026
Merged

chore: fix flaky coverage tests#6986
LesnyRumcajs merged 1 commit intomainfrom
fix-flaky-coverage

Conversation

@LesnyRumcajs
Copy link
Copy Markdown
Member

@LesnyRumcajs LesnyRumcajs commented Apr 30, 2026

Summary of changes

Changes introduced in this pull request:

  • this tentatively fixes the timeout issue we've been seeing on coverage tests; a potential culprit is the unsafe setenv we're calling from many threads https://man7.org/linux/man-pages/man3/setenv.3.html - this is MT-Unsafe so anything goes, including hangs.
  • added timeout per snapshot just in case

Reference issue to close (if applicable)

Closes

Other information and links

Change checklist

  • I have performed a self-review of my own code,
  • I have made corresponding changes to the documentation. All new code adheres to the team's documentation standards,
  • I have added tests that prove my fix is effective or that my feature works (if possible),
  • I have made sure the CHANGELOG is up-to-date. All user-facing changes should be reflected in this document.

Outside contributions

  • I have read and agree to the CONTRIBUTING document.
  • I have read and agree to the AI Policy document. I understand that failure to comply with the guidelines will lead to rejection of the pull request.

Summary by CodeRabbit

Release Notes

  • Tests
    • Added timeout protection to regression test execution with improved timeout error reporting
    • Enhanced test initialization process for random number generation

@LesnyRumcajs LesnyRumcajs requested a review from a team as a code owner April 30, 2026 10:25
@LesnyRumcajs LesnyRumcajs requested review from sudo-shashank and removed request for a team April 30, 2026 10:25
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Apr 30, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: c9a00334-01eb-4dc2-a816-5fa8484f3f94

📥 Commits

Reviewing files that changed from the base of the PR and between ea93dfa and e362ab3.

📒 Files selected for processing (1)
  • src/tool/subcommands/api_cmd/test_snapshot.rs
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/tool/subcommands/api_cmd/test_snapshot.rs

Walkthrough

Modifies test infrastructure in test_snapshot.rs to implement process-wide RNG seed initialization using LazyLock and wraps test execution within a 300-second timeout that panics explicitly upon expiration.

Changes

Cohort / File(s) Summary
Test Infrastructure Updates
src/tool/subcommands/api_cmd/test_snapshot.rs
Introduces LazyLock-based RNG seed initialization to replace per-test environment variable setup. Wraps the complete test sequence (proof parameter caching, downloads, RPC snapshot fetching, and execution) in a 300-second tokio::time::timeout with explicit panic on timeout.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Suggested reviewers

  • sudo-shashank
  • hanabi1224
  • akaladarshi
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Title check ❓ Inconclusive The title 'chore: fix flaky coverage tests' is generic and vague. It uses 'flaky' without specifying the root cause (thread-unsafe setenv usage) or the technical approach (LazyLock initializer and timeout wrapping). Consider a more specific title that captures the key technical change, such as 'chore: fix flaky coverage tests by removing thread-unsafe setenv and adding timeouts' or 'chore: replace per-test setenv with LazyLock and add timeout to snapshot tests'.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix-flaky-coverage
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch fix-flaky-coverage

Review rate limit: 4/5 reviews remaining, refill in 12 minutes.

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 30, 2026

❌ 28 Tests Failed:

Tests completed Failed Passed Skipped
2029 28 2001 0
View the top 3 failed test(s) by shortest run time
forest-filecoin::key_management::wallet::tests::secp_verify
Stack Traces | 0.007s run time
thread 'key_management::wallet::tests::secp_verify' (42669) panicked at src/key_management/wallet.rs:502:9:
assertion failed: sig.verify(&msg, &invalid_addr).is_err()
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::message_pool::msgpool::tests::test_message_pool
Stack Traces | 0.007s run time
thread 'message_pool::msgpool::tests::test_message_pool' (45225) panicked at .../message_pool/msgpool/mod.rs:433:69:
called `Result::unwrap()` on an `Err` value: Key already exists
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::cli::subcommands::mpool_cmd::tests::message_filtering_from
Stack Traces | 0.008s run time
thread 'cli::subcommands::mpool_cmd::tests::message_filtering_from' (42123) panicked at .../cli/subcommands/mpool_cmd.rs:364:69:
called `Result::unwrap()` on an `Err` value: Key already exists
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::key_management::wallet::tests::list_addr
Stack Traces | 0.008s run time
thread 'key_management::wallet::tests::list_addr' (42666) panicked at src/key_management/wallet.rs:418:61:
called `Result::unwrap()` on an `Err` value: KeyExists
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::message_pool::msgpool::selection::test_selection::test_optimal_msg_selection2
Stack Traces | 0.008s run time
thread 'message_pool::msgpool::selection::test_selection::test_optimal_msg_selection2' (45189) panicked at .../message_pool/msgpool/selection.rs:1483:26:
called `Result::unwrap()` on an `Err` value: GasPriceTooLow
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::cli::subcommands::mpool_cmd::tests::message_filtering_none
Stack Traces | 0.009s run time
thread 'cli::subcommands::mpool_cmd::tests::message_filtering_none' (42126) panicked at .../cli/subcommands/mpool_cmd.rs:296:69:
called `Result::unwrap()` on an `Err` value: Key already exists
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::message_pool::msgpool::tests::test_async_message_pool
Stack Traces | 0.009s run time
thread 'message_pool::msgpool::tests::test_async_message_pool' (45211) panicked at .../message_pool/msgpool/mod.rs:433:69:
called `Result::unwrap()` on an `Err` value: Key already exists
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::db::blockstore_with_write_buffer::tests::test_buffer_flush
Stack Traces | 0.01s run time
thread 'db::blockstore_with_write_buffer::tests::test_buffer_flush' (42213) panicked at src/db/blockstore_with_write_buffer.rs:101:17:
assertion failed: mem_db.has(k).unwrap()
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::db::blockstore_with_read_cache::tests::test_blockstore_read_cache
Stack Traces | 0.011s run time
thread 'db::blockstore_with_read_cache::tests::test_blockstore_read_cache' (42210) panicked at src/db/blockstore_with_read_cache.rs:146:13:
assertion `left == right` failed
  left: 1
 right: 2
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::key_management::wallet::tests::import_export
Stack Traces | 0.011s run time
thread 'key_management::wallet::tests::import_export' (42662) panicked at src/key_management/wallet.rs:393:54:
called `Result::unwrap_err()` on an `Ok` value: KeyInfo { key_type: Secp256k1, private_key: [84, 178, 162, 30, 10, 40, 180, 42, 153, 219, 72, 188, 158, 55, 109, 226, 155, 125, 95, 82, 34, 238, 34, 251, 14, 164, 195, 242, 211, 110, 37, 126] }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::message_pool::msgpool::tests::test_msg_chains
Stack Traces | 0.011s run time
thread 'message_pool::msgpool::tests::test_msg_chains' (45227) panicked at .../message_pool/msgpool/mod.rs:786:65:
called `Result::unwrap()` on an `Err` value: Key already exists
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::rpc::methods::eth::filter::store::tests::test_add_filter
Stack Traces | 0.011s run time
thread 'rpc::methods::eth::filter::store::tests::test_add_filter' (45453) panicked at .../eth/filter/store.rs:117:9:
assertion failed: store.add(filter2.clone()).is_ok()
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::message_pool::msgpool::selection::test_selection::test_optimal_msg_selection3
Stack Traces | 0.013s run time
thread 'message_pool::msgpool::selection::test_selection::test_optimal_msg_selection3' (45191) panicked at .../message_pool/msgpool/selection.rs:1571:30:
called `Result::unwrap()` on an `Err` value: GasPriceTooLow
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::message_pool::msgpool::tests::test_per_actor_limit
Stack Traces | 0.013s run time
thread 'message_pool::msgpool::tests::test_per_actor_limit' (45232) panicked at .../message_pool/msgpool/mod.rs:452:69:
called `Result::unwrap()` on an `Err` value: Key already exists
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::rpc::methods::eth::filter::tests::test_eth_uninstall_filter
Stack Traces | 0.013s run time
thread 'rpc::methods::eth::filter::tests::test_eth_uninstall_filter' (45476) panicked at .../eth/filter/mod.rs:1254:68:
called `Result::unwrap()` on an `Err` value: Adding filter failed: Filter already registered
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::key_management::wallet::tests::contains_key
Stack Traces | 0.015s run time
thread 'key_management::wallet::tests::contains_key' (42640) panicked at src/key_management/wallet.rs:303:9:
assertion failed: !wallet.has_key(&address)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::rpc::methods::wallet::tests::wallet_delete_default_key
Stack Traces | 0.018s run time
thread 'rpc::methods::wallet::tests::wallet_delete_default_key' (45879) panicked at .../rpc/methods/wallet.rs:365:53:
called `Result::unwrap()` on an `Err` value: KeyExists
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::rpc::methods::wallet::tests::wallet_delete_non_existent_key
Stack Traces | 0.02s run time
thread 'rpc::methods::wallet::tests::wallet_delete_non_existent_key' (45883) panicked at .../rpc/methods/wallet.rs:354:9:
assertion failed: crate::key_management::remove_key(&key2.address, &mut keystore).is_err()
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::cli::subcommands::mpool_cmd::tests::message_filtering_to
Stack Traces | 0.021s run time
thread 'cli::subcommands::mpool_cmd::tests::message_filtering_to' (42127) panicked at .../cli/subcommands/mpool_cmd.rs:397:69:
called `Result::unwrap()` on an `Err` value: Key already exists
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::key_management::wallet::tests::delegated_verify
Stack Traces | 0.021s run time
thread 'key_management::wallet::tests::delegated_verify' (42652) panicked at src/key_management/wallet.rs:541:9:
assertion failed: sig.verify(&msg, &invalid_addr).is_err()
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::key_management::wallet::tests::bls_verify_test
Stack Traces | 0.022s run time
thread 'key_management::wallet::tests::bls_verify_test' (42622) panicked at src/key_management/wallet.rs:521:9:
assertion failed: sig.verify(&msg, &invalid_addr).is_err()
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::cli::subcommands::mpool_cmd::tests::message_filtering_local
Stack Traces | 0.023s run time
thread 'cli::subcommands::mpool_cmd::tests::message_filtering_local' (42128) panicked at .../cli/subcommands/mpool_cmd.rs:320:69:
called `Result::unwrap()` on an `Err` value: Key already exists
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::shim::crypto::tests::test_verify_delegated_sig_wrong_address
Stack Traces | 0.033s run time
thread 'shim::crypto::tests::test_verify_delegated_sig_wrong_address' (46014) panicked at src/shim/crypto.rs:414:9:
Signature should not verify for wrong address
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::message_pool::msgpool::selection::test_selection::message_selection_trimming_gas
Stack Traces | 0.034s run time
thread 'message_pool::msgpool::selection::test_selection::message_selection_trimming_gas' (45177) panicked at .../message_pool/msgpool/selection.rs:1104:26:
called `Result::unwrap()` on an `Err` value: DuplicateSequence
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::message_pool::msgpool::selection::test_selection::message_selection_priority
Stack Traces | 0.038s run time
thread 'message_pool::msgpool::selection::test_selection::message_selection_priority' (45174) panicked at .../message_pool/msgpool/selection.rs:1359:26:
called `Result::unwrap()` on an `Err` value: DuplicateSequence
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::message_pool::msgpool::selection::test_selection::basic_message_selection
Stack Traces | 0.041s run time
thread 'message_pool::msgpool::selection::test_selection::basic_message_selection' (45173) panicked at .../message_pool/msgpool/selection.rs:953:26:
called `Result::unwrap()` on an `Err` value: DuplicateSequence
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::key_management::keystore::test::test_encrypt_message
Stack Traces | 0.125s run time
thread 'key_management::keystore::test::test_encrypt_message' (42612) panicked at src/key_management/keystore.rs:446:9:
assertion `left != right` failed: Ciphertexts use secure initialization vectors
  left: [223, 16, 25, 4, 95, 105, 217, 174, 177, 229, 40, 175, 48, 137, 224, 134, 234, 203, 136, 134, 99, 218, 189, 240, 13, 252, 34, 243, 42, 84, 178, 162, 30, 10, 40, 180, 42, 153, 219, 72, 188, 158, 55, 109, 226, 155, 125, 95, 82, 34, 238, 34, 251]
 right: [223, 16, 25, 4, 95, 105, 217, 174, 177, 229, 40, 175, 48, 137, 224, 134, 234, 203, 136, 134, 99, 218, 189, 240, 13, 252, 34, 243, 42, 84, 178, 162, 30, 10, 40, 180, 42, 153, 219, 72, 188, 158, 55, 109, 226, 155, 125, 95, 82, 34, 238, 34, 251]
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::libp2p_bitswap::tests::request_manager::tests::request_manager_e2e_test
Stack Traces | 1.54s run time
thread 'libp2p_bitswap::tests::request_manager::tests::request_manager_e2e_test' (42723) panicked at .../libp2p_bitswap/tests/request_manager.rs:68:18:
Should timeout, it does not fail fast (atm) in this case to reduce code complexity.: true
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 30, 2026

Codecov Report

❌ Patch coverage is 80.00000% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 64.10%. Comparing base (ad9abdd) to head (e362ab3).
⚠️ Report is 3 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/tool/subcommands/api_cmd/test_snapshot.rs 80.00% 2 Missing and 2 partials ⚠️
Additional details and impacted files
Files with missing lines Coverage Δ
src/tool/subcommands/api_cmd/test_snapshot.rs 84.07% <80.00%> (-1.55%) ⬇️

... and 7 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ad9abdd...e362ab3. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@LesnyRumcajs LesnyRumcajs added this pull request to the merge queue Apr 30, 2026
Merged via the queue into main with commit a86c6ba Apr 30, 2026
34 checks passed
@LesnyRumcajs LesnyRumcajs deleted the fix-flaky-coverage branch April 30, 2026 12:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants