Skip to content

test(csharp): add long-running (hours-long) query E2E tests scheduled nightly #570

Open

Description

@eric-wang-1990

Problem

We have no CI coverage for genuinely long-running (hours-long) queries. The doc calls for such tests scheduled nightly, not per-commit.

Current state

  • The repo has exactly one scheduled workflow: .github/workflows/reyden-rest-nightly.yml (cron '0 2 * * *', lines 17-21). Everything else is per-PR/push/dispatch.
  • The longest test today, StatusPollerKeepsQueryAlive (csharp/test/E2E/StatementTests.cs:949-975), is a real 30-minute test (Task.Delay(30 min) over RANGE(30000000)) — but it is excluded from every CI run: Skip.If(TestConfiguration.IsCITesting, …) (line 955), and both the nightly (reyden-rest-nightly.yml:164) and per-PR e2e (e2e-tests.yml:232) set isCITesting: true. So no long query ever executes in CI.
  • There is no test-category convention — zero [Trait]/[Category] attributes in csharp/test. CI runs dotnet test with no --filter (ci/scripts/csharp_test_databricks_e2e.sh).
  • The nightly failure-classification/dashboard machinery (ci: classify Reyden nightly E2E failures by root cause #512/feat(ci): split Reyden gap signatures + per-error dashboard detail #513) is staged on branch ci/reyden-rest-nightly, not yet on main — a new long-running tier should coordinate with it.

Proposed scope

  1. Introduce a long-running test category — either an xUnit [Trait("Category","LongRunning")] driven by dotnet test --filter, or a dedicated config flag analogous to IsCITesting (note: the existing Skip.If(IsCITesting) on StatusPollerKeepsQueryAlive must be inverted/relaxed so it can run in the long-running tier).
  2. Add hours-long query tests targeting existing large data (e.g. main.tpcds_sf100_delta.*, already used by DriverResilienceTests.cs:98-135; larger scale factors available).
  3. Wire them into reyden-rest-nightly.yml (or a sibling scheduled workflow).

Caveats to design around

  • GitHub-hosted runners have a 6-hour job limit; reyden-rest-nightly uses concurrency … cancel-in-progress: false.
  • dotnet test currently has no timeout override — hours-long tests need explicit per-test timeout handling.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    .NETPull requests that update .NET codeenhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions