Skip to content

ci: split test run into per-suite steps - #3533

Merged
TheTisiboth merged 1 commit into
masterfrom
ci/split-test-steps
Oct 29, 2026
Merged

TheTisiboth merged 1 commit into
masterfrom
ci/split-test-steps

Conversation

@TheTisiboth

@TheTisiboth TheTisiboth commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #2720

Topic and Scope

Splits the single "Run the tests" step in both test jobs into three steps (unit, integration, migration), so each suite shows its own duration in the Actions UI. This gives a baseline for the upcoming CI speed work (umbrella issue 2719).

  • Test steps run as long as fixture creation succeeded, even if an earlier suite failed. This matches the previous behaviour, where test.sh ran every suite and only failed at the end.
  • Each run-locally.py call writes coverage to its own run-<timestamp> folder. The copy step now collects coverage.*.xml from all of them instead of only latest/.
  • Dropped the unused test_result env var.

Concerns/issues

  • --install-composer-deps is needed on each step: test.sh runs rsync --delete into the app folder, which wipes vendor/. That adds some overhead, which these timings will measure.

Formal requirements

  • I did check that the app can still be opened and does not throw any browser logs
  • I created tests for newly added PHP code (check this if no PHP changes were made)
  • I updated the OpenAPI specs and added an entry to the API changelog (check if API was not modified)
  • I notified the matrix channel if I introduced an API change

AI disclosure: prepared with Claude Code (claude-opus-5-5); reviewed by the author.

🤖 Generated with Claude Code

Run unit, integration and migration tests as separate steps to expose per-suite timings.
Collect coverage XML from all runs since each call creates its own run folder.

Assisted-by: Claude Code:claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Léo Jan <leo.jan@doctolib.com>
@nextcloud-command nextcloud-command added the AI assisted This PR contains AI-assisted commits label Oct 29, 2026
@TheTisiboth
TheTisiboth marked this pull request as ready for review October 29, 2026 18:17
@TheTisiboth
TheTisiboth enabled auto-merge (squash) October 29, 2026 18:17
@github-actions

Copy link
Copy Markdown
Contributor

Test Results

   12 files    600 suites   3m 11s ⏱️
  605 tests   597 ✅  8 💤 0 ❌
2 420 runs  2 387 ✅ 33 💤 0 ❌

Results for commit 2682c99.

@coderabbitai

coderabbitai Bot commented Oct 29, 2026

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 0d152dcc-56ad-4bbe-b30c-ea818d5e4cc5
Commits

Reviewing files that changed from the base of the PR and between 56213cf and 2682c99.

Files selected for processing: 1
  • .github/workflows/tests.yml

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 3 remain after this review.

Walkthrough

Both test jobs now run unit, integration, and migration tests in separate steps. Each test step requires successful fixture creation and a workflow that is not cancelled. Coverage staging now copies XML reports from all run-* directories.

Priority: Low

Merge Risk: Minimal · up to 2682c

The suites run separately and their coverage reports are collected. No merge-blocking risk is evident.

Pre-merge checks: Passed 5
Check name Status Explanation
Title check Passed The title clearly and concisely describes the main CI change: splitting the test run into separate suite steps.
Description check Passed The description includes the required Topic and Scope, Concerns/issues, and Formal requirements sections. It explains the workflow changes, test behavior, coverage handling, and dependency installatio…
Linked Issues check Passed Issue #2720 requests multiple test sections so that timings are visible. The PR separates both test jobs into unit, integration, and migration steps. The supplied change summary also states that fixtu…
Out of Scope Changes check Passed The coverage collection change, repeated Composer installation, fixture-step gating, and removal of the unused test_result variable support the test-step split or preserve existing behavior. No unre…
Docstring Coverage Passed No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Full details: Description check

Explanation

The description includes the required Topic and Scope, Concerns/issues, and Formal requirements sections. It explains the workflow changes, test behavior, coverage handling, and dependency installation details.

Full details: Linked Issues check

Explanation

Issue #2720 requests multiple test sections so that timings are visible. The PR separates both test jobs into unit, integration, and migration steps. The supplied change summary also states that fixture gating, continued suite execution, Composer installation, and coverage collection preserve the prior test behavior.

Full details: Out of Scope Changes check

Explanation

The coverage collection change, repeated Composer installation, fixture-step gating, and removal of the unused test_result variable support the test-step split or preserve existing behavior. No unrelated product or API changes are reported.

Full details: Docstring Coverage

Explanation

No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 0 functions across 0 files. (1 skipped: 1 unsupported.)

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@TheTisiboth TheTisiboth mentioned this pull request Oct 29, 2026
4 tasks done

@christianlupus christianlupus left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

@TheTisiboth
TheTisiboth merged commit 665c61e into master Oct 29, 2026
25 checks passed
@TheTisiboth
TheTisiboth deleted the ci/split-test-steps branch October 29, 2026 20:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI assisted This PR contains AI-assisted commits

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Split GH action for Tests into multiple sections

3 participants