Skip to content

feat(EN-1353): default volume deletion protection to on (opt-out)#1480

Merged
paul-nicolas merged 3 commits into
release/v3.0from
EN-1353-default-deletion-protection
Jul 2, 2026
Merged

feat(EN-1353): default volume deletion protection to on (opt-out)#1480
paul-nicolas merged 3 commits into
release/v3.0from
EN-1353-default-deletion-protection

Conversation

@paul-nicolas

@paul-nicolas paul-nicolas commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Summary

Flips the ledger PVC/PV deletion protection (originally shipped opt-in in formancehq/ledger-v3-poc#567) to default-ON, explicit opt-out — the reverse of today. A freshly deployed ledger is now protected without any extra configuration; both layers must be explicitly turned off to opt out.

Jira: https://formance-team.atlassian.net/browse/EN-1353

Changes

  • Cluster layerhelm/operator/values.yaml: pvcProtection.enabled false → true. The cluster-scoped ValidatingAdmissionPolicy installs by default.
  • Per-CR layerspec.persistence.deletionProtection: bool (+kubebuilder:default=false) → *bool (+kubebuilder:default=true). The pointer is what makes "default true, but an explicit false still opts out" correct — a plain bool + omitempty can't distinguish unset from false.
    • New PersistenceSpec.DeletionProtectionEnabled() (nil → true) so in-process construction and the runtime resolve the default consistently; all consumers route through it. hash.go zeroes the field to nil.
  • Regenerated CRDs (config/crd/bases + helm/crds/templates, both now default: true) and deepcopy via controller-gen.
  • Rewrote the operator README and values.yaml comments from opt-in to opt-out.

Old-cluster safety (K8s < 1.30)

ValidatingAdmissionPolicy is GA only in Kubernetes >= 1.30. Defaulting pvcProtection.enabled to true would otherwise break helm install/upgrade on older clusters ("no matches for kind ValidatingAdmissionPolicy"). Fixed by:

  • Gating the VAP/VAPBinding template on .Capabilities.APIVersions.Has "admissionregistration.k8s.io/v1/ValidatingAdmissionPolicy" — an old cluster silently skips the objects and the default install succeeds. (helm template offline: pass --api-versions admissionregistration.k8s.io/v1/ValidatingAdmissionPolicy to force-render.)
  • A NOTES.txt warning when protection is enabled but the API is absent, so the degrade-to-unprotected is visible (upgrade to >=1.30, or set enabled=false).
  • The runtime probe already reports "not installed" on such clusters (the unregistered resource answers the GET with a 404 that IsNotFound matches), so reconcile does not error on every pass — clarified in a comment.

Verified with helm template: 0 VAP objects rendered without the capability, 4 with it.

Multi-release caveat worth reviewing

Making the Helm layer default-ON interacts with the singleton constraint from #567: the cluster VAP objects have fixed, release-independent names, so a second operator release in the same cluster now fails its helm install/upgrade by default (ownership conflict). In multi-release clusters, set pvcProtection.enabled=false on all but one release. Documented in both values.yaml and the README.

The other consequence — legitimate kubectl delete pvc / GitOps teardown now needs the formance.com/allow-deletion annotation (or an exempt SA) — is the intended safety trade-off.

Testing

  • go build ./..., go test ./..., golangci-lint run ./... all green (operator module); helm lint passes.
  • controller-gen regeneration is stable (no drift).
  • Helm render verified for both K8s >= 1.30 and < 1.30.
  • Existing chainsaw e2e (e2e/tests/pvc-deletion-protection) still valid: it sets deletionProtection: true explicitly and exercises the toggle-to-false opt-out path.

Flip both opt-in layers of ledger PVC/PV deletion protection to default-ON,
so a freshly deployed ledger is protected without extra configuration and must
be explicitly opted out instead of opted in.

- helm pvcProtection.enabled: false -> true (installs the cluster-scoped
  ValidatingAdmissionPolicy by default).
- spec.persistence.deletionProtection: bool(default false) -> *bool(default
  true). Pointer so an omitted field defaults to true while an explicit false
  still opts out. Add PersistenceSpec.DeletionProtectionEnabled() (nil -> true)
  and route all consumers through it; hash.go zeroes it to nil.
- Regenerate CRDs (config + chart) and deepcopy; update README and values docs
  from opt-in to opt-out, including the multi-release singleton caveat (all but
  one release must set pvcProtection.enabled=false).

Reverse of formancehq/ledger-v3-poc#567.
@coderabbitai

coderabbitai Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (1)
  • main

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: ccbfac95-4d9b-452a-b441-c818cec1bdb4

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch EN-1353-default-deletion-protection

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@paul-nicolas paul-nicolas marked this pull request as ready for review July 2, 2026 13:25
@NumaryBot

NumaryBot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

💬 Comments — automated review

The PR correctly implements opt-out semantics for volume deletion protection across API types, controller helpers, CRD regeneration, hash exclusion, and Helm gating. All modified lines are covered by tests. The only finding is a documentation gap in the Helm NOTES.txt: the opt-out guidance tells operators to set the Helm-level flag to suppress the DeletionProtectionInactive warning, but omits the necessary per-CR opt-out (spec.persistence.deletionProtection=false), which is the actual mechanism the controller reads. No code correctness issues were identified.

…atingAdmissionPolicy

Defaulting pvcProtection.enabled to true broke helm install/upgrade on
Kubernetes < 1.30, where the ValidatingAdmissionPolicy kind does not exist
(the API server rejects the rendered objects with 'no matches for kind').

- Gate the VAP/VAPBinding template on
  .Capabilities.APIVersions.Has "admissionregistration.k8s.io/v1/ValidatingAdmissionPolicy"
  so an old cluster silently skips the objects and the default install succeeds.
- Add NOTES.txt warning when protection is enabled but the API is absent, so the
  degrade-to-unprotected is visible (upgrade to >=1.30 or set enabled=false).
- Clarify deletionProtectionPolicyInstalled: the probe's IsNotFound branch also
  covers old clusters (unregistered resource answers GET with 404), so reconcile
  reports not-installed instead of erroring on every pass.
- Update README/values docs: old clusters auto-skip (install succeeds) rather
  than fail; note --api-versions for offline helm template.
@codecov

codecov Bot commented Jul 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.95%. Comparing base (71e591e) to head (acf16f0).
⚠️ Report is 3 commits behind head on release/v3.0.

Additional details and impacted files
@@               Coverage Diff                @@
##           release/v3.0    #1480      +/-   ##
================================================
- Coverage         74.01%   73.95%   -0.06%     
================================================
  Files               388      388              
  Lines             39676    39734      +58     
================================================
+ Hits              29366    29387      +21     
- Misses             7797     7831      +34     
- Partials           2513     2516       +3     
Flag Coverage Δ
e2e 73.95% <ø> (-0.06%) ⬇️
scenario 73.95% <ø> (-0.06%) ⬇️
unit 73.95% <ø> (-0.06%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

- deletion_protection_test.go: probe reports not-installed on a bare 404 from an
  unregistered ValidatingAdmissionPolicy API (K8s < 1.30) via IsNotFound's
  code-404 fallback, and propagates any other API error instead of swallowing it.
- api/v1alpha1 persistence_test.go: DeletionProtectionEnabled() resolves nil to
  true (protected), pinning the opt-out default; explicit true/false honored.
- justfile test-helm-render: cluster-free check that the VAP is skipped when the
  ValidatingAdmissionPolicy capability is absent and rendered when present,
  guarding the default pvcProtection.enabled=true against breaking old-cluster installs.

@NumaryBot NumaryBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NumaryBot posted 1 new inline finding.

Summary: #1480 (comment)

Comment thread misc/operator/helm/operator/templates/NOTES.txt
@paul-nicolas paul-nicolas merged commit aa373d5 into release/v3.0 Jul 2, 2026
10 checks passed
@paul-nicolas paul-nicolas deleted the EN-1353-default-deletion-protection branch July 2, 2026 13:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants