Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions docs/architecture/concepts/external-key-management.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
---
title: "External Key Management"
description: "An external KMS keeps the keys that protect simplyblock volume encryption material outside the storage cluster, enabling separation of duty, rotation, and audit."
weight: 30220
---

Volume encryption protects data at rest by ciphering every block written to a logical volume. The cipher itself
needs a key; the question of *where that key lives and who controls it* is the responsibility of the key management
layer.

By default, simplyblock manages encryption keys internally. For environments where the team operating the storage
cluster should not also be in possession of the long-lived key material — common in regulated environments and any
deployment that separates storage and security duties — the key-encryption keys can be offloaded to an external Key
Management Service (KMS). Simplyblock supports [Hashicorp Vault](https://www.vaultproject.io/){:target="_blank" rel="noopener"}
and [Openbao](https://openbao.org/){:target="_blank" rel="noopener"} as KMS backends.

## Two-Layer Key Model

When an external KMS is configured, simplyblock applies a two-layer model:

- **Data Encryption Keys (DEKs)** are generated per volume and used to encrypt the blocks of that volume. They are
short-lived in cluster memory and never stored in plaintext at rest.
- **Key Encryption Keys (KEKs)** live in the KMS. The cluster asks the KMS to wrap each DEK on creation and to unwrap
it when the volume is brought online. The KEKs never leave the KMS.

Compromise of the cluster does not compromise the KEKs; compromise of the KMS does not directly expose any specific
volume's data, because DEKs are wrapped against the KEK rather than stored alongside it.

## Authentication and Trust

The KMS authenticates simplyblock components using a client certificate issued by the
`simplyblock-certificate-authority-issuer` ClusterIssuer, which the operator creates as part of its mTLS setup.
Because the KMS depends on this CA, [control-plane mTLS](../../deployments/kubernetes/security.md#transport-layer-security-mtls)
must be configured before an external KMS can be wired up.

Operationally, this means the KMS team and the storage team share only the CA bundle and an agreed-upon DNS name for
the simplyblock client — no static passwords or long-lived tokens are exchanged.

For the setup steps, see [Securing the Control Plane: External KMS](../../deployments/kubernetes/security.md#external-key-management-kms).
14 changes: 5 additions & 9 deletions docs/deployments/kubernetes/k8s-control-plane.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,17 +32,13 @@ helm upgrade --install simplyblock -n simplyblock simplyblock/spdk-csi \
```

!!! important "TLS Encryption"
{{experimental}}
{{ experimental }}

Simplyblock has just added support for TLS encryption for all internal communication. At the moment, it's an
experimental feature only available when installed into OpenShift clusters.
All internal control-plane traffic can be encrypted with TLS. On OpenShift, the cluster's built-in certificate
manager is used out of the box. Mutual TLS (mTLS), where components additionally authenticate each other with
client certificates, is supported only with cert-manager — on any Kubernetes distribution, including OpenShift.

It generally is a good idea to install the operator with TLS support enabled on OpenShift to ensure that all
internal communication is encrypted and secure.

To enable TLS, add the `--set tls.enabled=true` flag to the `helm install` command.

In the future, the support will be extended to further Kubernetes distributions by enabling Cert-Manager support.
See [Securing the Control Plane](security.md#transport-layer-security-mtls) for configuration.

After installation, verify the operator is running:

Expand Down
6 changes: 6 additions & 0 deletions docs/deployments/kubernetes/k8s-storage-plane.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,12 @@ been registered, it has no storage nodes yet. Those are added in the next step.
as NVMe-oF transport security, backup configuration, capacity thresholds, and more, are available at
[Cluster Deployment Options](../cluster-deployment-options.md).

!!! tip "External KMS"
If volumes in this cluster should offload their encryption keys to an external KMS, set
`spec.hashicorpVaultSettings.base_url` on the `StorageCluster` now. The setting can also be added later, but
configuring it up front means encrypted volumes use the external KMS from day one. See
[Securing the Control Plane: External KMS](security.md#external-key-management-kms).

## Add Storage Nodes

Now, Kubernetes worker nodes will be transformed into simplyblock storage nodes. To initiate the process, a
Expand Down
211 changes: 211 additions & 0 deletions docs/deployments/kubernetes/security.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,211 @@
---
title: "Securing the Control Plane"
description: "Configure mTLS for simplyblock control plane communication and offload at-rest encryption keys to an external KMS (Hashicorp Vault or Openbao)."
weight: 30050
---

This page covers two security features for simplyblock on Kubernetes: transport-layer encryption and mutual
authentication for the control plane (mTLS), and offloading volume encryption keys to an external Key Management
Service (KMS).

mTLS must be configured before an external KMS can be wired up: the KMS authenticates simplyblock components using a
certificate issued by the operator-managed certificate authority, which is only provisioned when mTLS is active.

## Transport Layer Security (mTLS)

{{ experimental }}

Internal control-plane traffic between the webappapi, the operator, and the storage-node handlers can be encrypted
with TLS. When mutual TLS is additionally enabled, every component must present a valid client certificate, which
means components authenticate each other rather than relying on network position alone.

**mTLS is only supported with the cert-manager provider.** On OpenShift, the cluster's built-in certificate manager
provides one-way TLS (server certificates) but does not issue the client certificates required for mutual
authentication; running mTLS on OpenShift therefore also requires installing
[cert-manager](https://cert-manager.io/){:target="_blank" rel="noopener"} and switching the provider over.

### Prerequisites

- cert-manager installed in the cluster.
- A `ClusterIssuer` (or namespaced `Issuer`) that cert-manager can use to mint certificates. Most installations point
this at an internal corporate CA or at the cluster-local self-signed issuer; any issuer that simplyblock components
can trust is acceptable.

### Enabling mTLS

mTLS is configured at Helm install time by setting four values on the operator chart:

```yaml title="Helm values for mTLS"
tls:
enabled: true
mutual_enabled: true
provider: cert-manager
cert-manager:
issuer: my-cluster-issuer
```

Apply the values during the operator install (see [Install Simplyblock Operator](k8s-control-plane.md)):

```bash title="Install the operator with mTLS"
helm upgrade --install simplyblock -n simplyblock simplyblock/spdk-csi \
--create-namespace \
--set operator.enabled=true \
--set tls.enabled=true \
--set tls.mutual_enabled=true \
--set tls.provider=cert-manager \
--set tls.cert-manager.issuer=my-cluster-issuer
```

Replace `my-cluster-issuer` with the name of the `ClusterIssuer` the operator should use to obtain its certificates.

### What the Operator Provisions

When mTLS is enabled, the operator creates a dedicated `ClusterIssuer` named
`simplyblock-certificate-authority-issuer` and issues all internal component certificates from it. The same issuer can
be used to mint certificates for other workloads that need to talk to simplyblock — most notably an external KMS, as
described in the next section.

!!! note "OpenShift"
On OpenShift, setting `tls.enabled=true` with the default `tls.provider=openshift` activates one-way TLS using
OpenShift-managed certificates. Mutual TLS is **not** available with this provider — `tls.mutual_enabled=true`
requires `tls.provider=cert-manager` regardless of the underlying Kubernetes distribution.

## External Key Management (KMS)

{{ experimental }}

By default, simplyblock manages volume encryption keys internally. For environments that require stricter key handling
— separation of duty between storage administrators and key custodians, regular rotation, or audit trails — the
cluster can be configured to keep the key-encryption material in an external KMS. Both
[Hashicorp Vault](https://www.vaultproject.io/){:target="_blank" rel="noopener"} and
[Openbao](https://openbao.org/){:target="_blank" rel="noopener"} are supported; the configuration is identical for
either.

### Prerequisites

- [mTLS configured](#transport-layer-security-mtls) — required, because the vault is authenticated to the cluster via
a certificate issued by the operator's `simplyblock-certificate-authority-issuer`.
- A Vault or Openbao instance reachable from the simplyblock namespace. The instance must be initialized and unsealed
before configuring authentication.

### Step 1 — Issue a TLS certificate for the vault

Create a cert-manager `Certificate` resource that uses the operator-managed issuer. The resulting Secret holds the
TLS material the vault serves to clients, and is trusted by the simplyblock components because it chains to the same
CA.

```yaml title="vault-tls.yaml"
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: vault-tls
namespace: vault
spec:
secretName: vault-tls
issuerRef:
name: simplyblock-certificate-authority-issuer
kind: ClusterIssuer
commonName: vault
dnsNames:
- vault
- vault.vault
- vault.vault.svc
- vault.vault.svc.cluster.local
```

Mount the resulting `vault-tls` Secret into the vault deployment as its serving certificate. Mount the issuer's CA
bundle (typically `ca.crt`) at a path the vault can read — the example below assumes `/vault/tls/ca.crt` for Vault
and `/bao/tls/ca.crt` for Openbao.

### Step 2 — Deploy the vault

Install Vault or Openbao using the upstream Helm chart and expose it inside the cluster. For the rest of this guide
the in-cluster Service is assumed to be `vault.vault:8200`; adjust the URL to match the actual deployment.

### Step 3 — Configure auth, policy, and secret engines

Configure the vault with a policy that grants simplyblock access to the `transit` and `kv` backends, enable
certificate authentication bound to the simplyblock CA, and enable the required secret engines. The script below
works for both Vault (`vault`) and Openbao (`bao`) — assign the appropriate CLI to `$CLI`.

```bash title="Configure the vault for simplyblock"
CLI=vault # or: CLI=bao

# Policy granting access to the transit and kv backends
$CLI policy write webappapi-policy - <<EOF
path "transit/keys/*" {
capabilities = ["create", "update", "read", "delete"]
}

path "transit/datakey/plaintext/*" {
capabilities = ["create", "update"]
}

path "transit/datakey/wrapped/*" {
capabilities = ["create", "update"]
}

path "transit/decrypt/*" {
capabilities = ["create", "update"]
}

path "kv/*" {
capabilities = ["create", "read", "update", "delete"]
}
EOF

# Certificate authentication, bound to the simplyblock cluster CA
$CLI auth enable cert
$CLI write auth/cert/certs/webappapi \
certificate=@/${CLI}/tls/ca.crt \
allowed_dns_sans="simplyblock-webappapi" \
token_policies=webappapi-policy \
token_ttl=10m \
token_max_ttl=30m

# Secret engines used by simplyblock
$CLI secrets enable transit
$CLI secrets enable -version=1 kv
```

- The **policy** gives simplyblock the minimum capabilities it needs: managing keys and performing envelope
encryption on the `transit` backend, and storing per-volume key material on the `kv` backend.
- The **cert auth** role only accepts clients that present a certificate chaining to the simplyblock CA *and* whose
DNS SAN is `simplyblock-webappapi`. Tokens are short-lived (10 min, 30 min maximum) so a compromised token expires
quickly.
- **Transit** is used for wrapping data-encryption keys; **kv** version 1 is used as the per-volume metadata store.

### Step 4 — Point the StorageCluster at the vault

Set `spec.hashicorpVaultSettings.base_url` on the `StorageCluster` resource:

```yaml title="StorageCluster with external KMS"
apiVersion: storage.simplyblock.io/v1alpha1
kind: StorageCluster
metadata:
name: simplyblock-cluster
namespace: simplyblock
spec:
clusterName: production
mgmtIfname: eth0
fabricType: tcp
haType: ha
hashicorpVaultSettings:
base_url: "https://vault.vault:8200/"
```

The operator picks the setting up on the next reconcile. From that point on, volume encryption keys for this cluster
are wrapped against the vault's transit backend instead of being held inside the cluster.

### Verification

Once configured, check the operator and webappapi pod logs for vault connection messages and watch the cluster
status:

```bash title="Verify the KMS connection"
kubectl get storagecluster -n simplyblock
kubectl logs -n simplyblock deploy/simplyblock-operator
```

Creating a new encrypted volume after the vault is wired up exercises the path end-to-end; the volume's encryption key
material is then stored in the vault rather than alongside the cluster.
69 changes: 67 additions & 2 deletions docs/deployments/kubernetes/volume-encryption.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,72 @@
---
title: "Volume Encryption"
description: "Configure encrypted Simplyblock volumes on Kubernetes using CSI, including key generation, Kubernetes Secret setup, and encrypted StorageClass/PVC examples."
description: "Encrypt simplyblock logical volumes at rest using the AES_XTS crypto bdev. Keys are managed by the cluster and can optionally be offloaded to an external KMS."
weight: 40000
---

{% include 'kubernetes-csi-encryption.md' %}
Simplyblock supports encryption of logical volumes at rest, ensuring that sensitive data remains protected across the
distributed storage cluster. Internally, simplyblock uses the industry-proven
[crypto bdev](https://spdk.io/doc/bdev.html){:target="_blank" rel="noopener"} provided by SPDK, with an AES_XTS
variable-length block cipher.

Encryption is enabled per StorageClass and applies to every volume provisioned from it.

!!! warning
Encryption must be specified at the time of volume creation. Existing logical volumes cannot be retroactively
encrypted.

## Enabling Encryption on a StorageClass

To enable encryption, set the `encryption` parameter on the StorageClass to `"True"`. Every PersistentVolumeClaim
that references the StorageClass is then provisioned as an encrypted volume.

```yaml title="Encrypted StorageClass"
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: my-encrypted-volumes
provisioner: csi.simplyblock.io
parameters:
encryption: "True"
# ... other parameters
reclaimPolicy: Delete
volumeBindingMode: Immediate
allowVolumeExpansion: true
```

A PersistentVolumeClaim using this StorageClass is then encrypted automatically:

```yaml title="Encrypted PersistentVolumeClaim"
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: my-encrypted-volume-claim
spec:
storageClassName: my-encrypted-volumes
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 200Gi
```

## Key Management

Encryption keys are generated and managed by the simplyblock cluster. No user-supplied keys, per-PVC Secrets, or
annotations are required to encrypt a volume.

!!! warning "Migration from earlier versions"
Previous releases required a user-managed Kubernetes Secret (containing `crypto_key1` and `crypto_key2`) to be
referenced from each PVC via the `simplyblock.io/secret-name` (or legacy `simplybk/secret-name`) annotation.
That mechanism is **no longer used** for new volumes. Existing encrypted volumes provisioned with user-supplied
keys continue to work, but new PVCs should not set those annotations.

## Hardening Key Storage with an External KMS

For environments that require stricter handling of key material — separation of duty between storage and key
custodians, regular rotation, or audit trails — the cluster can be configured to keep encryption keys in an external
Hashicorp Vault or Openbao instance. The setup is configured once per `StorageCluster` and applies to every encrypted
volume in that cluster.

See [Securing the Control Plane: External KMS](security.md#external-key-management-kms) for the full setup, or
[External Key Management](../../architecture/concepts/external-key-management.md) for the architectural background.
12 changes: 12 additions & 0 deletions docs/reference/kubernetes/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,18 @@ Additional, uncommonly configured CSI driver parameters:
| `benchmarks` | Enables benchmark resources when set to non-zero. | `0` |
| `autoClusterActivate` | Enables automatic cluster activation when sufficient nodes are up. | `false` |

## Operator & Control Plane Parameters

| Parameter | Description | Default |
|---------------------------------|--------------------------------------------------------------------------------------------------------------|-------------|
| `operator.enabled` | Enables the simplyblock operator that manages StorageCluster, StorageNode, Pool, and Lvol CRDs. | `false` |
| `tls.enabled` | Enables TLS encryption for all control-plane internal communication. | `false` |
| `tls.mutual_enabled` | Enables mutual TLS (mTLS) authentication between control-plane components. Requires `tls.enabled=true` and `tls.provider=cert-manager`. | `false` |
| `tls.provider` | TLS certificate provider. `cert-manager` for generic Kubernetes, `openshift` for OpenShift-managed certs. | `openshift` |
| `tls.cert-manager.issuer` | Name of the cert-manager `ClusterIssuer` to use. **Required when `tls.provider=cert-manager`**. | `<empty>` |

For details, see [Securing the Control Plane](../../deployments/kubernetes/security.md).

## Storage Node Parameters

| Parameter | Description | Default |
Expand Down
1 change: 1 addition & 0 deletions docs/reference/operator.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ spec:
| `warningThreshold.provisionedCapacity` | int | Provisioned capacity warning threshold (percent). |
| `criticalThreshold.provisionedCapacity` | int | Provisioned capacity critical threshold (percent). |
| `action` | string | Lifecycle action: `activate` or `expand`. |
| `hashicorpVaultSettings.base_url` | string | Base URL of an external Hashicorp Vault or Openbao instance used to manage volume encryption keys (e.g., `https://vault.vault:8200/`). See [Securing the Control Plane: External KMS](../deployments/kubernetes/security.md#external-key-management-kms). |
| `backup.credentialsSecretRef.name` | string | Name of the Secret (in the same namespace) holding `access_key_id` and `secret_access_key`. **Required when `backup` is set**. |
| `backup.localEndpoint` | string | S3-compatible endpoint URL for backup storage. |
| `backup.snapshotBackups` | bool | Enable snapshot-based backups. |
Expand Down
Loading
Loading